Set Up Nx Cloud on Kubernetes

A lot of organizations deploy Nx Cloud to Kubernetes.

This guide references the nx-cloud-helm repo which contains:

Nx Cloud Helm Chart
Instructions on how to install Nx Cloud using Helm
Instructions on how to install Nx Cloud using kubectl. See here.

Deployments on AWS/EKS

If you're deploying on EKS, check out our AWS Guide. Otherwise, continue reading below.

Installing Using Helm

Steps:

Deploy MongoDB Kubernetes Operator
- using helm: https://github.com/mongodb/helm-charts
- using kubectl: https://github.com/mongodb/mongodb-kubernetes-operator
Create a mongodb replica set
Create a secret
Install Nx Cloud using helm

Step 1: Deploy MongoDB Kubernetes Operator

If you are using a hosted MongoDB installation (e.g., Mongo Atlas or CosmosSB, or you are running one yourself), you can skip steps 1 and 2.

> helm repo add mongodb https://mongodb.github.io/helm-charts
> helm install community-operator mongodb/community-operator

Step 2: Deploy a MongoDB replica set

> kubectl apply -f examples/mongodb.yml

This will create a secret. You can get the value of the secret as follows:

> kubectl get secret cloud-mongodb-nrwl-api-admin-user -o go-template='{{range $k,$v := .data}}{{"### "}}{{$k}}{{"n"}}{{$v|base64decode}}{{"nn"}}{{end}}'

You might need to wait a bit for the Pods to be created before this secret will be available.

The result should look like this: mongodb+srv://admin-user:DB_PASSWORD@cloud-mongodb-svc.default.svc.cluster.local/nrwl-api?replicaSet=cloud-mongodb&ssl=false .

Extract the connection string and paste it into your secret.yml.

Step 3: Create a secret

Create a secret by running kubectl apply -f examples/secret.yml

Step 4: Install Nx Cloud using helm

> helm repo add nx-cloud https://nrwl.github.io/nx-cloud-helm
> helm install nx-cloud nx-cloud/nx-cloud --values=overrides.yml

examples/overrides contains the min overrides files. You need to provision:

The image tag you want to install
nxCloudAppURL which is the url used to access ingress from CI and dev machines ( e.g., https://nx-cloud.myorg.com).
secret/name the name of the secret you created in Step 3.
secret/nxCloudMongoServerEndpoint, the name of the key from the secret. 5secret/adminPassword, the name of the key from the secret.

If you only applied the secret from Step 3, the only thing you will need to change is nxCloudAppURL.

Cloud Containers

The installation will create the following:

nx-cloud-frontend (deployment)
nx-cloud-api (deployment)
nx-cloud-nx-api (deployment)
nx-cloud-file-server (deployment)
nx-cloud-aggregator (cron job)

Ingress, IP, Certificates

You can configure Ingress. For instance, the following will see the ingress class to 'gce', the global static ip name to 'nx-cloud-ip', and will set a global Google managed certificate.

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

ingress:
  class: 'gce'
  globalStaticIpName: 'nx-cloud-ip'
  managedCertificates: 'cloud-cert'

secret:
  name: 'cloud'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  adminPassword: 'ADMIN_PASSWORD'

This configuration will look different for you. You will have a different global static ip and your cert name will also be different. If you are interested in creating the two using GKE, check out the following links:

If you aren't using GKE, ingress.class will also be different. For example, see our example config for AWS or check out the AWS Load Balancer set-up section here for AWS set-up instructions.

If you need to have a detailed Ingress configuration, you can tell the package to skip defining ingress:

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

ingress:
  skip: true

⤵️ and then define it yourself

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nx-cloud-ingress
  annotations:

  labels:
    app: nx-cloud
spec:
  rules:
    - http:
        paths:
          # define the next /file section only if you use the built-in file server
          - path: /file
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-file-server-service
                port:
                  number: 5000
          - path: /nx-cloud
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nx-api-service
                port:
                  number: 4203
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nrwl-api-service
                port:
                  number: 4000
          - path: /graphql
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nrwl-api-service
                port:
                  number: 4000
          - path: /auth
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nrwl-api-service
                port:
                  number: 4000
          - path: /download
            pathType: Prefix
            backend:
              service:
                name: nx-cloud-nrwl-api-service
                port:
                  number: 4000
                             - path: /download
          - pathType: Prefix
            backend:
               service:
                  name: nx-cloud-frontend-service
                  port:
                     number: 8080

Step 5: Connect Your Workspace

Run NX_CLOUD_API=https://nx-cloud.myorg.com nx g @nrwl/nx-cloud:init. Click on the link to connect the workspace to your admin account.

Variations

External File Storage

If you use AWS or Azure, you can configure Nx Cloud to store cached artifacts on S3 or Azure Blob. In this case, you won't need the PVC or the file-server container. S3 and Azure Blob also tend to be faster.

For S3 buckets, see the AWS Guide

For Azure:

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

azure:
  enabled: true
  container: 'nx-cloud'

secret:
  name: 'cloudsecret'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  adminPassword: 'ADMIN_PASSWORD'
  azureConnectionString: 'AZURE_CONNECTION_STRING'

Note that the secret for setting up Azure must contain AZURE_CONNECTION_STRING.

GitHub Auth

To use GitHub for user authentication, you can use the following configuration:

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

github:
  auth:
    enabled: true

secret:
  name: 'cloudsecret'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  githubAuthClientId: 'GITHUB_AUTH_CLIENT_ID'
  githubAuthClientSecret: 'GITHUB_AUTH_CLIENT_SECRET'

Note that the secret must contain GITHUB_AUTH_CLIENT_ID and GITHUB_AUTH_CLIENT_SECRET. Read here on how to get those values.

GitHub Integration

To enable the GitHub PR integration, you can use the following configuration:

image:
  tag: 'latest'

nxCloudAppURL: 'https://nx-cloud.myorg.com'

github:
  pr:
    enabled: true
    # apiUrl: '' uncomment when using github enterprise

secret:
  name: 'cloudsecret'
  nxCloudMongoServerEndpoint: 'NX_CLOUD_MONGO_SERVER_ENDPOINT'
  githubWebhookSecret: 'GITHUB_WEBHOOK_SECRET'
  githubAuthToken: 'GITHUB_AUTH_TOKEN'

Note that the secret must contain GITHUB_WEBHOOK_SECRET and GITHUB_AUTH_TOKEN. Read here on how to get those values.

More Information

You can find more information about Nx Cloud and running it on prem here.