Using Loft in CI/CD Pipelines

When using Loft in a CI/CD pipeline to create namespaces and virtual cluster, there are couple of things to consider:

1. You might want to generate your kube configs manually
2. You may want to use the offical loftsh/loft-ci image in a containerized CI/CD pipeline
3. You definitely want to authenticate using Access Keys in either case
4. If using GitHub Actions for CI/CD see the section on Using GitHub Actions
5. If using GitLab for CI/CD see Using GitLab CI/CD Pipelines

Authentication & Cluster Access

You can easily construct a kube config that can be used directly in any external CI/CD pipeline or tool to access a space, connected cluster or vcluster directly. For this you'll only need an access key.

Then you can create a kube config in this format:

Space

Create a kubeconfig.yaml with:

apiVersion: v1
kind: Config
clusters:
- cluster:    
    # Optional if untrusted certificate
    # insecure-skip-tls-verify: true
    server: https://my-loft-domain.com/kubernetes/cluster/$CLUSTER
  name: loft
contexts:
- context:
    cluster: loft
    namespace: $SPACE
    user: loft
  name: loft
current-context: loft
users:
- name: loft
  user:
    token: $ACCESS_KEY

Replace the $ACCESS_KEY with your generated access key, $CLUSTER with the name of the connected kubernetes cluster the space was created in and $SPACE with the name of the space. You can now use this kube config with any external applications such as kubectl or terraform.

Then run any command in the space with:

kubectl --kubeconfig kubeconfig.yaml get pods

Cluster

Create a kubeconfig.yaml with:

apiVersion: v1
kind: Config
clusters:
- cluster:    
    # Optional if untrusted certificate
    # insecure-skip-tls-verify: true
    server: https://my-loft-domain.com/kubernetes/cluster/$CLUSTER
  name: loft
contexts:
- context:
    cluster: loft
    user: loft
  name: loft
current-context: loft
users:
- name: loft
  user:
    token: $ACCESS_KEY

Replace the $ACCESS_KEY with your generated access key and $CLUSTER with the name of the connected kubernetes cluster. You can now use this kube config with any external applications such as kubectl or terraform.

Then run any command in the cluster with:

kubectl --kubeconfig kubeconfig.yaml get spaces

vcluster

Create a kubeconfig.yaml with:

apiVersion: v1
kind: Config
clusters:
- cluster:    
    # Optional if untrusted certificate
    # insecure-skip-tls-verify: true
    server: https://my-loft-domain.com/kubernetes/virtualcluster/$CLUSTER/$NAMESPACE/$VCLUSTER
  name: loft
contexts:
- context:
    cluster: loft
    user: loft
  name: loft
current-context: loft
users:
- name: loft
  user:
    token: $ACCESS_KEY

Replace the $ACCESS_KEY with your generated access key, $CLUSTER with the name of the connected kubernetes cluster the vcluster is running in, $NAMESPACE with the namespace the vcluster is running in and $VCLUSTER with the name of the vcluster. You can now use this kube config with any external applications such as kubectl or terraform.

Then run any command in the vcluster with:

kubectl --kubeconfig kubeconfig.yaml get ns

RBAC Permissions

In order for a user to access a virtual cluster the user needs the RBAC permission get on the resource virtualclusters in the api group storage.loft.sh with api version v1 either in the namespace where the virtual cluster was created in or cluster wide

Authentication

Create Access Keys

Loft UI - Profile: Create Access Key

Using the Container Image

When using Loft in a CI/CD pipeline that runs based on containers, you can use the official loft-ci image either as a base image or directly.

This image is based on alpine and contains:

• Loft CLI
• kubectl
• helm v3
• DevSpace

This is what the Dockerfile looks like:

FROM devspacesh/devspace:5

# Add helm
RUN wget -O helm.tar.gz https://get.helm.sh/helm-v3.3.3-linux-amd64.tar.gz \
 && tar -zxvf helm.tar.gz \
 && mv linux-amd64/helm /bin/helm

# Add Loft CLI (same version as the tag of this image)
COPY release/loft-linux-amd64 /bin/loft
RUN chmod +x /bin/loft

Login with Access Keys

loft login https://my-loft.url.tld --access-key [ACCESS_KEY] # Replace URL with your own

Integrations

GitHub Actions

Loft provides the following GitHub Actions for use in workflows:

• Setup DevSpace: Installs the devspace CLI
• Setup Loft: Installs the loft CLI and logs in to the Loft instance
• Create Space: Creates a Space
• Delete Space: Deletes a Space
• Create Virtual Cluster: Creates a Virtual Cluster
• Delete Virtual Cluster: Deletes a Virtual Cluster

Spaces for Pull Requests

These examples show how to create and delete Spaces for pull requests.

Basic

This example shows how to create and delete a space to test an application named my-app for pull requests.

# .github/workflows/prs.yaml
name: Pull Request Checks
on:
  pull_request:
    branches:
      - 'main'
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - name: Install Loft CLI
        uses: loft-sh/setup-loft@main
        with:
          url: ${{ secrets.LOFT_URL }}
          access-key: ${{ secrets.LOFT_ACCESS_KEY }}
      - name: Create Space for PR
        uses: loft-sh/create-space@main
        with:
          name: pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }}
      - name: Deploy Application
        run: kubectl apply -Rf ./kubernetes
      - name: Wait for Deployment
        run: kubectl rollout status -n pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }} deployments/my-app
      - name: Run Tests
        run: make e2e
      - name: Delete PR Space
        uses: loft-sh/delete-space@main
        with:
          name: pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }}

Explanation:

1. The Setup Loft action is used to install the Loft CLI and login using the provided url and access-key.
2. The Create Space action is used to create a unique space using information about the pull request. This will automatically configure the kube context for the following steps.
3. The next step deploys the application using the runner provided kubectl and manifests located under ./kubernetes.
4. Before running tests, we use kubectl to wait for the my-app deployment to become ready.
5. Now we run the end-to-end tests. In this example we're using make to run tests, but the command should be customized for your testing framework.
6. Finally, the Delete Space GitHub Action is used to delete the pull request's space.

Automatic Cleanup

This example shows how to automatically delete a Space after testing an application named my-app for pull requests.

# .github/workflows/prs.yaml
name: Pull Request Checks
on:
  pull_request:
    branches:
      - 'main'
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - name: Install Loft CLI
        uses: loft-sh/setup-loft@main
        with:
          url: ${{ secrets.LOFT_URL }}
          access-key: ${{ secrets.LOFT_ACCESS_KEY }}
      - name: Create Space for PR
        uses: loft-sh/create-space@main
        with:
          name: pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }}
          auto-cleanup: true
      - name: Deploy Application
        run: kubectl apply -Rf ./kubernetes
      - name: Wait for Deployment
        run: kubectl rollout status -n pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }} deployments/my-app
      - name: Run Tests
        run: make e2e

Explanation:

1. The Setup Loft action is used to install the Loft CLI and login using the provided url and access-key.
2. The Create Space action is used to create a unique space using information about the pull request. This will automatically configure the kube context for the following steps. Additionally, we have enabled the auto-cleanup option, which will delete the space after the job completes.
3. The next step deploys the application using the runner provided kubectl and manifests located under ./kubernetes.
4. Before running tests, we use kubectl to wait for the my-app deployment to become ready.
5. Finally we run the end-to-end tests. In this example we're using make to run tests, but the command should be customized for your testing framework. There's no need to delete the space since the auto-cleanup option was used when creating the space.

DevSpace

This example shows how use the Setup Devspace GitHub Action to install the DevSpace CLI and DevSpace commands to run tests.

# .github/workflows/prs.yaml
name: Pull Request Checks
on:
  pull_request:
    branches:
      - 'main'
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - name: Install DevSpace CLI
        uses: loft-sh/setup-devspace@main
      - name: Install Loft CLI
        uses: loft-sh/setup-loft@main
        with:
          url: ${{ secrets.LOFT_URL }}
          access-key: ${{ secrets.LOFT_ACCESS_KEY }}
      - name: Create Space for PR
        uses: loft-sh/create-space@main
        with:
          name: pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }}
          auto-cleanup: true
      - name: Run Tests
        run: devspace run e2e

Explanation:

1. The Setup DevSpace action installs the DevSpace CLI.
2. The Setup Loft action is used to install the Loft CLI and login using the provided url and access-key.
3. The Create Space action is used to create a unique space using information about the pull request. This will automatically configure the kube context for the following steps. Additionally, we have enabled the auto-cleanup option, which will delete the space after the job completes.
4. Finally we use devspace run e2e to perform the needed steps to deploy and test my-app.

Virtual Clusters for Pull Requests

These examples show how to create and delete Virtual Clusters for pull requests.

Basic

This example shows how to create and delete a virtual cluster for testing an application named my-app on pull requests.

# .github/workflows/vclusters.yaml
name: Pull Request Checks
on:
  pull_request:
    branches:
      - 'main'
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - name: Install Loft CLI
        uses: loft-sh/setup-loft@main
        with:
          url: ${{ secrets.LOFT_URL }}
          access-key: ${{ secrets.LOFT_ACCESS_KEY }}
      - name: Create Virtual Cluster for PR
        uses: loft-sh/create-vcluster@main
        with:
          name: pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }}
      - name: Deploy Application
        run: kubectl apply -Rf ./kubernetes
      - name: Wait for Deployment
        run: kubectl rollout status deployments/my-app
      - name: Run Tests
        run: make e2e
      - name: Delete PR Virtual Cluster
        uses: loft-sh/delete-vcluster@main
        with:
          name: pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }}

Explanation:

1. The Setup Loft action is used to install the Loft CLI and login using the provided url and access-key.
2. The Create Virtual Cluster action is used to create a unique virtual cluster using information about the pull request. This will automatically configure the kube context for the following steps.
3. The next step deploys the application using the runner provided kubectl and manifests located under ./kubernetes.
4. Before running tests, we use kubectl to wait for the my-app deployment to become ready.
5. Now we run the end-to-end tests. In this example we're using make to run tests, but the command should be customized for your testing framework.
6. Finally, the Delete Virtual Cluster GitHub Action is used to delete the virtual cluster.

Automatic Cleanup

This example shows how to automatically delete a Virtual Cluster after testing an application named my-app for pull requests.

# .github/workflows/vclusters.yaml
name: Pull Request Checks
on:
  pull_request:
    branches:
      - 'main'
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - name: Install Loft CLI
        uses: loft-sh/setup-loft@main
        with:
          url: ${{ secrets.LOFT_URL }}
          access-key: ${{ secrets.LOFT_ACCESS_KEY }}
      - name: Create Virtual Cluster for PR
        uses: loft-sh/create-vcluster@main
        with:
          name: pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }}
          auto-cleanup: true
      - name: Deploy Application
        run: kubectl apply -Rf ./kubernetes
      - name: Wait for Deployment
        run: kubectl rollout status deployments/my-app
      - name: Run Tests
        run: make e2e

Explanation:

1. The Setup Loft action is used to install the Loft CLI and login using the provided url and access-key.
2. The Create Virtual Cluster action is used to create a unique virtual cluster using information about the pull request. This will automatically configure the kube context for the following steps. Additionally, we have enabled the auto-cleanup option, which will delete the virtual cluster after the job completes.
3. The next step deploys the application using the runner provided kubectl and manifests located under ./kubernetes.
4. Before running tests, we use kubectl to wait for the my-app deployment to become ready.
5. Finally we run the end-to-end tests. In this example we're using make to run tests, but the command should be customized for your testing framework. There's no need to delete the virtual cluster since the auto-cleanup option was used when creating the virtual cluster.

DevSpace

This example shows how use the Setup Devspace GitHub Action to install the DevSpace CLI and DevSpace commands to run tests.

# .github/workflows/vclusters.yaml
name: Pull Request Checks
on:
  pull_request:
    branches:
      - 'main'
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - name: Install DevSpace CLI
        uses: loft-sh/setup-devspace@main
      - name: Install Loft CLI
        uses: loft-sh/setup-loft@main
        with:
          url: ${{ secrets.LOFT_URL }}
          access-key: ${{ secrets.LOFT_ACCESS_KEY }}
      - name: Create Virtual Cluster for PR
        uses: loft-sh/create-vcluster@main
        with:
          name: pr-${{ github.event.pull_request.number }}-${{ github.sha }}-${{ github.run_id }}
          auto-cleanup: true
      - name: Run Tests
        run: devspace run e2e

Explanation:

1. The Setup DevSpace action installs the DevSpace CLI.
2. The Setup Loft action is used to install the Loft CLI and login using the provided url and access-key.
3. The Create Virtual Cluster action is used to create a unique virtual cluster using information about the pull request. This will automatically configure the kube context for the following steps. Additionally, we have enabled the auto-cleanup option, which will delete the virtual cluster after the job completes.
4. Finally we use devspace run e2e to perform the needed steps to deploy and test my-app.

GitLab CI/CD

When using Loft with GitLab you can use the official loft-ci image as either a base image or directly. See Using the Container Image for details on the available tools. If additional tooling is needed for your CI/CD process, a custom image can be created.

Spaces for Merge Requests

This example shows how to create and delete a Space for running end-to-end tests for the default branch and merge requests. It assumes you have configured CI/CD variables LOFT_URL and LOFT_ACCESS_KEY.

image: loftsh/loft-ci

stages:
  - test

e2e:
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  stage: test
  before_script:
    - loft login $LOFT_URL --access-key $LOFT_ACCESS_KEY
    - loft create space "${CI_PROJECT_NAME}-${CI_COMMIT_SHORT_SHA}-${CI_PIPELINE_ID}"
    - apk add go make
  script:
    - kubectl apply -Rf ./kubernetes
    - kubectl rollout status deployments/my-app
    - make e2e
  after_script:
    - loft delete space "${CI_PROJECT_NAME}-${CI_COMMIT_SHORT_SHA}-${CI_PIPELINE_ID}"

Explanation:

1. The loftsh/loft-ci image is used for all pipeline jobs and provides the loft CLI, the devspace CLI, and kubectl.
2. The before_script first logs in to loft using the $LOFT_URL and $LOFT_ACCESS_KEY variables that you defined in GitLab. See the GitLab docs for more information
3. The before_script then creates a space using predefined GitLab variables to create a unique name for the space.
4. Next before_script installs some additional tooling needed to run the end-to-end tests. For more complex scenarios creating a custom image is recommended.
5. Then the script section uses kubectl to deploy the application to the space and waits for the my-app deployment to become ready. make is then used to run an end-to-end test suite.
6. Finally after_script deletes the space. By using after_script we can ensure the space is deleted even if the tests fail.

Virtual Clusters for Merge Requests

This example shows how to create and delete a Virtual Cluster for running end-to-end tests for the default branch and merge requests. It assumes you have configured CI/CD variables LOFT_URL and LOFT_ACCESS_KEY.

image: loftsh/loft-ci

stages:
  - test

e2e:
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  stage: test
  before_script:
    - loft login $LOFT_URL --access-key $LOFT_ACCESS_KEY
    - loft create vcluster "${CI_PROJECT_NAME}-${CI_COMMIT_SHORT_SHA}-${CI_PIPELINE_ID}"
    - apk add go make
  script:
    - kubectl apply -Rf ./kubernetes
    - kubectl rollout status deployments/my-app
    - make e2e
  after_script:
    - loft delete vcluster "${CI_PROJECT_NAME}-${CI_COMMIT_SHORT_SHA}-${CI_PIPELINE_ID}" --delete-space

Explanation:

1. The loftsh/loft-ci image is used for all pipeline jobs and provides the loft CLI, the devspace CLI, and kubectl.
2. The before_script first logs in to loft using the $LOFT_URL and $LOFT_ACCESS_KEY variables that you defined in GitLab. See the GitLab docs for more information
3. The before_script then creates a virtual cluster using predefined GitLab variables to create a unique name.
4. Next before_script installs some additional tooling needed to run the end-to-end tests. For more complex scenarios creating a custom image is recommended.
5. Then the script section uses kubectl to deploy the application to the space and waits for the my-app deployment to become ready. make is then used to run an end-to-end test suite.
6. Finally after_script deletes the virtual cluster, and passes --delete-space to ensure the corresponding space for the cluster is deleted. By using after_script we can ensure the space is deleted even if the tests fail.