Skip to main content
Skip table of contents

Store secrets in GitOps repository using SealedSecrets

Securely managing secrets in a GitOps workflow using SealedSecrets

For security reasons, Kubernetes secrets are usually the only resource that cannot be managed with a GitOps workflow. Instead of managing secrets outside of GitOps and having to use a third-party tool like Vault, SealedSecrets provides a way to keep all the advantages of using a GitOps workflow while avoiding exposing secrets. SealedSecrets is composed of two main components:

  • A CLI (Kubeseal) to encrypt secrets.

  • A cluster-side controller to decrypt the sealed secrets into regular Kubernetes secrets. Only this controller can decrypt sealed secrets, not even the original author.

This tutorial describes how to install these two components, configure the controller, and add or remove sealed secrets.

Set up

These instructions are used as an example. For instructions on the latest release, see the release page. For full documentation on SealedSecrets, see the GitHub repo.

Install Kubeseal CLI to encrypt your secrets

  • On MacOS

    brew install kubeseal
  • On Linux

    wget -O kubeseal
    sudo install -m 755 kubeseal /usr/local/bin/kubeseal

Install the SealedSecrets controller on your cluster

This controller will be able to decrypt SealedSecrets and create Kubernetes secrets.

  1. Create the controller:

    kubectl apply -f
  2. Fetch the certificate that you will use to encrypt your secrets into sealed secrets:

    kubeseal --fetch-cert > mycert.pem
  3. Commit mycert.pem to your git repo.

Add a secret

Secrets can be securely added to Git using sealed secrets:

  1. Export the project namespace that you are using for your GitOps repository

    export PROJECT_NAMESPACE=<your-project-with-gitops>
  2. Create a Kubernetes secret and pipe it into kubeseal using the certificate mycert.pem that you fetched from the controller in the setup:

    echo '---' >> secrets.yaml
    kubectl create secret -n ${PROJECT_NAMESPACE} generic mysecret --dry-run=client -o yaml --from-literal=my-secret=value | \
        kubeseal --format yaml --cert mycert.pem >> secrets.yaml
  3. Go to the end of secrets.yaml where you just added your new sealed secret. Remove any “creationTimestamp” fields from the YAML.

  4. Apply the secrets.yaml file to your namespace. If you do not have permission, commit your changes to the repo and let FluxCD apply the changes for you.

    kubectl apply -f secrets.yaml
  5. The sealed secret controller will then decrypt the sealed secret and generate a Kubernetes secret from it. Your secret got successfully created by running:

    kubectl get secret mysecret -n ${PROJECT_NAMESPACE} -o yaml
  6. If your sealed secret got created successfully but did not generate the matching secret, look at the logs of the controller:

    kubectl logs -l=name=sealed-secrets-controller -n kube-system
  7. Commit secrets.yaml to your repo if you have not already done so in step 3.

Remove a secret

  1. Following the same example from above in “Adding a secret”, now remove the manifest for mysecret in secrets.yaml and commit those changes to the repo.

  2. Delete the SealedSecret in the cluster:

    kubectl delete SealedSecret -n ${PROJECT_NAMESPACE} mysecret
  3. Delete the secret itself:

    kubectl delete secret -n ${PROJECT_NAMESPACE} mysecret

Rotate the controller’s sealing key

For added security, it is a good practice to rotate the key the controller uses to decrypt sealed secrets. By default, the controller generates a new key every 30 days. When this happens, you need to update the certificate you use to create sealed secrets by fetching the latest one:

   kubeseal --fetch-cert > mycert.pem

Do not forget to commit it back to the repo!

In a disaster case, let’s say your cluster gets destroyed, you would lose all your sealing keys, so you would not be able to recreate all the secrets from the sealed secrets in your GitOps repo. For this reason, you might want to back up the sealing keys. To do this every time a new sealing key is generated, run:

kubectl get secret -n kube-system -l -o yaml > sealing-key

Then store sealing-key with the others in a safe location such as OneLogin Notes or Vault. To restore from a backup after a disaster, recreate all of the sealing keys with kubectl apply -f sealing-key1 sealing-key2 ... before starting the controller. If the controller was already started, restart it: kubectl delete pod -n kube-system -l name=sealed-secrets-controller

To disable sealing key rotation For example, configure the controller’s command in the pod template with --key-renew-period=0. See the following YAML file.

Pod Template:
  Labels:           name=sealed-secrets-controller
  Service Account:  sealed-secrets-controller
    Port:       8080/TCP
    Host Port:  0/TCP
    Command:    controller

If required, edit the controller’s manifest with:

kubectl edit deployment.apps/sealed-secrets-controller -n kube-system

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.