Upgrade Kommander

This section describes how to upgrade your Kommander Management cluster and all Platform Applications to their supported versions in air-gapped, non-air-gapped and on-prem environments. To prevent compatibility issues, you must first upgrade Kommander on your Management Cluster before upgrading to DKP.

It is important you upgrade Kommander BEFORE upgrading the Kubernetes version (or Konvoy version for Managed Konvoy clusters) in attached clusters. This ensures that any changes required for new or changed Kubernetes API’s are already present.

Page Contents

Prerequisites

REQUIRED Before upgrading, create an on-demand backup of your current configuration with Velero.
Download and install the supported DKP CLI binary of this release on your computer.
Ensure you are attempting to run a supported DKP upgrade.
Ensure you are on Kubernetes version 1.23 or higher.
If you have attached clusters, ensure they are on Kubernetes versions 1.22 or higher.
Review the DKP 2.4.1 Components and Applications versions that are part of this upgrade.
For air-gapped environments with DKP Catalog Applications in a multi-cluster environment: Load the Docker images into your Docker registry
For air-gapped environments without DKP Catalog Applications: Load the Docker images into your Docker registry

DKP Upgrade with Insights Installed

If your environment has Insights installed, you must uninstall Insights BEFORE upgrading DKP.

Prerequisites for Configurations with NVIDIA GPU Nodes

If your cluster is running GPU nodes and has deployed the nvidia Platform Application, you must run additional steps prior to upgrading Kommander. In DKP 2.4, the nvidia application is replaced by nvidia-gpu-operator during upgrade, and the configuration for the new application must be created prior to the upgrade to ensure that the new application is deployed with the proper configuration, or it may not deploy successfully. If your cluster is not running GPU nodes, then skip these steps and proceed directly to the Upgrade Kommander section.

The GPU nodes will experience downtime during the upgrade due to the new version of the Nvidia application.

Create the ConfigMap with the necessary configuration overrides to set the correct Toolkit version. For example, if you’re using Centos 7.9 or RHEL 7.9 as the base operating system for your GPU enabled nodes, set the toolkit.version parameter:
CODE
```
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: kommander
  name: nvidia-gpu-operator-overrides
data:
  values.yaml: |
    toolkit:
      version: v1.10.0-centos7
EOF
```

Create an AppDeployment named nvidia-gpu-operator in the kommander namespace:

CODE

cat <<EOF | kubectl apply -f -
apiVersion: apps.kommander.d2iq.io/v1alpha3
kind: AppDeployment
metadata:
  name: nvidia-gpu-operator
  namespace: kommander
spec:
  appRef:
    name: nvidia-gpu-operator-1.11.1
    kind: ClusterApp
  configOverrides:
    name: nvidia-gpu-operator-overrides
EOF

You may now proceed to the Kommander upgrade steps.

Upgrade Kommander

Before running the following command, ensure that your dkp configuration references the Kommander Management cluster, otherwise it attempts to run the upgrade on the bootstrap cluster. You can do this by setting the KUBECONFIG environment variable to the appropriate kubeconfig file’s location.

In this version of DKP, pre-provisioned environments require a special upgrade path. Refer to Upgrade Kommander in a Pre-provisioned Environment for more information.

If you have configured a custom domain, running the upgrade command could result in an inaccessibility of your services via your custom domain for a few minutes.

As stated earlier, an alternative to initializing the KUBECONFIG environment variable is to use the –kubeconfig=cluster_name.conf flag. This ensures that Kommander upgrades on the workload cluster.

Use the DKP CLI to upgrade Kommander and all the Platform Applications in the Management Cluster:
1. For air-gapped environments:
  CODE
```
dkp upgrade kommander --charts-bundle ./application-charts/dkp-kommander-charts-bundle-v2.4.2.tar.gz --kommander-applications-repository ./application-repositories/kommander-applications-v2.4.2.tar.gz
```
2. For air-gapped environments with DKP Catalog Applications in a multi-cluster environment:
  CODE
```
dkp upgrade kommander --charts-bundle ./application-charts/dkp-kommander-charts-bundle-v2.4.2.tar.gz --charts-bundle ./application-charts/dkp-catalog-applications-charts-bundle-v2.4.2.tar.gz --kommander-applications-repository ./application-repositories/kommander-applications-v2.4.2.tar.gz
```
  After the upgrade, if you have DKP Catalog Applications deployed, update the GitRepository for dkp-catalog-applications.
  For the following section, ensure you modify the most recent kommander.yaml configuration file. It must be the file that reflects the current state of your environment. Reinstalling Kommander with an outdated kommander.yaml overwrites the list of platform applications that are currently running in your cluster.
  1. In the kommander.yaml you are currently using for your environment, update the DKP Catalog Applications by setting the correct DKP version:
    CODE
```
apiVersion: config.kommander.mesosphere.io/v1alpha1
kind: Installation
...
# The list of enabled/disabled apps here should reflect the current state of the environment, including configuration overrides!
...
catalog:
  repositories:
    - name: dkp-catalog-applications
      labels:
        kommander.d2iq.io/project-default-catalog-repository: "true"
        kommander.d2iq.io/workspace-default-catalog-repository: "true"
        kommander.d2iq.io/gitapps-gitrepository-type: "dkp"
      path: ./dkp-catalog-applications-v2.4.2.tar.gz # modify this version to match the DKP upgrade version
```
  2. Refresh the kommander.yaml to apply the updated tarball:
    Before running this command, ensure the kommander.yaml is the configuration file you are currently using for your environment. Otherwise, your previous configuration will be lost.
    CODE
```
dkp install kommander --installer-config kommander.yaml
```
3. For non-air-gapped environments:
  CODE
```
dkp upgrade kommander
```
  After the upgrade, if you have DKP Catalog Applications deployed, update the GitRepository for dkp-catalog-applications.
  1. Update the GitRepository with the tag of your updated DKP version on the kommander workspace:
    CODE
```
kubectl patch gitrepository -n kommander dkp-catalog-applications --type merge --patch '{"spec":{"ref":{"tag":"v2.4.2"}}}'
```
    This command updates the catalog application repositories for all workspaces.
  2. For any additional workspaces created outside the kommander.yaml configuration:
    Set the WORKSPACE_NAMESPACE environment variable to the namespace of the workspace:
    CODE
```
export WORKSPACE_NAMESPACE=<workspace namespace>
```
  3. Update the GitRepository for the additional workspace:
    CODE
```
kubectl patch gitrepository -n ${WORKSPACE_NAMESPACE} dkp-catalog-applications --type merge --patch '{"spec":{"ref":{"tag":"v2.4.2"}}}'
```
For Enterprise customers (multi-cluster environment): Upgrade your additional Workspaces on a per-Workspace basis to upgrade the Platform Applications on other clusters than the Management Cluster.
For Essential customers (single-cluster environment): Proceed with the Konvoy Upgrade.

Troubleshooting

If the upgrade fails, run the following command to get more information on the upgrade process:

CODE

dkp upgrade kommander -v 6

If you find any HelmReleases in a “broken” release state such as “exhausted” or “another rollback/release in progress”, you can trigger a reconciliation of the HelmRelease using the following commands:

CODE

kubectl -n kommander patch helmrelease <HELMRELEASE_NAME> --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": true}]'
kubectl -n kommander patch helmrelease <HELMRELEASE_NAME> --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": false}]'

You can always go back to the DKP Upgrade overview, to review the next steps depending on your environment and license type.