Pre-provisioned Azure only Configurations

After your bootstrap is running and your cluster is created, you will need to install the Azure Disk CSI Driver on your pre-provisioned Azure Kubernetes cluster. The DKP pre-provisioned provider installs by default the storage-local-static-provisioner CSI driver, which is not suitable for production environments. For this reason, it needs to be replaced by the Azure Disk CSI Driver.

Prerequisites:

Before you begin using DKP you must have:

An x86_64-based Linux or macOS machine.
Download the dkp binary for Linux, or macOS. To check which version of DKP you installed for compatibility reasons, run the dkp version command (dkp version).
A Container engine/runtime installed is required to install DKP:
- Version Docker® container engine version 18.09.2 or higher installed for Linux or MacOS - On macOS, Docker runs in a virtual machine which needs configured with at least 8 GB of memory.
- Version 4.0 of Podman or higher for Linux. Host requirements found here: Host Requirements
kubectl for interacting with the running cluster.
Azure CLI.
A valid Azure account with credentials configured.
Create a custom Azure image using KIB.

On macOS, Docker runs in a virtual machine. Configure this virtual machine with at least 8GB of memory.

Set Environment Variables with Credentials:

An Azure Service Principal is needed for deploying resources. To configure your Azure environment, follow below:

Log in to Azure:

CODE

az login

CODE

[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "a1234567-b132-1234-1a11-1234a5678b90",
    "id": "b1234567-abcd-11a1-a0a0-1234a5678b90",
    "isDefault": true,
    "managedByTenants": [],
    "name": "Mesosphere Developer Subscription",
    "state": "Enabled",
    "tenantId": "a1234567-b132-1234-1a11-1234a5678b90",
    "user": {
      "name": "user@azuremesosphere.onmicrosoft.com",
      "type": "user"
    }
  }
]

Create an Azure Service Principal (SP) by running the following command:
Note: If an SP with the name exists, this command will rotate the password.
CODE
```
az ad sp create-for-rbac --role contributor --name "$(whoami)-konvoy" --scopes=/subscriptions/$(az account show --query id -o tsv)
```
CODE
```
{
  "appId": "7654321a-1a23-567b-b789-0987b6543a21",
  "displayName": "azure-cli-2021-03-09-23-17-06",
  "password": "Z79yVstq_E.R0R7RUUck718vEHSuyhAB0C",
  "tenant": "a1234567-b132-1234-1a11-1234a5678b90"
}
```
For air-gapped environments, you need to create a resource management private link with a private endpoint to ensure the Azure CSI driver will run correctly in further steps. Private link resource can be deployed in different regions from the virtual network and private endpoint.
To set up a private link resource, use the following process.
1. Create the resource management private link using Azure CLI.
2. Create a private link association for the root management group which also references the resource ID for the resource management private link. Provide link association for a management group.
3. Add a private endpoint that references the resource management private link. Create a Private Endpoint using the Azure Documentation.

Set the required environment variables using that output:

CODE

export AZURE_SUBSCRIPTION_ID="<id>"       # b1234567-abcd-11a1-a0a0-1234a5678b90
export AZURE_TENANT_ID="<tenant>"         # a1234567-b132-1234-1a11-1234a5678b90
export AZURE_CLIENT_ID="<appId>"          # 7654321a-1a23-567b-b789-0987b6543a21
export AZURE_CLIENT_SECRET="<password>"   # Z79yVstq_E.R0R7RUUck718vEHSuyhAB0C
export AZURE_RESOURCE_GROUP="<resource group name>" # set to the name of the resorce group
export AZURE_LOCATION="westus"            # set to the location you are using

Set your KUBECONFIG environment variable:
CODE
```
export kubeconfig=${CLUSTER_NAME}.conf
```

Create the Secret with the Azure credentials, this will be used by the Azure CSI driver:

Create an azure.json file:

CODE

cat <<EOF > azure.json
{
  "cloud": "AzurePublicCloud",
  "tenantId": "$AZURE_TENANT_ID",
  "subscriptionId": "$AZURE_SUBSCRIPTION_ID",
  "aadClientId": "$AZURE_CLIENT_ID",
  "aadClientSecret": "$AZURE_CLIENT_SECRET",
  "resourceGroup": "$AZURE_RESOURCE_GROUP",
  "location": "$AZURE_LOCATION"
}
EOF

Create the Secret:

CODE

kubectl create secret generic azure-cloud-provider --namespace=kube-system --type=Opaque --from-file=cloud-config=azure.json

Install the Azure Disk CSI driver:

CODE

$ curl -skSL https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/v1.26.2/deploy/install-driver.sh | bash -s v1.26.2 snapshot –

Check the status to see if the driver is ready for use:

CODE

kubectl -n kube-system get pod -o wide --watch -l app=csi-azuredisk-controller
kubectl -n kube-system get pod -o wide --watch -l app=csi-azuredisk-node

Now Kubernetes knows that this is Azure disk, and will create clusters on Azure. You are ready to create the StorageClass for the Azure Disk CSI Driver:
CODE
```
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/master/deploy/example/storageclass-azuredisk-csi.yaml
```

Change the default storage class to this new StorageClass so that every new disk will be created in the Azure environment:

CODE

kubectl patch sc/localvolumeprovisioner -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
kubectl patch sc/managed-csi -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Verify that the StorageClass chosen is currently the default:
CODE
```
kubectl get storageclass
```

For more information about Azure Disk CSI for persistent storage and changing the default StorageClass, refer to that page in the documentation: Default Storage Providers in DKP

Next Step:

Pre-provisioned Modify the Calico Installation