Azure Kubernetes Service (AKS) with Azure Key Vault (AKV) Part 2— akv2k8s
This actually should not be a part 2 as it is not related to part 1 but actually a different way of setting AKS to get AKV’s resources. I name it as part 2 because I would people to go through either method.
Step-by-Step Guidance
- Create an AKS cluster by following this documentation
- Get AKS user-assigned identity
# get the AKS associated service principal
az aks show -g <resource group name> -n <AKS name> | grep identityProfile -A 5# note down the object ID of the service principal
- Create an AKV
# create resource group
az group create --name <resource group name> --location <ex: westus, eastus># create azure key vault via AZ CLI
az keyvault create -g <resource group name> -l <location> -n <AKV name>Ref: az keyvault | Microsoft Docs# create azure key vault via Azure portal
Quickstart - Create an Azure Key Vault with the Azure portal | Microsoft Docs
- Create a certificate into the AKV by following this documentation
- Provide the AAD SP the access and permissions to get both secrets and certificates within the target AKV. We are creating certificates in AKV but storing it as secrets in AKS. So, in this case, we would need both permissions.
# provide AAD SP the permission to get certificates
az keyvault set-policy --name <AKV name> --object-id <AAD SP objectId> --certificate-permissions get# provide AAD SP the permission to get secrets
az keyvault set-policy --name <AKV name> --object-id <objectId> --secret-permissions getRef: az keyvault | Microsoft Docs
- Install akv2k8s on AKS
# create a new namespace
kubectl create ns akv2k8s## install via HELM
# add HELM repo
helm repo add spv-charts https://charts.spvapi.no# update HELM repo
helm repo update# install akv2k8s
helm upgrade --install akv2k8s spv-charts/akv2k8s \
--namespace akv2k8s
- Create a resource of AzureKeyVaultSecret
The yellow part is self-defined; the green part is the AKV name; the blue part is the certificate within AKV and the pink part is what would be shown within your AKS environment after it gets synchronized.
- Create a NGINX Ingress Controller
You could definitely install the whole solution with one-liner below.
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.45.0/deploy/static/provider/baremetal/deploy.yamlHowever, this would be creating all the resources in the namespace called “ingress-nginx”. If you would like to create resources in your own namespace, you would first need to download the whole template from https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.45.0/deploy/static/provider/baremetal/deploy.yaml. Inside your Linux environment, use the command below to replace the namespace you would like.
# replace the namespace with something you need
sed -i ‘s/namespace: ingress-nginx/namespace: <new namespace name>/g’ input.txt# create the NGINX Ingress Controller
kubectl apply -f <whatever name you save>.yaml
- Create a test application deployment, service and Ingress
# the sample YAML file
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
labels:
app: httpbin
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
template:
metadata:
labels:
app: httpbin
spec:
containers:
- name: httpbin
image: kennethreitz/httpbin
---
apiVersion: v1
kind: Service
metadata:
name: realtime
labels:
app: httpbin
spec:
selector:
app: httpbin
ports:
- protocol: TCP
port: 80
targetPort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-akv2k8s
annotations:
kubernetes.io/ingress.class: nginx
spec:
tls:
- hosts:
- helloworld.info
secretName: nginx-cert
rules:
- host: helloworld.info
http:
paths:
- path: /
backend:
serviceName: realtime
servicePort: 80# create the deployment, service and Ingress
kubectl apply -f <whatever name you have>.yaml -n <namespace name>
Demonstration
# get the Ingress public IP address
kubectl get ing -n <namespace name># check the self-signed certificate
openssl s_client -connect <IP you see in Ingress>:443 -servername <server FQDN> -status -tlsextdebug# check the Ingress service
curl -vk --resolve <hostname set in certificate>:443:<IP address see in Ingress> https://<hostname in certificate>
Troubleshooting
If you are experiencing any kind of situation, such as AKVS resource not able to get AKV’s resource to use as AKS secret. The first thing would be checking the akv2k8s logs and see what are the error messages.
kubectl -n akv2k8s logs deployment/akv2k8s-controllerFor me personally, I was being blocked because lack of required permissions. The AKS user-assigned identity would need both get-secret and get-certificate permissions within AKV, but I was only giving get-certificate. After adding the right permission, the identity object could retrieve the certificate without issues.
For more troubleshooting thoughts, please refer to this section of the official documentation.
That is it! This is the other way how you could let AKS get AKV’s resource as native secrets and associate them in NGINX Ingress Controller! Hope this would save some people’s time by looking into multiple documentations for reference! Happy learning!
For more information, please check
