How to: Configure Kubernetes or OpenShift to use the most performant file storage on Azure

Because everyone is rocking containers these days…

On June 1st 2019, Microsoft announced the GA of their latest shared file service: Azure NetApp Files (ANF)– a PaaS shared file service supporting both NFS/SMB protocols for enterprise workloads. NetApp technology is widely used in many on-premise deployments with Kubernetes and OpenShift today, however, the launch of ANF allows Azure users to take advantage of this ultra-low latency and high-performance platform without having to know ANY NetApp at all.

Let’s just recap on that again:

“without having to know ANY NetApp at all”

Now, with the release of Trident (19.10) – a dynamic storage provisioner for Kubernetes / OpenShift / Docker, Azure users can quickly ensure that their containers and applications are able to easily take advantage of the benefits of ANF, automatically.

Why?

Great question, using Microsoft’s ANF as your underlying data platform allows you to take advantage of:

  • Consistent sub-millisecond latency – making your apps extremely responsive
  • Scalability – grow the volumes up to 100TB in seconds
  • Simplicity -with three performance levels and simple to understand performance, you can ensure you’ll focus on the important stuff as opposed to worrying about your application responsiveness because of the underlying data platform.

How?

Getting started is simple, and we’ll run through a sample end to end deployment in this section. Before we do, let’s take a moment to understand what deployment options we have with Azure NetApp Files and Kubernetes in Azure:

NOTE: UPDATE 10 MARCH 2020 – AKS now supports trident for dynamic provisioning.

Supported Deployment Options

This guide will focus on using AKS-Engine to deploy our Kubernetes clusters as we want full control of the management nodes in order to use advanced features like dynamic provisioning of Azure NetApp Files storage.

The overall workflow is:

  1. Deploy Kubernetes cluster in Azure using AKS-Engine
  2. Install Trident into the master node
  3. Configure Azure NetApp Files
  4. Done!

Let’s Get Started… (Grab a coffee first!)

Deploy Kubernetes cluster in Azure using AKS-Engine

In order to configure the Kubernetes features for advanced data management offered by Trident you will need to enable the flags described here: https://netapp-trident.readthedocs.io/en/stable-v19.07/support/requirements.html

aks-engine is the deployment mechanism used by AKS to deploy clusters. Using it directly gives you more control over the management node, whilst still allowing you to perform easy upgrades, etc. Full deployment steps can be found here: https://github.com/Azure/aks-engine/blob/master/docs/tutorials/quickstart.md

Create a resource group

az group create — name k8s — location <yourLocation>

For example, I use westeurope as my location, you can run an az list account-locations to view all the location codes. Be sure that you’re deploying your cluster in a location/region with Azure NetApp Files!

Find your subscription:

az account list -o table

Take note of your subscription ID, you’ll need it later

Create the service principal for the Kubernetes cluster

We will use a service principal in order to provide security and permissions to Trident over our Azure NetApp Files resources.

az ad sp create-for-rbac — role=”Contributor” — scopes=”/subscriptions/<yourSubId>/resourceGroups/Kubernetes”

Once you’ve created your service principal, assign it a role via Subscriptions -> select your sub -> IAM ->Add Role Assignment

Role Assignment

Deploy Kubernetes cluster using aks-engine

Make a note of the appId and password as you’ll use these in the next command:

Next, we’ll use aks-engine to deploy our cluster. An example configuration file called kubernetes.json is available at my GitHub repo (https://github.com/kirkryan/aks-engine/blob/master/examples/kubernetes.json).

The example will deploy a small cluster using Standard_B2s VM sizes. Be sure to change api-model below to match the correct path of your config file.

aks-engine deploy — subscription-id <yourAzureSubID> \
— dns-prefix <enterAnyName> \
— resource-group <yourResourceGroupName>\
— location <westeurope> \
— api-model examples/kubernetes.json \
— client-id <appId> \
— client-secret <password>\
— set servicePrincipalProfile.clientId=”<appId>” \
— set servicePrincipalProfile.secret=”<password>”

This will deploy your cluster. If you used the example file provided above you will have the latest version of Kubernetes deployed.

Download and install Trident

Now that your cluster is up and running, log into the management node via SSH and run the following commands:

wget https://github.com/NetApp/trident/releases/download/v19.07.0/trident-installer-19.07.0.tar.gz
tar -xf trident-installer-19.07.0.tar.gz
cd trident-installer

Now install Trident with the following command

./tridentctl install -n trident

Once finished you can check the status of your installation with the following command:

./tridentctl -n trident version


and

kubectl get pod -n trident

That’s Trident installed, next up is configuring the backend configuration. Trident supports a wide variety of backends, and in this instance, I have provided a quick start template for you here: https://github.com/kirkryan/quickstartTridentANF/blob/master/backend-azure-anf-advanced.json.

Note: More advanced configurations can be found here: https://netapp-trident.readthedocs.io/en/stable-v19.07/kubernetes/operations/tasks/backends/anf.html#example-configurations

With the backend configuration created, run:

./tridentctl -n trident create backend -f backend-azure-anf-advanced.json

Next, let’s add a storage class, a quickstart template that will create a class called azure-netapp-files is here: https://github.com/kirkryan/quickstartTridentANF/blob/master/storage-class-csi.yaml

kubectl create -f storage-class-csi.yaml

Check the status of your storage class (shown above) with:

kubectl get sc azure-netapp-files

Next, create a persistent volume claim (PVC) (quickstart here: https://github.com/kirkryan/quickstartTridentANF/blob/master/pvc-basic.yaml).

kubectl create -f pvc-basic.yaml

You can monitor the status of your PVC with the following command. A volume typically takes 1–3 minutes to provision.

Congratulations! Your Kubernetes cluster is now using Azure NetApp Files!

Optional: You can set Azure NetApp Files to be the default storage class with the following two commands:

Set the storage class called azure-netapp-files (if you used the quickstart — or your storage class name if you specified differently)

kubectl patch storageclass azure-netapp-files -p ‘{“metadata”: {“annotations”:{“storageclass.kubernetes.io/is-default-class”:”true”}}}’

Remove the old default (Azure Managed Disks)

kubectl patch storageclass default -p ‘{“metadata”: {“annotations”:{“storageclass.kubernetes.io/is-default-class”:”false”}}}’

That’s it! Your applications are now able to take advantage of the fastest shared file storage in Azure!

Published by Kirk

I'm a technical lead working for NetApp & Microsoft on advanced data capabilities in Azure, primarily the Azure NetApp Files solution and use cases. I like to share everything I've learnt along the way in the hope that somewhere along the line, someone will find it useful.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: