How to: K3s on Raspberry Pi

Taking a break from pure Azure deployments, I decided to try my hand at building a small Kubernetes cluster that would allow me to learn and develop applications quickly at home without burning any of my Azure credits. Of course I could easily deploy on an AKS cluster, but I want to learn more about the management clusters in Kubernetes in order to fully understand the benefits any managed solution brings (sometimes it’s best to understand the pain points first hand to truly see the value in a service/solution).

The Build

In order to build my personal Kubernetes cluster, I decided to invest in four of the latest Raspberry Pi’s with the largest RAM option available (Raspberry Pi 4 – 4GB). In addition to these, I happened to have a Raspberry Pi 3+ lying around not doing much, so that will become the master/API node to the cluster, leaving the beefier RPi4 available for use as the worker nodes.

High-Level Overview

In order to build my Kubernetes cluster I had to make two decisions:

  1. OS Distribution – this was between Raspbian and Ubuntu. I chose Ubuntu as I have used it in enterprise environments and have experience with it.
  2. Kubernetes tooling – this was between microk8s (https://microk8s.io/) and K3s (https://rancher.com/). I opted for K3s as it was simply the first set of tutorials I came across – I would like to give microK8s a go in future. Both offer optimisation for edge environments such as the Raspberry Pi.

Installation

Luckily the folks over at rancher have made this simple. Run the following command and you’ll have a single node K3s single node cluster up and running in no time.

curl -sfL https://get.k3s.io | sh -

Once this is installed you can check that your single node cluster is up and running with the following command – if this fails for you check the next section (additional configuration/troubleshooting)

kubectl get node

At this stage, start your other nodes and run the following command on them to install k3s and configure them to join the existing cluster:

curl -sfL https://get.k3s.io | K3S_URL=https://192.168.1.120:6443 K3S_TOKEN=K10373f030f773c58a476ca0332eda0beb8ef8ddfc6d3e1c909642939bc59c4d096::server:d2ec6a145a753bb676bb0034f71bb6c0 sh -

And that’s it! You now have a cluster up and running, ready for your next project! You can check your node status with the command sudo kubectl get nodes

If you liked this post then subscribe as I build upon this journey and start to connect more advanced use-cases and cloud integration to my projects!

What’s next?

I’m thinking of playing with Azure Arc and connecting it to my cluster to see what it is capable of and gaining experience with it. I also have a raft of projects (mainly cycling related) that I will deploy on this cluster! Keep tuned for more articles and videos!

Additional Configuration

When using the Raspberry Pi there are some additional configuration steps that must be performed in order to get K3s up and running successfully.

Ubuntu 20.04 LTS on Raspberry Pi

When I followed the above instructions, I hit a snag where no matter what I did, I couldn’t get the K3s service to start. After a bit of digging I found that a required flag for cgroups must be set on the Raspberry Pi. In order to do so follow these instructions (source: https://microk8s.io/docs/install-alternatives#heading–arm).

sudo vi /boot/firmware/nobtcmd.txt
# or for older Raspberry Pis (such as the 3+)
sudo vi /boot/firmware/cmdline.txt

The add the following line to the existing options:

cgroup_memory=1 cgroup_enable=memory

Then simply reboot your node and it will enable cgroups. You’re existing agent will then start correctly and be visible when running a kubectl get nodes

Enable legacy iptables on Raspian Buster

If you are using Raspbian Buster then ensure to enable legacy iptables as it defaults to nftables, which will not work correctly with K3s networking.

sudo iptables -F
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
sudo reboot

(https://rancher.com/docs/k3s/latest/en/advanced/)

Published by Kirk

I'm a technical lead working for NetApp & Microsoft on advanced data capabilities in Azure, primarily the Azure NetApp Files solution and use cases. I like to share everything I've learnt along the way in the hope that somewhere along the line, someone will find it useful.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: