Taking a break from pure Azure deployments, I decided to try my hand at building a small Kubernetes cluster that would allow me to learn and develop applications quickly at home without burning any of my Azure credits. Of course I could easily deploy on an AKS cluster, but I want to learn more about the management clusters in Kubernetes in order to fully understand the benefits any managed solution brings (sometimes it’s best to understand the pain points first hand to truly see the value in a service/solution).
In order to build my personal Kubernetes cluster, I decided to invest in four of the latest Raspberry Pi’s with the largest RAM option available (Raspberry Pi 4 – 4GB). In addition to these, I happened to have a Raspberry Pi 3+ lying around not doing much, so that will become the master/API node to the cluster, leaving the beefier RPi4 available for use as the worker nodes.
In order to build my Kubernetes cluster I had to make two decisions:
- OS Distribution – this was between Raspbian and Ubuntu. I chose Ubuntu as I have used it in enterprise environments and have experience with it.
- Kubernetes tooling – this was between microk8s (https://microk8s.io/) and K3s (https://rancher.com/). I opted for K3s as it was simply the first set of tutorials I came across – I would like to give microK8s a go in future. Both offer optimisation for edge environments such as the Raspberry Pi.
Luckily the folks over at rancher have made this simple. Run the following command and you’ll have a single node K3s single node cluster up and running in no time.
curl -sfL https://get.k3s.io | sh -
Once this is installed you can check that your single node cluster is up and running with the following command – if this fails for you check the next section (additional configuration/troubleshooting)
kubectl get node
At this stage, start your other nodes and run the following command on them to install k3s and configure them to join the existing cluster:
curl -sfL https://get.k3s.io | K3S_URL=https://192.168.1.120:6443 K3S_TOKEN=K10373f030f773c58a476ca0332eda0beb8ef8ddfc6d3e1c909642939bc59c4d096::server:d2ec6a145a753bb676bb0034f71bb6c0 sh -
And that’s it! You now have a cluster up and running, ready for your next project! You can check your node status with the command sudo kubectl get nodes
If you liked this post then subscribe as I build upon this journey and start to connect more advanced use-cases and cloud integration to my projects!
I’m thinking of playing with Azure Arc and connecting it to my cluster to see what it is capable of and gaining experience with it. I also have a raft of projects (mainly cycling related) that I will deploy on this cluster! Keep tuned for more articles and videos!
When using the Raspberry Pi there are some additional configuration steps that must be performed in order to get K3s up and running successfully.
Ubuntu 20.04 LTS on Raspberry Pi
When I followed the above instructions, I hit a snag where no matter what I did, I couldn’t get the K3s service to start. After a bit of digging I found that a required flag for cgroups must be set on the Raspberry Pi. In order to do so follow these instructions (source: https://microk8s.io/docs/install-alternatives#heading–arm).
sudo vi /boot/firmware/nobtcmd.txt # or for older Raspberry Pis (such as the 3+) sudo vi /boot/firmware/cmdline.txt
The add the following line to the existing options:
Then simply reboot your node and it will enable cgroups. You’re existing agent will then start correctly and be visible when running a kubectl get nodes
Enable legacy iptables on Raspian Buster
If you are using Raspbian Buster then ensure to enable legacy iptables as it defaults to nftables, which will not work correctly with K3s networking.
sudo iptables -F sudo update-alternatives --set iptables /usr/sbin/iptables-legacy sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy sudo reboot