Kubernetes on Pi

Post Disclaimer

I am a total newbie when it comes to Docker and Kubernetes. I put this together to document what worked for me as there wasn’t one single source that covered everything.

Why?

I did this for the same reason that I have done most of my other posts — I want to learn something new. Docker and Kubernetes have been around for a while now, but I haven’t had a use case for them at work, so I need to do it on my time.

Hardware List

Component Count
Raspberry Pi 3 Model B Motherboard 6
Samsung 64GB 80MB/s EVO Select Micro SDXC Memory Card (MB-ME64DA/AM) 6
Anker 60W 6-Port USB Wall Charger 1
D-Link DGS-2208 8-Port 10/100/1000 Desktop Switch 1
CAT 6 Flat Ethernet Cable (6 Pack) 1
Anker 8-in-1 USB 3.0 Portable Card Reader 1

Provisioning

I’ve done my provisioning only on a Windows computer, so this will cover only that scenario.

Requirements

Win32 Disk Imager (Download and Install)
Hypriot Docker Image for Raspberry Pi (Download and Extract)

Process

Run Win32 Disk Imager as an Administrator
Do the following for each of the SD cards you wish to use in your cluster:

  1. Place the SD card into a card reader (like the Anker 8-in-1 USB 3.0 Portable Card Reader).
  2. Plug the card reader into your computer.
  3. Using Win32 Disk Imager, browse to the image extracted from the Hypriot Docker Image for Raspberry Pi download.
  4. Select the drive letter representing the SD card that you wish to flash.
  5. Press the “Write” button and wait for the process to complete.
  6. Once the process is done, go to the driver that is labeled “boot.”
  7. Open the “user-data” file in your favorite text editor (Note: This file is only referenced ONCE — when the Raspberry Pi does the FIRST boot).
    1. Change the hostname value to something more reasonable (I used node01 through node06).
    2. Change the user data to something more reasonable (I used my first name and a random password, same for all nodes).
    3. Browse through the file and see if there are any other changes to make. I also modified:
      1. locale
      2. timezone
      3. package_update
      4. package_upgrade
      5. package_reboot_if_required
  8. Save the file if you had made any modifications.
  9. Unmount the SD card.
  10. Unplug the card reader.
  11. Remove the SD card from the card reader.
  12. Place the SD card into the Raspberry Pi.

Hooking Everything Up

Now that we have all of the SD cards provisioned, we can start wiring things up.

  1. Make sure the network switch you wish to use is plugged in and has access to the internet.
  2. Using the Ethernet cables, plug each device into the switch.
  3. Plug each Raspberry Pi into the Anker 60W 6-Port USB Wall Charger.
  4. Give each Raspberry Pi a few minutes to boot up and run it’s updates (I gave each one around 5 minutes). You can hook them up to monitors to track progress if you so desire.

Software Setup

SSH into each of the nodes. Once authenticated, run the following commands on each node to install the tools necessary to set up and manage a Kubernetes cluster:

sudo su -
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list
apt-get update
apt-get install -y kubeadm

Designate one of your nodes as the master node. I used node01.

SSH into your master node. Once authenticated, run the following commands to initiate the Kubernetes cluster:

sudo su -
kubeadm init --pod-network-cidr 10.244.0.0/16 --token-ttl=0

If that process fails to complete (which it did for me a few times), you can run the following command to undo everything (and then try again of course 🙂 ).

kubeadm reset

Once you are able to run that process without any errors, it will output some commands to run, to have other nodes join the cluster. Keep note of what those exact commands are (it will be something like “kubeadm join –token …”). Run the following command to return to the user you had logged in as:

exit

If you are using DHCP, scroll down to “Changes for DHCP” and apply the changes to the master node.

As a normal user, run the following commands to set up the local configuration file for managing your Kubernetes cluster:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
echo "export KUBECONFIG=${HOME}/.kube/config" >> ~/.bashrc
source ~/.bashrc

We are now to the point of having a master node setup and we can use kubectl to manage it.

Network Software Setup

I could never get flannel to run properly on my cluster. I always got an error that I couldn’t get past. But I was able to set up weave.

I’m not 1000% sure that the following is required, but I did it anyway. It doesn’t hurt, right? 🙂

On the master node, run the following command to download weave and setup weave:

sudo curl -L git.io/weave -o /usr/local/bin/weave
sudo chmod a+x /usr/local/bin/weave

Once done, let’s setup weave for use with Kubernetes. To do that, run the following commands:

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

Once that has executed without any issues, I waited a few minutes to let things sink in, then I issued a reboot …

sudo reboot

Pod Setup

Pods are, for the most part, worker nodes. Each pod has to be associated to the master node.

This is where the command I told you to jot down earlier comes into play.

On each of your worker nodes (again, all of them that are not the master node), run the following commands to associate the worker node with the cluster:

sudo su -
PUT THE COMMAND YOUR WROTE DOWN HERE

All of your worker nodes should now be associated with the master node.

If you are using DHCP, scroll down to “Changes for DHCP” and apply the changes to the worker nodes.

kubectl

kubectl is the application used to control your Kubernetes cluster.

Common Commands

Command Description Example(s)
describe Display the detailed state of one or more resources.
  • kubectl describe nodes
  • kubectl describe node node01
apply Apply a configuration change to a resource from a file or stdin.
  • kubectl apply -f https://goo.gl/jT8oto
  • kubectl apply -f https://goo.gl/E3kQ8k
delete Delete resources either from a file, stdin, or specifying label selectors, names, resource selectors, or resources.
  • kubectl delete -f https://goo.gl/E3kQ8k
  • kubectl delete -f https://goo.gl/jT8oto
run Run a specified image on the cluster.
  • kubectl run hypriot –image=hypriot/rpi-busybox-httpd –replicas=3 –port=80
expose Expose a replication controller, service, or pod as a new Kubernetes service.
  • kubectl expose deployment hypriot –port 80

Example of Running an Image

In this example we will setup traefik to act as a reverse proxy and load balancer. This will sit in front of the rpi-busybox-httpd application.

Setup traefik

On the master node, run the following commands to setup traefik as a DaemonSet on your Kubernetes cluster.

kubectl apply -f https://goo.gl/jT8oto
kubectl apply -f https://goo.gl/E3kQ8k

By default, those commands will setup traefik on each of your nodes.

Deploy rpi-busybox-httpd

On the master node, run the following commands to deploy the rpi-busybox-httpd application to three nodes and expose port 80 to the weave network (this will create a service and deployment called “hypriot“):

kubectl run hypriot --image=hypriot/rpi-busybox-httpd --replicas=3 --port=80
kubectl expose deployment hypriot --port 80

Give your cluster a few minutes to download and start the image on your cluster.

To view the endpoints of each of the deployed instances of the application, run the following command on your master node:

kubectl get endpoints hypriot

Setup Ingress for hypriot

The following commands will allow us to route requests coming into our traefik service to the hypriot service on our cluster.

cat > hypriot-ingress.yaml <<EOF
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hypriot
spec:
  rules:
  - host: hypriot-demo
    http:
      paths:
      - path: /
        backend:
          serviceName: hypriot
          servicePort: 80
EOF
kubectl apply -f hypriot-ingress.yaml

Accessing hypriot

In the previous set of commands, we set up Ingress for our hypriot service. It access requests coming into the host name “hypriot-demo.”

To access the load balanced application, you will either need to update your DNS server to add a record for hypriot-demo pointing to one of your nodes (either by CNAME or A).

Or you could update your machines hosts file to add a record. If you do not know the IP address of one of your nodes, SSH into it and run ifconfig. Look for eth0 and take note of the “inet addr” value.

In your browser, you should now be able to access http://hypriot-demo/.

traefik User Interface

The following commands will allow us to access the traefik user interface:

cat > ui.yaml <<EOF
---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web
EOF
kubectl apply -f ui.yaml

To access the user interface, you will need to do what you’ve done for hypriot-demo in terms of the DNS or hosts file change, but rather than setting one up for “hypriot-demo”, set one up for “traefik.”

In your browser, you should now be able to access http://traefik/.

Undoing What We’ve Done

Now that you’ve seen how to setup an application and have it load balanced, we can remove the example as to not take up any needed resources.

The following commands will remove the Ingress items we have set up for hypriot:

kubectl delete -f hypriot-ingress.yaml

The following commands will remove the hypriot deployment and service:

kubectl delete deployment hypriot
kubectl delete services hypriot

The following commands will remove traefik:

kubectl delete -f ui.yaml
kubectl delete -f https://goo.gl/E3kQ8k
kubectl delete -f https://goo.gl/jT8oto

Changes for DHCP

You might want to update a few configuration (.conf) files to reference the host name rather than IP address, otherwise you’ll run into some issues if the IP address ever changes.

To do that, as a superuser, update each .conf file in the /etc/kubernetes/ directory. Where you see the IP address of the master node, update it to the name of the master node (again, for me, this was node01).

🙂

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.