Kubernetes cluster on single board computers

I want to start off by saying i'm new to kubernetes so this is what I've done to get it working in my cluster. A lot of commands in the post I got from this post by Hypriot.

Setting up the Master Node

My master node is a Rock64 by Pine64 a nice little computer. I installed there modified version of ubuntu called "Ubuntu 18.04 Bionic minimal 64bit Image [microSD / eMMC Boot]" as the os im using an microSD card right now.

Install docker

First lets install docker. Below is my favorite command to get it installed on a Linux machine.

$ curl -L http://get.docker.com | sudo sh

Configure docker

Now lets configure docker to it uses the cgroup driver systemd. These commands came from kubernetes cri installation documentation.

Modify the docker daemon.json file to have the following lines. Its usually located at /etc/docker/daemon.json

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}

Make the docker.service.d directory

$ sudo mkdir -p /etc/systemd/system/docker.service.d

Restart Docker

$ sudo systemctl daemon-reload
$ sudo systemctl restart docker

Disable Swap

For me swap was already setup on ubuntu so I had to disable it and uninstall zram-config so it would not be enabled during boot.

$ sudo swapoff -a
$ sudo apt remove zram-config

Install Kubernetes

Now we need to add the APT for the kubernetes repository. You'll notice it says kubernetes-xenial not kubernetes-bionic the reason being kubeadm was not in the bionic branch but it was in the xenial branch.

$ sudo bash
# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
# echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list

Update and install kubernetes

# apt-get update
# apt-get install -y kubeadm

Initialize The Cluster

Lets initialize the cluster now. Note the ipaddress/subnet definition it appears to be needed for flannel.

$ sudo kubeadm init --pod-network-cidr 10.244.0.0/16

Wait for it to finish. Copy the join command or you'll need to generate another token. Copy the config to your home directory.

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

Now lets install flannel so our cluster can come online.

$ curl -sSL https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml | kubectl apply -f -

Use this command to see wait for your master node to come online It will say its Ready. Then you can proceed to the other nodes.

$ kubectl get nodes

Setup Other Nodes

My other nodes are raspberry pis running Raspbian Stretch Lite.

Install docker

First lets install docker. Using the same command as with the master node.

$ curl -L http://get.docker.com | sudo sh

Configure docker

Now lets configure docker to it uses the cgroup driver systemd. These commands came from kubernetes cri installation documentation.

Modify the docker daemon.json file to have the following lines. Its usually located at /etc/docker/daemon.json

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}

Make the docker.service.d directory

$ sudo mkdir -p /etc/systemd/system/docker.service.d

Restart Docker

$ sudo systemctl daemon-reload
$ sudo systemctl restart docker

Disable Swap

For me swap was already setup on raspbian but the below command worked to disable it.

$ sudo sync
$ sudo swapoff -a
$ sudo apt-get purge -y dphys-swapfile
$ sudo rm /var/swap
$ sudo sync

Enable cgroup memory

Edit /boot/cmdline.txt with your favorite text editor add the below text before waitroot. Be sure to reboot after adding this so it can take affect.

cgroup_enable=memory

Install Kubernetes

Now we need to add the APT for the kubernetes repository.

$ sudo bash
# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
# echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list

Update and install kubernetes

# apt-get update
# apt-get install -y kubeadm

Initialize The Node

Run the join command you get when you initialized the master node it will look something like below

$ sudo kubeadm join 192.168.0.2:6443 --token aaaaaa.bbbbccccddddffff --discovery-token-ca-cert-hash sha256:57796aebb75426e4141c9f809607e3ec69b8a41174efffe5e283b24b13525f66

You may need to reboot to get the node to show as ready on the master.

This is what worked for me hopfully its helped you out as well.