H4Z-Blog

Raspberry PI docker swarm

2 min read

Hardware:
pi1: rpi5 8gb
pi2/3: rpi4 4gb

Install raspberrypi os 64bit lite onto each pi and ensure setup keys are enabled

Setting up docker

setup and install docker and update OS and reboot:

sudo apt update && sudo apt upgrade -y && sudo apt-get update && sudo apt-get install ca-certificates curl && sudo install -m 0755 -d /etc/apt/keyrings && sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc && sudo chmod a+r /etc/apt/keyrings/docker.asc && echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \ 
&& sudo tee /etc/apt/sources.list.d/docker.list > /dev/null && sudo apt-get update && sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin && sudo groupadd docker && sudo usermod -aG docker $USER && sudo reboot

Once installed need to configure and setup docker swarm
on master node

docker swarm init --advertise-addr 192.168.1.21

on all other nodes:

docker swarm join --token {TOKEN} {internal address}

to get the token again from the master:

docker swarm join-token worker

or to add another manager

docker swarm join-token manager

Setting up shared storage

I have a nfs drive on my truenas instance
First mark the mount location
mkdir -p /nfs/cluster
Then to connect that to the pi’s add the following line to /etc/fstab
192.168.1.5:mnt/firefly/Cluster /nfs/cluster nfs defaults 0 0
Followed by rebooting the pi and checking if it’s attached
You also need to make sure that the nodes have access and the correct permissions

Deploying services with swarmcd

For raspberrypi image needs to be built locally from:
https://github.com/h4z-dev/swarm-cd.git
using
docker build -t git.berglot.net/h4z/swarmcd:arm64
Then creating and configuring the following files on the nfs drive
/nfs/cluster/SwarmCD/repos.yaml

# repos.yaml
bootstrap:
  url: "https://github.com/h4z-dev/production-swarm-mirror.git"
  username: h4z-dev
  password: {PASSWORD}
production:
  url: "https://git.berglot.net/h4z-dev/production-swarm.git"
testing:
  url: "https://git.berglot.net/h4z-dev/testing-swarm.git"

Then we define the stack in /nfs/cluster/SwarmCD/stacks.yaml

# stacks.yaml
homepage:
  repo: production
  branch: main
  compose_file: homepage/compose.yaml
gitea:
  repo: bootstrap
  branch: main
  compose_file: gitea/compose.yaml

And finally, we deploy SwarmCD to the cluster using the following docker-compose file:
/nfs/cluster/SwarmCD/docker-compose.yaml

# docker-compose.yaml
version: '3.7'
services:
  swarm-cd:
    image: git.berglot.net/h4z/swarmcd:arm64
    deploy:
      placement:
        constraints:
          - node.role == manager
    ports:
      - 4001:8080
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /nfs/cluster/SwarmCD/repos.yaml:/app/repos.yaml:ro
      - /nfs/cluster/SwarmCD/stacks.yaml:/app/stacks.yaml:ro

to enable simply run
docker stack deploy --compose-file docker-compose.yaml swarm-cd

Yes there is a circular dependency between my gitea and where the arm image is hosted

I have made a pull request to update the swarmcd image so hopefully I can pull from docker hub in the future

Graph View