Patrick Scott [GDD] Clusters (Part 1 of 2) 02 December 2018

So…
 
Ready to run a cluster with an orchestrator?
 
 
Let's run a cluster!
 
 
Before we run anything on AWS though, we will set up a cluster locally using Virtual Machines to demonstrate the techniques before there is any money involved.
 
 
First, install VirtualBox and Docker For Mac if you have not already. Both are available on [Homebrew-Cask](https://caskroom.github.io/):
 
# "tap" homebrew cask if you have not
brew tap caskroom/cask
 
 
# install docker and virtualbox
brew cask install docker
brew cask install virtualbox
Btw, I have these installed as part of my mac development environment setup script:

 
 
And finally, let’s set up a Docker Swarm on our local machine.
 
 
I have some scripts available to make the process easier.
 
 
Go ahead and git clone [email protected]:unbounded-tech/scripts.git

Let's take a look:
 
 
> git clone [email protected]:unbounded-tech/scripts.git
 
> cd scripts
 
> ls
docker-swarm-kill.sh docker-swarm-large.sh docker-swarm-small.sh
 
Using "ls" to list the directory's contents revealed three files...
 
Let’s start by checking out `docker-swarm-small.sh` to see what running it will do
 
#!/usr/bin/env bash

 
# Create nodes
for i in 1 2; do
  docker-machine create -d virtualbox node-$i
done
 
 
NODE1_IP=$(docker-machine ip node-1)
 
# Initialize the swarm cluster, and set up manager on node-1
 
eval $(docker-machine env node-1)
 
docker swarm init --advertise-addr $NODE1_IP
 
 
# Generate a worker token for joining the cluster
WORKER_TOKEN=$(docker swarm join-token -q worker)
 
 
# Join the next node as a worker
for i in 2; do
  eval $(docker-machine env node-$i)
  docker swarm join --token $WORKER_TOKEN $NODE1_IP:2377
done
 
 
echo ">> The swarm cluster is up and running"
 
 
# Print our instructions
docker-machine env node-1
 
It’s pretty simple - if you are not familiar with bash scripting it might seem a bit foreign, but essentially, it is just creating two machines provisioned with virtualbox. The first node does the initialization process, making it a manager node, and the second requests to join as a worker.

 
I think the comments are sufficient to explain what is going on inside.
 
 
As I mentioned, two machines would not be sufficient for serious production workloads, but it’s ok for our needs of running a small example cluster.
 
 
The last line of the script prints out some instructions.
 
 
Let’s give it a try.
 
sh docker-swarm-small.sh
Running pre-create checks...
Creating machine...
(node-1) Copying /Users/patrickscott/.docker/machine/cache/boot2docker.iso to /Users/patrickscott/.docker/machine/machines/node-1/boot2docker.iso...
(node-1) Creating VirtualBox VM...
(node-1) Creating SSH key...
(node-1) Starting the VM...
(node-1) Check network to re-create if needed...
(node-1) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env node-1
Running pre-create checks...
Creating machine...
(node-2) Copying 

...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env node-2
Swarm initialized: current node (mac1vviy48lfjsqo4kv65d6yp) is now a manager.
 
To add a worker to this swarm, run the following command:
 
    docker swarm join --token SWMTKN-1-1m4ammwlcyw36ulgvemydqipaou85sqqyb7cb2k9bnxfp3g8gm-2le18s5qn3w3br6nux7ndg6ng 192.168.99.100:2377
 
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
 
This node joined a swarm as a worker.
 
>> The swarm cluster is up and running
 
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/Users/patrickscott/.docker/machine/machines/node-1"
export DOCKER_MACHINE_NAME="node-1"
 
# Run this command to configure your shell:
# eval $(docker-machine env node-1)
There’s a lot of output from virtualbox at the top as the two nodes are provisioned.

 
Then, when the manager node is created we see a “Swarm initialized” message, telling us how we can add workers to the swarm.
 
 
The script then does just that, by joining node-2 as a worker. The last line of output tells us to configure our shell we need to run a command - let’s play along:
 
eval $(docker-machine env node-1)
 
 
All this line does is set some environment variables in our shell that are used by docker to know which host to control.
 
 
Now, when we run `docker` commands, it will be against the local swarm we just created. (You can run the command "env" to see.)
 
 
Try it out by running `docker node ls`
 
➜ scripts git:(master) docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
mac1vviy48lfjsqo4kv65d6yp * node-1 Ready Active Leader 18.03.1-ce
jopupigfzxrvcr1pvd4e3t5ep node-2 Ready Active 18.03.1-ce
 
 
We can see the two nodes that we just created listed here.

 
Congratulations! You are now running a cluster! 🎉
 
 
Even though it looks essentially the same as our docker-compose files, cause it is, when we create it for production, we call it a "stack".
 
So the "stack" is the service, and maybe it's dependencies like a database or a proxy configured for production.
 
Tomorrow, we’ll explore how to deploy our “stack” we’ve been working on to the cluster.
 
 
For now, you can stop the cluster, we’ll pick back up where we left off tomorrow.
 
 
Run `sh docker-swarm-kill.sh` to stop the cluster.
 
sh docker-swarm-kill.sh
 
WARNING: This will stop all running docker-machine's!
 
Below are your running machines:
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
node-1 * virtualbox Running tcp://192.168.99.100:2376 v18.03.1-ce
node-2 - virtualbox Running tcp://192.168.99.101:2376 v18.03.1-ce
Are you sure you wish to continue? [y/n] y
Stopping "node-1"...
Stopping "node-2"...
Machine "node-2" was stopped.
Machine "node-1" was stopped.
 
Patrick “Swarm Killer” Scott
P. S. Btw, this was actually MORE difficult than running a Docker Swarm in production, believe it or not. :)