Deploy a Stack to a Swarm
In a previous article, I deployed services in my ORDOID-C2 cluster using the Docker command line. It works but there must be a better way to do deployment especially when an application requires multiple components working together. Docker 1.13.x introduced the new docker stack deployment feature to allow deployment of a complete application stack to the swarm. A stack is a collection of services that make up an application. This new feature automatically deploys multiple services that are linked to each other obviating the need to define each one separately. In other words, docker-compose in swarm mode. To do this, I have to upgrade my Docker Engine from V1.12.6 that I installed using apt-get from the Ubuntu software repository to V1.13.x. Lucky for me, I’ve already built V1.13.1 on my ODROID-C2 myself months ago when I was experimenting unsuccessfully with swarm mode due to missing kernel modules in the OS I used at the time. See my previous article for more information on this. It is just a matter of upgrading all my ODROID-C2 nodes to V1.13.1 and I am in business.
The httpd-visualizer Stack
The first thing I did was to deploy the same applications (httpd and Visualizer) in my previous article using ‘docker stack deploy’. To do this, I need to create a yaml file. This is actually docker-compose yaml file version “3”. This is relative easy to do as data persistence is not required. Here is the yaml file:
version: "3" services: httpd: # simple httpd demo image: mrdreambot/arm64-busybox-httpd deploy: replicas: 3 restart_policy: condition: on-failure resources: limits: cpus: "0.1" memory: 20M ports: - "80:80" networks: - httpd-net visualizer: image: mrdreambot/arm64-docker-swarm-visualizer ports: - "8080:8080" volumes: - "/var/run/docker.sock:/var/run/docker.sock" deploy: placement: constraints: [node.role == manager] networks: - httpd-net networks: httpd-net:
Note that the use of “Networks” in the yaml file is not strictly necessary. If omitted, docker will create a default one as you will see in a later section. And the 2 applications do not need to talk to each other anyway!
To deploy it, just change to the directory where your yaml file is located and issue the command:
docker stack deploy -c simple-stacks.yml httpd-dsv
This creates a stack named httpd-dsv. You can find out regarding the state of the stack by issuing a number of stack commands as shown in the following screenshot.
You can point your browser to the swarm manager or any swarm node at port 8080 to visualize the deployment using the Visualizer.
Here is a screenshot of using the VuShell for that from a previous stack deployment:
To undeploy the stack, issue the command:
docker stack rm httpd-dsv
Migrating My WordPress Blog to the Swarm
To illustrate a more realistic stack deployment, I decided that a good test is to migrate my blog to the swarm. This is useful to me as that enables me to bring up my blog easily to another machine;-). To do this, I have to do some preparation work:
- Take a dump of the wordpress database using mysqldump named mysql.dmp.
- Use a text editor to replace all references of my domain name (mrdreambot.ddns.net) in the dmp file with the swarm manager’s IP which is 192.168.1.100.
- Tar up /var/www/html directory which contains scripts and uploaded assets
- Pick the docker images to use: mrdreambot/arm64-mysql and arm64v8/wordpress.
Armed with the above, I can proceed to create a docker stack deployment for my blog.
State Persistence Using Bind-mount Volumes
The first approach I took was to use host directories as data volumes (also called bind-mount volumes) for data persistence. The yaml file is shown below:
version: '3' services: db: image: mrdreambot/arm64-mysql volumes: - /nfs/common/services/wordpress/db_data:/u01/my3306/data - /nfs/common/services/wordpress/db_root:/root environment: MYSQL_ROOT_PASSWORD: Password456 MYSQL_DATABASE: wordpress MYSQL_USER: wordpressuser MYSQL_PASSWORD: Password456 deploy: restart_policy: condition: on-failure placement: constraints: [node.role == manager] wordpress: depends_on: - db image: arm64v8/wordpress volumes: - /nfs/common/services/wordpress/www_src/html:/usr/src/wordpress - /nfs/common/services/wordpress/www_data:/var/www/html ports: - 80:80 environment: WORDPRESS_DB_HOST: db:3306 WORDPRESS_DB_USER: wordpressuser WORDPRESS_DB_PASSWORD: Password456 WORDPRESS_DB_NAME: wordpress deploy: replicas: 3 restart_policy: condition: on-failure placement: constraints: [node.role == manager]
The following is the screenshot for the stack deployment:
Note that the wordpress site has lost some of its customized look as arm64v8/wordpress does not provide any php customization or libraries.
As mentioned earlier, if you do not define Networks in your yaml file, docker creates a ‘wordpress_default’ overlay network for the deployment automatically. The overlay network is required such that wordpress can reference the MySQL database using its name ‘db’ in:
The data volumes used warrant some explanation. First thing to note is that all the host directories used as data volumes are NFS mounted and accessible to all swarm nodes.
The host directory /nfs/common/services/wordpress/db_data is an empty directory. It is mapped to the container’s /u01/my3306/data directory where the MySQL database is located. How its content is created will be described next.
I pre-populated the host directory /nfs/common/services/wordpress/db_root with 2 files:
- sh – the MySQL startup script which replaces the one located in the container’s /root directory. This script is the entry point to the MySQL container. I changed the script to look for the mysql.dmp file located also in /root. If it is there, import the dump file into MySQL ie, populating /u01/my3306/data. Otherwise do nothing in additional to the usual processing.
- dmp – the dump file of my Blog’s MySQL database
The changes in the run.sh file is shown below:
... DMP_FILE=/root/mysql.dmp ... if [ "$MYSQL_DATABASE" ]; then mysql -uroot -e "CREATE DATABASE IF NOT EXISTS \`$MYSQL_DATABASE\`" if [ -f "$DMP_FILE" ]; then mysql -uroot $MYSQL_DATABASE < $DMP_FILE fi fi ...
Note that this is required only when you run the container for the first time. Subsequent deployment will not require this volume mapping. This means that you can comment out this line in the yaml file after successfully deploying this stack once as follows:
# – /nfs/common/services/wordpress/db_root:/root
arm64v8/wordpress initializes WordPress by copying the contents in its /usr/src/wordpress directory to its /var/www/html directory on startup if /var/www/html has no content. By pre-populating the host directory /nfs/common/services/wordpress/www_src/html with the untar’ed content from the tar file created earlier, arm64v8/wordpress will initialize WordPress with my Blog’s content. This is required only when you run the container for the first time. This means that you can comment out this line in the yaml file after successfully deploying this stack once as follows:
# - /nfs/common/services/wordpress/www_src/html:/usr/src/wordpress
The host directory /nfs/common/services/wordpress/www_data is an empty directory whose content will be initialized by the arm64v8/wordpress script as described above.
Why not use docker-compose?
You may be wondering why I did not use docker-compose to run the yaml file, for example, using once-off commands as the docker documentation suggests? The reason for it is that the docker-compose I installed is version 1.8.0 which does not understand docker-compose yaml file version 3 which is required for ‘docker stack deploy’! I tried to build a lter version of docker-compose from source without success. This is the reason I am not using docker-compose.
State Persistence Using Shared-storage Volumes
Using bind-mount volumes is host-dependent. Use of shared volumes has the benefit of being host-independent. A shared volume can be made available on any host that a container is started on as long as it has access to the shared storage backend, and has the proper volume plugin (driver) installed that allow you to use different storage backends such as: Amazon EC2, GCE, Isilon, ScaleIO, Glusterfs, just to name a few. There are lots of volume plugins or drivers available such as Flocker, Rex-Ray, etc. Unfortunately, no binaries for those plugins are available for ARM64 machines such as ODROID-C2. Fortunately, the inbuilt ‘local’ driver supports NFS. And it is the driver I am using for shared volume deployment.
The yaml file for this is shown below:
version: '3' services: db: image: mrdreambot/arm64-mysql volumes: - db_data:/u01/my3306/data - /nfs/common/services/wordpress/db_root:/root environment: MYSQL_ROOT_PASSWORD: Password456 MYSQL_DATABASE: wordpress MYSQL_USER: wordpressuser MYSQL_PASSWORD: Password456 deploy: placement: constraints: [node.role == manager] replicas: 1 restart_policy: condition: on-failure wordpress: depends_on: - db image: arm64v8/wordpress volumes: - /nfs/common/services/wordpress/www_src/html:/usr/src/wordpress - www_html:/var/www/html ports: - "80:80" environment: WORDPRESS_DB_HOST: db:3306 WORDPRESS_DB_USER: wordpressuser WORDPRESS_DB_PASSWORD: Password456 WORDPRESS_DB_NAME: wordpress deploy: placement: constraints: [node.role == manager] replicas: 3 restart_policy: condition: on-failure volumes: db_data: external: name: db_data www_html: external: name: www_html
Again, the volumes warrant some explanation:
It serves the same purpose as in the bind-mount volume section. It is needed only when you run the stack for the first time to initialize the MySQL database.
It serves the same purpose as in the bind-mount volume section. It is needed only when you run the stack for the first time to initialize the wordpress content.
db_data is a shared volume created outside of the stack deployment meaning it is created before the yaml file is deployed. It is used to store the MySQL database content and is uninitialized on creation.
www_html is a shared volume created outside of the stack deployment meaning it is created before the yaml file is deployed. It is used to store the wordpress content and is uninitialized on creation.
Creating the Shared Volumes
You have probably noticed the section in the yaml file that reads:
volumes: db_data: external: name: db_data www_html: external: name: www_html
The db_data and www_html shared volumes are created using the commands:
docker volume create --driver local \ --opt type=nfs \ --opt o=addr=192.168.1.100,rw \ --opt device=:/media/sata/nfsshare/db_data \ db_data docker volume create --driver local \ --opt type=nfs \ --opt o=addr=192.168.1.100,rw \ --opt device=:/media/sata/nfsshare/www_html \ www_html
The directories /media/sata/nfsshare/db_data and /media/sata/nfsshare/www_htm must exist before you create the volumes. My /etc/exports file has an entry:
To prove that the shared volumes work, I initially deployed only 1 mySQL and 1 WordPress replica on the Docker manager and let them initialize the shared volumes.
Then I commented out:
# placement: # constraints: [node.role == manager]
for worpress and
the 2 bind-mount volumes:
# - /nfs/common/services/wordpress/db_root:/root # - /nfs/common/services/wordpress/www_src/html:/usr/src/wordpress
Next I want to deploy 3 replicas of wordpress on multiple nodes.
Since we are using the ‘local’ driver, we have to create the volumes on each node. In the screenshot below, I am using “parallel ssh” to create them on all nodes using just 2 commands. They are followed by the volume and the stack deployment:
I checked that all replicas are using the shared volumes by ‘docker exec -it’ into the wordpress containers on the nodes they were running and examining the /var/www/html directory. Everything works.
Under the covers, both approaches use NFS for sharing among the nodes. However, shared volumes provide a higher host-independent abstraction than bind-mount volumes. Potentially, you can recreate the shared volumes using other storage backends other than NFS such as AWS EC2 or Glusterfs. Bibd-mount is tied to your host file system.
I learned something new exploring the use of ‘docker stack deploy’. I hope you’ll find this article useful and informative. During my experimentation with Docker on ODROID-C2 in the past months, I’ve come to the conclusion that ARM64 is still not mainstream in the server market. Consequently, all open source projects such as docker, docker volume drivers, kubernetes, openshift only provide pre-built binaries for the INTEL 64 bit platform. For arm64, they are either non-existent or several versions back meaning that you cannot use them to explore new features. If we want this to change this, we need to have a strong arm64 community. To have a strong community, we need strength in number which, I am afraid, we don’t have at the moment! I’d like to hear from you how we can change this – TOGETHER!