wordpress-stacks-volume-yaml

Using Docker Stack Deploy on my ODROID-C2 Cluster

 

Deploy a Stack to a Swarm

In a previous article, I deployed services in my ORDOID-C2 cluster using the Docker command line. It works but there must be a better way to do deployment especially when an application requires multiple components working together. Docker 1.13.x introduced the new docker stack deployment feature to allow deployment of a complete application stack to the swarm. A stack is a collection of services that make up an application. This new feature automatically deploys multiple services that are linked to each other obviating the need to define each one separately. In other words, docker-compose in swarm mode. To do this, I have to upgrade my Docker Engine from V1.12.6 that I installed using apt-get from the Ubuntu software repository to V1.13.x. Lucky for me, I’ve already built V1.13.1 on my ODROID-C2 myself months ago when I was experimenting unsuccessfully with swarm mode due to missing kernel modules in the OS I used at the time. See my previous article for more information on this. It is just a matter of upgrading all my ODROID-C2 nodes to V1.13.1 and I am in business.

 

The httpd-visualizer Stack

The first thing I did was to deploy the same applications (httpd and Visualizer) in my previous article using ‘docker stack deploy’. To do this, I need to create a yaml file. This is actually docker-compose yaml file version “3”. This is relative easy to do as data  persistence is not required. Here is the yaml file:

version: "3"
services:
  httpd:
    # simple httpd demo
    image: mrdreambot/arm64-busybox-httpd
    deploy:
      replicas: 3
      restart_policy:
        condition: on-failure
      resources:
        limits:
          cpus: "0.1"
          memory: 20M
    ports:
      - "80:80"
    networks:
      - httpd-net
  visualizer:
    image: mrdreambot/arm64-docker-swarm-visualizer
    ports:
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
    deploy:
      placement:
        constraints: [node.role == manager]
    networks:
      - httpd-net
networks:
  httpd-net:

Note that the use of “Networks” in the yaml file is not strictly necessary. If omitted, docker will create a default one as you will see in a later section. And the 2 applications do not need to talk to each other anyway!

To deploy it, just change to the directory where your yaml file is located and issue the command:

docker stack deploy -c simple-stacks.yml httpd-dsv

This creates a stack named httpd-dsv. You can find out regarding the state of the stack by issuing a number of stack commands as shown in the following screenshot.

httpd-dsv-stack-commands
httpd-dsv-stack-commands

You can point your browser to the swarm manager or any swarm node at port 8080 to visualize the deployment using the Visualizer.

Here is a screenshot of using the VuShell for that from a previous stack deployment:

vushell-visualizer
vushell-visualizer

To undeploy the stack, issue the command:

docker stack rm httpd-dsv

Migrating My WordPress Blog to the Swarm

To illustrate a more realistic stack deployment, I decided that a good test is to migrate my blog to the swarm.  This is useful to me as that enables me to bring up my blog easily to another machine;-). To do this, I have to do some preparation work:

  1. Take a dump of the wordpress database using mysqldump named mysql.dmp.
  2. Use a text editor to replace all references of my domain name (mrdreambot.ddns.net) in the dmp file with the swarm manager’s IP which is 192.168.1.100.
  3. Tar up /var/www/html directory which contains scripts and uploaded assets
  4. Pick the docker images to use: mrdreambot/arm64-mysql and arm64v8/wordpress.

Armed with the above, I can proceed to create a docker stack deployment for my blog.

State Persistence Using Bind-mount Volumes

The first approach I took was to use host directories as data volumes (also called bind-mount volumes) for data persistence. The yaml file is shown below:

version: '3'

services:
   db:
     image: mrdreambot/arm64-mysql
     volumes:
       - /nfs/common/services/wordpress/db_data:/u01/my3306/data
       - /nfs/common/services/wordpress/db_root:/root
     environment:
       MYSQL_ROOT_PASSWORD: Password456
       MYSQL_DATABASE: wordpress
       MYSQL_USER: wordpressuser
       MYSQL_PASSWORD: Password456
     deploy:
       restart_policy:
         condition: on-failure
       placement:
         constraints: [node.role == manager]  
   wordpress:
     depends_on:
       - db
     image: arm64v8/wordpress
     volumes:
       - /nfs/common/services/wordpress/www_src/html:/usr/src/wordpress
       - /nfs/common/services/wordpress/www_data:/var/www/html
     ports:
       - 80:80
     environment:
       WORDPRESS_DB_HOST: db:3306
       WORDPRESS_DB_USER: wordpressuser
       WORDPRESS_DB_PASSWORD: Password456
       WORDPRESS_DB_NAME: wordpress
     deploy:
       replicas: 3
       restart_policy:
         condition: on-failure
              placement:
         constraints: [node.role == manager]

The following is the screenshot for the stack deployment:

wordpress-bind-mount-volume-deployment
wordpress-bind-mount-volume-deployment

 

wordpress-running
wordpress-running

Note that the wordpress site has lost some of its customized look as arm64v8/wordpress does not provide any php customization or libraries.

As mentioned earlier, if you do not define Networks in your yaml file, docker creates a ‘wordpress_default’ overlay network for the deployment automatically. The overlay network is required such that wordpress can reference the MySQL database using its name ‘db’ in:

WORDPRESS_DB_HOST: db:3306

The data volumes used warrant some explanation. First thing to note is that all the host directories used as data volumes are NFS mounted and accessible to all swarm nodes.

/nfs/common/services/wordpress/db_data:/u01/my3306/data

The host directory /nfs/common/services/wordpress/db_data is an empty directory. It is mapped to the container’s /u01/my3306/data directory where the MySQL database is located. How its content is created will be described next.

/nfs/common/services/wordpress/db_root:/root

I pre-populated the host directory /nfs/common/services/wordpress/db_root with 2 files:

  • sh – the MySQL startup script which replaces the one located in the container’s /root directory. This script is the entry point to the MySQL container. I changed the script to look for the mysql.dmp file located also in /root. If it is there, import the dump file into MySQL ie, populating /u01/my3306/data. Otherwise do nothing in additional to the usual processing.
  • dmp – the dump file of my Blog’s MySQL database

The changes in the run.sh file is shown below:

...

DMP_FILE=/root/mysql.dmp

...

if [ "$MYSQL_DATABASE" ]; then

  mysql -uroot -e "CREATE DATABASE IF NOT EXISTS \`$MYSQL_DATABASE\`"

  if [ -f "$DMP_FILE" ]; then

    mysql -uroot $MYSQL_DATABASE < $DMP_FILE

  fi

fi

...

Note that this is required only when you run the container for the first time. Subsequent deployment will not require this volume mapping. This means that you can comment out this line in the yaml file after successfully deploying this stack once as follows:

#      – /nfs/common/services/wordpress/db_root:/root

/nfs/common/services/wordpress/www_src/html:/usr/src/wordpress

arm64v8/wordpress initializes WordPress by copying the contents in its /usr/src/wordpress directory to its /var/www/html directory on startup if /var/www/html has no content. By pre-populating the host directory /nfs/common/services/wordpress/www_src/html with the untar’ed content from the tar file created earlier, arm64v8/wordpress will initialize WordPress with my Blog’s content. This is required only when you run the container for the first time. This means that you can comment out this line in the yaml file after successfully deploying this stack once as follows:

#      - /nfs/common/services/wordpress/www_src/html:/usr/src/wordpress

/nfs/common/services/wordpress/www_data:/var/www/html

The host directory /nfs/common/services/wordpress/www_data is an empty directory whose content will be initialized by the arm64v8/wordpress script as described above.

Why not use docker-compose?

You may be wondering why I did not use docker-compose to run the yaml file, for example, using once-off commands as the docker documentation suggests? The reason for it is that the docker-compose I installed is version 1.8.0 which does not understand docker-compose yaml file version 3 which is required for ‘docker stack deploy’! I tried to build a lter version of docker-compose from source without success. This is the reason I am not using docker-compose.

State Persistence Using Shared-storage Volumes

Using bind-mount volumes is host-dependent. Use of shared volumes has the benefit of being host-independent. A shared volume can be made available on any host that a container is started on as long as it has access to the shared storage backend, and has the proper volume plugin (driver) installed that allow you to use different storage backends such as: Amazon EC2, GCE, Isilon, ScaleIO, Glusterfs, just to name a few. There are lots of volume plugins or drivers available such as Flocker, Rex-Ray, etc. Unfortunately, no binaries for those plugins are available for ARM64 machines such as ODROID-C2. Fortunately, the inbuilt ‘local’ driver supports NFS. And it is the driver I am using for shared volume deployment.

The yaml file for this is shown below:

version: '3'

services:
   db:
     image: mrdreambot/arm64-mysql
     volumes:
       - db_data:/u01/my3306/data
       - /nfs/common/services/wordpress/db_root:/root
     environment:
       MYSQL_ROOT_PASSWORD: Password456
       MYSQL_DATABASE: wordpress
       MYSQL_USER: wordpressuser
       MYSQL_PASSWORD: Password456
     deploy:
       placement:
         constraints: [node.role == manager]
       replicas: 1
       restart_policy:
         condition: on-failure
   wordpress:
     depends_on:
       - db
     image: arm64v8/wordpress
     volumes:
       - /nfs/common/services/wordpress/www_src/html:/usr/src/wordpress
       - www_html:/var/www/html
     ports:
       - "80:80"
     environment:
       WORDPRESS_DB_HOST: db:3306
       WORDPRESS_DB_USER: wordpressuser
       WORDPRESS_DB_PASSWORD: Password456
       WORDPRESS_DB_NAME: wordpress
     deploy:
       placement:
         constraints: [node.role == manager]
       replicas: 3
       restart_policy:
         condition: on-failure
volumes:
     db_data:
       external:
         name: db_data       
     www_html:
       external:
         name: www_html
         

Again, the volumes warrant some explanation:

/nfs/common/services/wordpress/db_root:/root

It serves the same purpose as in the bind-mount volume section. It is needed only when you run the stack for the first time to initialize the MySQL database.

/nfs/common/services/wordpress/www_src/html:/usr/src/wordpress

It serves the same purpose as in the bind-mount volume section. It is needed only when you run the stack for the first time to initialize the wordpress content.

db_data:/u01/my3306/data

db_data is a shared volume created outside of the stack deployment meaning it is created before the yaml file is deployed. It is used to store the MySQL database content and is uninitialized on creation.

www_html:/var/www/html

www_html is a shared volume created outside of the stack deployment meaning it is created before the yaml file is deployed. It is used to store the wordpress content and is uninitialized on creation.

Creating the Shared Volumes

You have probably noticed the section in the yaml file that reads:

volumes:

  db_data:

    external:

      name: db_data

  www_html:

    external:

      name: www_html

The db_data and www_html shared volumes are created using the commands:

docker volume create --driver local \

--opt type=nfs \

--opt o=addr=192.168.1.100,rw \

--opt device=:/media/sata/nfsshare/db_data \

db_data

 

docker volume create --driver local \

--opt type=nfs \

--opt o=addr=192.168.1.100,rw \

--opt device=:/media/sata/nfsshare/www_html \

www_html

The directories /media/sata/nfsshare/db_data and /media/sata/nfsshare/www_htm must exist before you create the volumes. My /etc/exports file has an entry:

/media/sata/nfsshare 192.168.1.0/255.255.255.0(rw,sync,no_root_squash,no_subtree_check,fsid=0)

To prove that the shared volumes work, I initially deployed only 1 mySQL and 1 WordPress replica on the Docker manager and let them initialize the shared volumes.

wordpress-shared-volume-deployment-master
wordpress-shared-volume-deployment-master

Then I commented out:

#       placement:

#         constraints: [node.role == manager]

for worpress and

the 2 bind-mount volumes:

#       - /nfs/common/services/wordpress/db_root:/root

#       - /nfs/common/services/wordpress/www_src/html:/usr/src/wordpress

Next I want to deploy 3 replicas of wordpress on multiple nodes.

Since we are using the ‘local’ driver, we have to create the volumes on each node. In the screenshot below, I am using “parallel ssh” to create them on all nodes using just 2 commands. They are followed by the volume and the stack deployment:

wordpress-shared-volume-deployment
wordpress-shared-volume-deployment

I checked that all replicas are using the shared volumes by ‘docker exec -it’ into the wordpress containers on the nodes they were running and examining the /var/www/html directory. Everything works.

Under the covers, both approaches use NFS for sharing among the nodes. However, shared volumes provide a higher host-independent abstraction than bind-mount volumes. Potentially, you can recreate the shared volumes using other storage backends other than NFS such as AWS EC2 or Glusterfs. Bibd-mount is tied to your host file system.

Conclusion

I learned something new exploring the use of ‘docker stack deploy’. I hope you’ll find this article useful and informative. During my experimentation with Docker on ODROID-C2 in the past months, I’ve come to the conclusion that ARM64 is still not mainstream in the server market. Consequently, all open source projects such as docker, docker volume drivers, kubernetes, openshift only provide pre-built binaries for the INTEL 64 bit platform. For arm64, they are either non-existent or several versions back meaning that you cannot use them to explore new features. If we want this to change this, we need to have a strong arm64 community. To have a strong community, we need strength in number which, I am afraid, we don’t have at the moment! I’d like to hear from you how we can change this – TOGETHER!