Even with all those elastic superscaling whatsoever buzzword stuff, sometimes it just comes down to simple upgrading a component like Docker daemon itself or a new kernel.
With swarm, this means draining the node, do our stuff, probably reboot and then set it active again.
Being ephemeral is cool, so we could spawn new (updated) swarm members using Ansible and then destroy the old hard-working bees afterwards. But to me, this seems more like a “just because we can”, even more then running small or medium sized swarms/clusters.
Using my dynamic Ansible inventory for Docker swarm, we could just run any playbook against an existing swarm. Upgrading now is just simple playbook like this (please consider this as an exmaple only…):
---
- hosts: '{{ target }}'
serial: 1
tasks:
- name: docker info
command: docker info
register: docker_info
- name: drain docker node
environment:
DOCKER_HOST: "tcp://my-swarm.example.com:2375"
local_action: shell docker node update --availability drain '{{ inventory_hostname }}'
- name: apt dist-upgrade
apt:
update_cache: yes
upgrade: dist
autoremove: yes
- name: reboot
become: yes
shell: sleep 2 && /sbin/shutdown -r now "Ansible reboot"
async: 1
poll: 0
ignore_errors: true
- name: waiting for server to come back
local_action: wait_for host={{ ansible_default_ipv4.address }} port=22 state=started delay=30 timeout=240
become: false
- name: wait for swarm manager
wait_for:
host: "{{ inventory_hostname }}"
port: "2377"
delay: 5
state: started
timeout: 1200
when: "docker_info.stdout | search('Is Manager: true')"
- name: activate docker node
environment:
DOCKER_HOST: "tcp://my-swarm.example.com:2375"
local_action: shell docker node update --availability active '{{ inventory_hostname }}'
The serial: 1 takes care of just one host after another, so the swarm can take care of spawning new services on other hosts to fulfil the requirements.
I like using a variable for the target hosts, so calling this one using my dynamic inventory with all workers only would be:
ansible-playbook -i /etc/ansible/docker-swarm-nodes.py -e target=worker update.yml
As always, this is just a hint, of course you should have secured access to your swarm.
Also one could use pre/post tasks in Ansible!