Reconcile Problems with Containers

We will use the orchestration tool named “Docker Swarm” that comes built-in to the Docker Engine; Docker Swarm is composed of manager and worker nodes. It attempts to maintain and reconcile to make sure the actual state equals the desired state.
Docker Swarm perform reconciliation when something goes wrong. For example, when a node in the swarm goes down, it might take down running containers with it. The swarm will recognize this loss of containers and will attempt to reschedule containers on available nodes to achieve the desired state for that service.
To demonstrate this we are going to remove a node and see tasks of our nginx1 service be rescheduled on other nodes automatically.
 
1.    Defining a Service
(For better understanding, we create a new service by copying the following line)
     $ docker service create --detach=true --name nginx2 --replicas=5 --publish 81:80  --mount source=/etc/hostname,target=/usr/share/nginx/html/index.html,type=bind,ro nginx:1.12      (entire command to be written as single statement)
 
In order to avoid conflicts with our existing service, we now changed the name and the publish port. We also, added the --replicas option to scale the service with five instances.
 
2.    Watch the update from the output  (On node1, type the following command)
               $ watch -n 1 docker service ps nginx2
 
3.    Delete one node to check the reconciliation ( shifting of service to live nodes)
On  node3 and enter the command to leave the swarm cluster:
               $ docker swarm leave
 
4.    Observe reconciliation in action
On node, now you can watch the reconciliation in action. You should see that containers that were running on node3 will switch to node1 and node2 automatically.
[Ctrl+ c is used to get out of watch screen (in node 1)]