Hi,
I was looking into the possibility of using Rsync as a platform agnostic
approach to manage deployment synchronization of artifacts that are
dynamically generated in a containerized environment. The scenario I
considered was the artifact generation in the Gateway when an API is
published in the API Manager distributed setup.
We can consider either a push or pull based method in Rsync to perform
deployment synchronization between the GW manager and the worker nodes. The
manager and worker will be two Kubernetes pods, liable to be spawned on
different Kubernetes nodes.
In the push method, it's the GW manager that initiates the process.
1. Add a folder watcher (inotifywatch[1]) to
repository/deployment/server/synapse-configs
2. When triggered
1. Contact Kubernetes API and get list of WORKER_SVC container IPs
2. for each container IP, Rsync with --delete
The pull method works the other way, i.e. initiated by the GW worker nodes
and has to be run continuously on a loop.
The problem with these two approaches is that it depends on a single GW
manager node. For the push method, multiple GW managers should not
repeatedly do Rsync calls to the workers. In the pull method, if multiple
GW manager nodes are present (and fronted with a load balancer with
Active/Passive failover), they should have the artifacts synchronized. This
could be possible using a hostPath volume between the GW Manager pod
replicas however, it is yet to be tested.
Additionally, Kubernetes supports a volume plugin named Git Volume [2],
which is basically an emptyDir volume with an initial "git clone" command
done on the provided remote repository.
The issue with this is that there is no push functionality when the
contents are updated. This might be solved by extending the Git Repo volume
plugin and writing a Carbon Volume Plugin for Kubernetes, however IMO it
would come up with the same set of problems we have in the current SVN
based deployment synchronization, only with an additional code base.
NFS volume based approach was also considered, however because of the
limitations in moving the mount between the nodes (solutions like Flocker
works on Block Level storage [3]), and managing read-write capability of
multiple containers it also seems to be a complex path.
IMO out of these approaches, Rsync is the possible candidate (specifically
the push method), although it takes a few workarounds to achieve
functionality. GW Managers would need to synchronize artifacts between
themselves, as well as towards the worker nodes, and the push job should
only run from the active manager node.
I highly appreciate any input on this.
[1] - http://linux.die.net/man/1/inotifywatch
[2] - http://kubernetes.io/docs/user-guide/volumes/#gitrepo
[3] -
https://docs.clusterhq.com/en/latest/faq/#can-i-attach-a-single-volume-to-multiple-hosts
Regards,
Chamila de Alwis
Committer and PMC Member - Apache Stratos
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
