On Tue, Jun 7, 2016 at 10:33 AM, Chamila De Alwis <[email protected]> wrote:
> Hi, > > I was looking into the possibility of using Rsync as a platform agnostic > approach to manage deployment synchronization of artifacts that are > dynamically generated in a containerized environment. The scenario I > considered was the artifact generation in the Gateway when an API is > published in the API Manager distributed setup. > > We can consider either a push or pull based method in Rsync to perform > deployment synchronization between the GW manager and the worker nodes. The > manager and worker will be two Kubernetes pods, liable to be spawned on > different Kubernetes nodes. > > In the push method, it's the GW manager that initiates the process. > > 1. Add a folder watcher (inotifywatch[1]) to > repository/deployment/server/synapse-configs > 2. When triggered > 1. Contact Kubernetes API and get list of WORKER_SVC container IPs > 2. for each container IP, Rsync with --delete > > > > The pull method works the other way, i.e. initiated by the GW worker nodes > and has to be run continuously on a loop. > > > > The problem with these two approaches is that it depends on a single GW > manager node. For the push method, multiple GW managers should not > repeatedly do Rsync calls to the workers. In the pull method, if multiple > GW manager nodes are present (and fronted with a load balancer with > Active/Passive failover), they should have the artifacts synchronized. This > could be possible using a hostPath volume between the GW Manager pod > replicas however, it is yet to be tested. > > Additionally, Kubernetes supports a volume plugin named Git Volume [2], > which is basically an emptyDir volume with an initial "git clone" command > done on the provided remote repository. > > The issue with this is that there is no push functionality when the > contents are updated. This might be solved by extending the Git Repo volume > plugin and writing a Carbon Volume Plugin for Kubernetes, however IMO it > would come up with the same set of problems we have in the current SVN > based deployment synchronization, only with an additional code base. > > NFS volume based approach was also considered, however because of the > limitations in moving the mount between the nodes (solutions like Flocker > works on Block Level storage [3]), and managing read-write capability of > multiple containers it also seems to be a complex path. > > IMO out of these approaches, Rsync is the possible candidate (specifically > the push method), although it takes a few workarounds to achieve > functionality. GW Managers would need to synchronize artifacts between > themselves, as well as towards the worker nodes, and the push job should > only run from the active manager node. > > I highly appreciate any input on this. > > [1] - http://linux.die.net/man/1/inotifywatch > [2] - http://kubernetes.io/docs/user-guide/volumes/#gitrepo > [3] - > https://docs.clusterhq.com/en/latest/faq/#can-i-attach-a-single-volume-to-multiple-hosts > > Regards, > Chamila de Alwis > Committer and PMC Member - Apache Stratos > Software Engineer | WSO2 | +94772207163 > Blog: code.chamiladealwis.com > > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Gayan Gunarathne Technical Lead, WSO2 Inc. (http://wso2.com) Committer & PMC Member, Apache Stratos email : [email protected] | mobile : +94 775030545 <%2B94%20766819985>
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
