Hi all, I have started to working on $subject [1] problem definition as proposed solution as below.
Problem Jenkins remains one of App Cloud weakest links: it is not highly available, does not scale well, and increasingly fails with OOM. The current deployment of Jenkins is not scalable. We give one Jenkins per tenant. Therefore we have come with a more scalable model for jenkins in App Factory. Proposed solution Instead of having one jenkins per tenant, we will have jenkins as an underlying service. According to the [2] and [3] distributing the jobs among multiple jenkins(clusters) is the more scalable way for jenkins. When we create a job, the job will be created in the format of "tenantDomain_appkey_version" and each tenant will have a shared maven repo. But with growth of number tenants, number of jobs will be increased. Therefore we have to come up with a job distributing mechanism with Jenkins clusters. If we use a simple job distributing method, based on the number of jenkins clusters, when number of clustered has to be increased, we have to migrate all jobs according to their new cluster values. As a solution for that, after discussing with Srinath, we came up with a bucketing mechanism to do the job distribution. Solution would be like below. [image: Inline image 1] We'll have pre-defined number of buckets(large value eg: 256). Each bucket will deployed on one of the jenkins clusters and one jenkins cluster will have multiple buckets. During the job creation, we decide bucket id for the job, based on the tenantDomain and number of buckets.(Therefore jobs with same tenantDomain will always be stored in same bucket) ex: hash(tenantDomain) % number of buckets = bukcetId Therefore once the cluster size is increased we only have move the relevant buckets, instead of redistributing all the jobs. Please let me know any suggestions/feedback. [1] https://redmine.wso2.com/issues/3757 [1] https://wiki.jenkins-ci.org/display/JENKINS/Consideration+for+Large+Scale+Jenkins+Deployment [2] http://www.slideshare.net/andrewbayer/7-habits-of-highly-effective-jenkins-users [4] http://soldering-iron.blogspot.com/2014/01/jenkins-performance-hints.html Thanks, Samith -- Best Regards Samith Dassanayake Software Engineer | Cloud TG WSO2, Inc. | http://wso2.com lean. enterprise. middleware Mobile : +947 76207351
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
