+1 This looks clean and easy to scale.
On Wed, Apr 22, 2015 at 1:26 PM, Samith Dassanayake <[email protected]> wrote: > Hi all, > I have started to working on $subject [1] problem definition as proposed > solution as below. > > Problem > Jenkins remains one of App Cloud weakest links: it is not highly > available, does not scale well, and increasingly fails with OOM. The > current deployment of Jenkins is not scalable. We give one Jenkins per > tenant. Therefore we have come with a more scalable model for jenkins in > App Factory. > > Proposed solution > Instead of having one jenkins per tenant, we will have jenkins as > an underlying service. According to the [2] and [3] distributing the jobs > among multiple jenkins(clusters) is the more scalable way for jenkins. When > we create a job, the job will be created in the format of > "tenantDomain_appkey_version" and each tenant will have a shared maven repo. > But with growth of number tenants, number of jobs will be increased. > Therefore we have to come up with a job distributing mechanism with > Jenkins clusters. If we use a simple job distributing method, based on > the number of jenkins clusters, when number of clustered has to be > increased, we have to migrate all jobs according to their new cluster > values. As a solution for that, after discussing with Srinath, we came up > with a bucketing mechanism to do the job distribution. Solution would be > like below. > > > [image: Inline image 1] > > > We'll have pre-defined number of buckets(large value eg: 256). > Each bucket will deployed on one of the jenkins clusters and one jenkins > cluster will have multiple buckets. > During the job creation, we decide bucket id for the job, based on the > tenantDomain and number of buckets.(Therefore jobs with same tenantDomain > will always be stored in same bucket) > > ex: hash(tenantDomain) % number of buckets = bukcetId > > Therefore once the cluster size is increased we only have move the > relevant buckets, instead of redistributing all the jobs. > > Please let me know any suggestions/feedback. > > [1] https://redmine.wso2.com/issues/3757 > [1] > https://wiki.jenkins-ci.org/display/JENKINS/Consideration+for+Large+Scale+Jenkins+Deployment > [2] > http://www.slideshare.net/andrewbayer/7-habits-of-highly-effective-jenkins-users > [4] > http://soldering-iron.blogspot.com/2014/01/jenkins-performance-hints.html > > Thanks, > Samith > > -- > Best Regards > > Samith Dassanayake > Software Engineer | Cloud TG > WSO2, Inc. | http://wso2.com > lean. enterprise. middleware > > Mobile : +947 76207351 > -- *Amila Maharachchi* Senior Technical Lead WSO2, Inc.; http://wso2.com Blog: http://maharachchi.blogspot.com Mobile: +94719371446
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
