Hi all, This model will have a problem at app creation. Imagine the number of executors we give are 10. At a time where all 10 executors are engaged what will happen a person clicks on "CreateApplication".
Then AF has to wait until at least one job completes and the person who creates the application waits. This will be a major UX problem and not acceptable. We need to work around it. Something that comes into my head is to have a priority method for jobs that are relevant to app creation. Ok. The easiest would be to over allocate the number of executors :D, but that is too dangerous and impractical. thanks, dimuthu On Wed, Apr 22, 2015 at 3:40 PM, Amila Maha Arachchi <[email protected]> wrote: > +1 > > This looks clean and easy to scale. > > On Wed, Apr 22, 2015 at 1:26 PM, Samith Dassanayake <[email protected]> > wrote: > >> Hi all, >> I have started to working on $subject [1] problem definition as proposed >> solution as below. >> >> Problem >> Jenkins remains one of App Cloud weakest links: it is not highly >> available, does not scale well, and increasingly fails with OOM. The >> current deployment of Jenkins is not scalable. We give one Jenkins per >> tenant. Therefore we have come with a more scalable model for jenkins in >> App Factory. >> >> Proposed solution >> Instead of having one jenkins per tenant, we will have jenkins as >> an underlying service. According to the [2] and [3] distributing the >> jobs among multiple jenkins(clusters) is the more scalable way for jenkins. >> When >> we create a job, the job will be created in the format of >> "tenantDomain_appkey_version" and each tenant will have a shared maven repo. >> But with growth of number tenants, number of jobs will be increased. >> Therefore we have to come up with a job distributing mechanism with >> Jenkins clusters. If we use a simple job distributing method, based on >> the number of jenkins clusters, when number of clustered has to be >> increased, we have to migrate all jobs according to their new cluster >> values. As a solution for that, after discussing with Srinath, we came up >> with a bucketing mechanism to do the job distribution. Solution would be >> like below. >> >> >> [image: Inline image 1] >> >> >> We'll have pre-defined number of buckets(large value eg: 256). >> Each bucket will deployed on one of the jenkins clusters and one jenkins >> cluster will have multiple buckets. >> During the job creation, we decide bucket id for the job, based on the >> tenantDomain and number of buckets.(Therefore jobs with same tenantDomain >> will always be stored in same bucket) >> >> ex: hash(tenantDomain) % number of buckets = bukcetId >> >> Therefore once the cluster size is increased we only have move the >> relevant buckets, instead of redistributing all the jobs. >> >> Please let me know any suggestions/feedback. >> >> [1] https://redmine.wso2.com/issues/3757 >> [1] >> https://wiki.jenkins-ci.org/display/JENKINS/Consideration+for+Large+Scale+Jenkins+Deployment >> [2] >> http://www.slideshare.net/andrewbayer/7-habits-of-highly-effective-jenkins-users >> [4] >> http://soldering-iron.blogspot.com/2014/01/jenkins-performance-hints.html >> >> Thanks, >> Samith >> >> -- >> Best Regards >> >> Samith Dassanayake >> Software Engineer | Cloud TG >> WSO2, Inc. | http://wso2.com >> lean. enterprise. middleware >> >> Mobile : +947 76207351 >> > > > > -- > *Amila Maharachchi* > Senior Technical Lead > WSO2, Inc.; http://wso2.com > > Blog: http://maharachchi.blogspot.com > Mobile: +94719371446 > > -- Dimuthu Leelarathne Architect & Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: [email protected] Mobile : 0773661935 Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
