Hi, What if we take the revolutionary approach of not building the initial artifact! We can keep an already built artefact, modify it and commit it to Stratos Git. We can keep the built artefact in app type zip or even in maven archtype. If we keep it in maven archtype we can modify it during archetype generation.
thanks, dimuthu On Mon, May 4, 2015 at 8:18 PM, Dimuthu Leelarathne <[email protected]> wrote: > Hi Samith, > > It is not job priority - it is the build priority. The build triggered at > app creation must be prioritised. > > We need to research whether it is possible to trigger a build with > priority. One possibility - there could be a separate set of executors to > attend build jobs at app creation only. > > thanks, > dimuthu > > > On Mon, May 4, 2015 at 7:59 PM, Samith Dassanayake <[email protected]> > wrote: > >> Hi all, >> Since number of executors are dependent on resources[1] we have to come >> up with a new strategy. If we are going with a priority based method, It >> seems like we could use a LB plugin("Scoring Load Balancer plugin"[2][3] or >> "Least Load Plugin"[4]) to select the best slave node and the Priority >> Sorter Plugin[5] to prioritize the jobs. >> >> [1] >> http://stackoverflow.com/questions/9626899/jenkins-maximum-number-of-concurrent-jobs >> [2] >> https://wiki.jenkins-ci.org/display/JENKINS/Scoring+Load+Balancer+plugin >> [3] >> http://stackoverflow.com/questions/25900538/can-jenkins-nodes-be-given-priority-for-build-jobs >> [4] https://wiki.jenkins-ci.org/display/JENKINS/Least+Load+Plugin >> [5] https://wiki.jenkins-ci.org/display/JENKINS/Priority+Sorter+Plugin >> >> Thanks, >> Samith >> >> On Mon, May 4, 2015 at 5:02 PM, Dimuthu Leelarathne <[email protected]> >> wrote: >> >>> Hi all, >>> >>> This model will have a problem at app creation. Imagine the number of >>> executors we give are 10. At a time where all 10 executors are engaged what >>> will happen a person clicks on "CreateApplication". >>> >>> Then AF has to wait until at least one job completes and the person who >>> creates the application waits. This will be a major UX problem and not >>> acceptable. We need to work around it. >>> >>> Something that comes into my head is to have a priority method for jobs >>> that are relevant to app creation. Ok. The easiest would be to over >>> allocate the number of executors :D, but that is too dangerous and >>> impractical. >>> >>> thanks, >>> dimuthu >>> >>> On Wed, Apr 22, 2015 at 3:40 PM, Amila Maha Arachchi <[email protected]> >>> wrote: >>> >>>> +1 >>>> >>>> This looks clean and easy to scale. >>>> >>>> On Wed, Apr 22, 2015 at 1:26 PM, Samith Dassanayake <[email protected]> >>>> wrote: >>>> >>>>> Hi all, >>>>> I have started to working on $subject [1] problem definition as >>>>> proposed solution as below. >>>>> >>>>> Problem >>>>> Jenkins remains one of App Cloud weakest links: it is not highly >>>>> available, does not scale well, and increasingly fails with OOM. The >>>>> current deployment of Jenkins is not scalable. We give one Jenkins per >>>>> tenant. Therefore we have come with a more scalable model for jenkins in >>>>> App Factory. >>>>> >>>>> Proposed solution >>>>> Instead of having one jenkins per tenant, we will have jenkins as >>>>> an underlying service. According to the [2] and [3] distributing the >>>>> jobs among multiple jenkins(clusters) is the more scalable way for >>>>> jenkins. When >>>>> we create a job, the job will be created in the format of >>>>> "tenantDomain_appkey_version" and each tenant will have a shared maven >>>>> repo. >>>>> But with growth of number tenants, number of jobs will be increased. >>>>> Therefore we have to come up with a job distributing mechanism with >>>>> Jenkins clusters. If we use a simple job distributing method, based >>>>> on the number of jenkins clusters, when number of clustered has to be >>>>> increased, we have to migrate all jobs according to their new cluster >>>>> values. As a solution for that, after discussing with Srinath, we came up >>>>> with a bucketing mechanism to do the job distribution. Solution would be >>>>> like below. >>>>> >>>>> >>>>> [image: Inline image 1] >>>>> >>>>> >>>>> We'll have pre-defined number of buckets(large value eg: 256). >>>>> Each bucket will deployed on one of the jenkins clusters and one >>>>> jenkins cluster will have multiple buckets. >>>>> During the job creation, we decide bucket id for the job, based on the >>>>> tenantDomain and number of buckets.(Therefore jobs with same tenantDomain >>>>> will always be stored in same bucket) >>>>> >>>>> ex: hash(tenantDomain) % number of buckets = bukcetId >>>>> >>>>> Therefore once the cluster size is increased we only have move the >>>>> relevant buckets, instead of redistributing all the jobs. >>>>> >>>>> Please let me know any suggestions/feedback. >>>>> >>>>> [1] https://redmine.wso2.com/issues/3757 >>>>> [1] >>>>> https://wiki.jenkins-ci.org/display/JENKINS/Consideration+for+Large+Scale+Jenkins+Deployment >>>>> [2] >>>>> http://www.slideshare.net/andrewbayer/7-habits-of-highly-effective-jenkins-users >>>>> [4] >>>>> http://soldering-iron.blogspot.com/2014/01/jenkins-performance-hints.html >>>>> >>>>> Thanks, >>>>> Samith >>>>> >>>>> -- >>>>> Best Regards >>>>> >>>>> Samith Dassanayake >>>>> Software Engineer | Cloud TG >>>>> WSO2, Inc. | http://wso2.com >>>>> lean. enterprise. middleware >>>>> >>>>> Mobile : +947 76207351 >>>>> >>>> >>>> >>>> >>>> -- >>>> *Amila Maharachchi* >>>> Senior Technical Lead >>>> WSO2, Inc.; http://wso2.com >>>> >>>> Blog: http://maharachchi.blogspot.com >>>> Mobile: +94719371446 >>>> >>>> >>> >>> >>> -- >>> Dimuthu Leelarathne >>> Architect & Product Lead of App Factory >>> >>> WSO2, Inc. (http://wso2.com) >>> email: [email protected] >>> Mobile : 0773661935 >>> >>> Lean . Enterprise . Middleware >>> >> >> >> >> -- >> Best Regards >> >> Samith Dassanayake >> Software Engineer | Cloud TG >> WSO2, Inc. | http://wso2.com >> lean. enterprise. middleware >> >> Mobile : +947 76207351 >> > > > > -- > Dimuthu Leelarathne > Architect & Product Lead of App Factory > > WSO2, Inc. (http://wso2.com) > email: [email protected] > Mobile : 0773661935 > > Lean . Enterprise . Middleware > -- Dimuthu Leelarathne Architect & Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: [email protected] Mobile : 0773661935 Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
