Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
HI sahmika,
I discussed about this with sameera jayasoma furthermore . As he said you
better to maintain two runtime (shared and exclusive). Better if you can
have a discussion about this with him.
*Harsha Thirimanna*
Senior Software Engineer; WSO2, Inc.; http://wso2.com
* http://www.apache.org/**
email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770**
twitter: **http://twitter.com/ http://twitter.com/afkham_azeez**
harshathirimann
linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//
www.linkedin.com/pub/harsha-thirimanna/10/ab8/122*
*
*
*Lean . Enterprise . Middleware*
*
*
On Tue, Jul 30, 2013 at 12:50 AM, Shamika Ariyawansa sham...@wso2.comwrote:
HI Harsha,
Ya this worked and also have to put EnvironmentsJENKINS,
Carbon/Environments
to make sure carbon libraries are loaded to the class path.
Thanks
On Mon, Jul 29, 2013 at 9:20 PM, Harsha Thirimanna hars...@wso2.comwrote:
Hi Shamika,
As I told you, you can create a runtime for the JENKINS
in ${carbon.home}/lib/runtimes/jenkins and put that libraries in to that.
Put a new entry to
${carbon.home}/repository/conf/tomcat/webapp-classloading-environments.xml
ExclusiveEnvironment
NameJENKINS/Name
Classpath${carbon.home}/lib/runtimes/cxf/*.jar;${carbon.home}/lib/runtimes/jenkins//Classpath
/ExclusiveEnvironment
then you have to put a new webapp-classloading.xml in to jenkins.war
with this change to that
EnvironmentsJENKINS/Environments
This runtime will be loaded as per webapp.
If you want to have a shared runtime to all the jenkins wars in a
appserver , then you can have another runtime and put it
as SharedEnvironment and use it in Environments.
--
Shamika Ariyawansa
Senior Software Engineer
Mob:+ 94 772929486
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
HI Janaka, We have decided to leave out 2nd and 3rd options go with the 1st option. Comments are inline. On Mon, Jul 29, 2013 at 10:37 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Shamika, On Wed, Jul 24, 2013 at 5:20 PM, Ajanthan Balachandran ajant...@wso2.comwrote: On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. How will this handle the load of a build job? Say that each of these Jenkins server instances can run a number of build jobs(more than 1 build per tenant). How can we handle the concurrent build load? As Ajanthan suggested we are going to minimize the Jenkins app by removing common components and plugins and making them available as common libraries. We tried that on a Tomcat instance and worked fine. Now working on that on AS. This Jenkins app per tenant work as the Master. We could have multiple slaves per tenant and handle the load. Later we can have a slave pool where all the masters can share the slaves on demand depending on the work. 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. Jenkins has lots of extension points. Can we have a concept of folders/collections in Jenkins jobs? If so we can group jobs by tenant. This could be done as a part of the Role-Strategy plugin. As a found so far there is no such categorization available and not possible since they have one JENKINS_HOME per instance. Since it is storing all the jobs in once location, it is not scalable either even though we could have come up with a role-based plugin to filter them out by the tenant. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. We should be able to work with an existing Jenkins deployment without doing any modifications to the code. If this can be done through a plugin, then I guess it is fine. Otherwise I'm -1 for this. Yes ass you pointed out, this would be not a good option if we cant do it as a plugin. Also the way they have written the code this modification is not realistic. Thanks, Janaka WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 -- ajanthan -- Ajanthan Balachandiran Senior Software Engineer; Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/ email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497 blog: http://bkayts.blogspot.com/ Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com* E-mail: jan...@wso2.com **M: **+94 718370861* * *Lean . Enterprise . Middleware -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
Hi shamika , When you minimize the replication of runtime of jenkins , I think you can create SharedEnvironment and Exclusive environment to load the jars for JENKINS. *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/** email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770** twitter: **http://twitter.com/ http://twitter.com/afkham_azeez** harshathirimann linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**// www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* * * *Lean . Enterprise . Middleware* * * On Mon, Jul 29, 2013 at 2:22 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Shamika, On Mon, Jul 29, 2013 at 11:31 AM, Shamika Ariyawansa sham...@wso2.comwrote: HI Janaka, We have decided to leave out 2nd and 3rd options go with the 1st option. Comments are inline. On Mon, Jul 29, 2013 at 10:37 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Shamika, On Wed, Jul 24, 2013 at 5:20 PM, Ajanthan Balachandran ajant...@wso2.com wrote: On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. How will this handle the load of a build job? Say that each of these Jenkins server instances can run a number of build jobs(more than 1 build per tenant). How can we handle the concurrent build load? As Ajanthan suggested we are going to minimize the Jenkins app by removing common components and plugins and making them available as common libraries. We tried that on a Tomcat instance and worked fine. Now working on that on AS. This Jenkins app per tenant work as the Master. We could have multiple slaves per tenant and handle the load. Later we can have a slave pool where all the masters can share the slaves on demand depending on the work. I guess you can use [1] for spawning instances on demand(haven't tried this yet). Thanks, Janaka [1] https://wiki.jenkins-ci.org/display/JENKINS/Scripted+Cloud+plugin 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. Jenkins has lots of extension points. Can we have a concept of folders/collections in Jenkins jobs? If so we can group jobs by tenant. This could be done as a part of the Role-Strategy plugin. As a found so far there is no such categorization available and not possible since they have one JENKINS_HOME per instance. Since it is storing all the jobs in once location, it is not scalable either even though we could have come up with a role-based plugin to filter them out by the tenant. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. We should be able to work with an existing Jenkins deployment without doing any modifications to the code. If this can be done through a plugin, then I guess it is fine. Otherwise I'm -1 for this. Yes ass you pointed out, this would be not a good option if we cant do it as a plugin. Also the way they have written the code this modification is not realistic. Thanks, Janaka WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
Hi Shamika,
As I told you, you can create a runtime for the JENKINS
in ${carbon.home}/lib/runtimes/jenkins and put that libraries in to that.
Put a new entry to
${carbon.home}/repository/conf/tomcat/webapp-classloading-environments.xml
ExclusiveEnvironment
NameJENKINS/Name
Classpath${carbon.home}/lib/runtimes/cxf/*.jar;${carbon.home}/lib/runtimes/jenkins//Classpath
/ExclusiveEnvironment
then you have to put a new webapp-classloading.xml in to jenkins.war
with this change to that
EnvironmentsJENKINS/Environments
This runtime will be loaded as per webapp.
If you want to have a shared runtime to all the jenkins wars in a
appserver , then you can have another runtime and put it
as SharedEnvironment and use it in Environments.
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
Hi Shamika, On Wed, Jul 24, 2013 at 5:20 PM, Ajanthan Balachandran ajant...@wso2.comwrote: On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. How will this handle the load of a build job? Say that each of these Jenkins server instances can run a number of build jobs(more than 1 build per tenant). How can we handle the concurrent build load? 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. Jenkins has lots of extension points. Can we have a concept of folders/collections in Jenkins jobs? If so we can group jobs by tenant. This could be done as a part of the Role-Strategy plugin. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. We should be able to work with an existing Jenkins deployment without doing any modifications to the code. If this can be done through a plugin, then I guess it is fine. Otherwise I'm -1 for this. Thanks, Janaka WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 -- ajanthan -- Ajanthan Balachandiran Senior Software Engineer; Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/ email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497 blog: http://bkayts.blogspot.com/ Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com* E-mail: jan...@wso2.com **M: **+94 718370861** *Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
*Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/** email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770** twitter: **http://twitter.com/ http://twitter.com/afkham_azeez** harshathirimann linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**// www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* * * *Lean . Enterprise . Middleware* * * On Mon, Jul 29, 2013 at 1:07 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Shamika, On Wed, Jul 24, 2013 at 5:20 PM, Ajanthan Balachandran ajant...@wso2.comwrote: On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. How will this handle the load of a build job? Say that each of these Jenkins server instances can run a number of build jobs(more than 1 build per tenant). How can we handle the concurrent build load? As ajanthan said , if we can deploy jenkins instance for every tenant then we can maintain multiple slave node to every master node to handle the load. This point was mentioned that with previous email thread by ajanthan. Subject : [Architecture] [Appfactory] Tenant Isolation for Jenkins. 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. Jenkins has lots of extension points. Can we have a concept of folders/collections in Jenkins jobs? If so we can group jobs by tenant. This could be done as a part of the Role-Strategy plugin. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. We should be able to work with an existing Jenkins deployment without doing any modifications to the code. If this can be done through a plugin, then I guess it is fine. Otherwise I'm -1 for this. Thanks, Janaka WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 -- ajanthan -- Ajanthan Balachandiran Senior Software Engineer; Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/ email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497 blog: http://bkayts.blogspot.com/ Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com* E-mail: jan...@wso2.com **M: **+94 718370861* * *Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
Does that mean a build cloud or a build server per tenant? On Mon, Jul 29, 2013 at 10:53 AM, Harsha Thirimanna hars...@wso2.comwrote: *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/** email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770** twitter: **http://twitter.com/ http://twitter.com/afkham_azeez** harshathirimann linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**// www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* * * *Lean . Enterprise . Middleware* * * On Mon, Jul 29, 2013 at 1:07 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Shamika, On Wed, Jul 24, 2013 at 5:20 PM, Ajanthan Balachandran ajant...@wso2.com wrote: On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. How will this handle the load of a build job? Say that each of these Jenkins server instances can run a number of build jobs(more than 1 build per tenant). How can we handle the concurrent build load? As ajanthan said , if we can deploy jenkins instance for every tenant then we can maintain multiple slave node to every master node to handle the load. This point was mentioned that with previous email thread by ajanthan. Subject : [Architecture] [Appfactory] Tenant Isolation for Jenkins. 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. Jenkins has lots of extension points. Can we have a concept of folders/collections in Jenkins jobs? If so we can group jobs by tenant. This could be done as a part of the Role-Strategy plugin. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. We should be able to work with an existing Jenkins deployment without doing any modifications to the code. If this can be done through a plugin, then I guess it is fine. Otherwise I'm -1 for this. Thanks, Janaka WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 -- ajanthan -- Ajanthan Balachandiran Senior Software Engineer; Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/ email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497 blog: http://bkayts.blogspot.com/ Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com* E-mail: jan...@wso2.com **M: **+94 718370861* * *Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks, Samisa... Samisa Abeysinghe VP Engineering WSO2 Inc. http://wso2.com http://wso2.org ___ Architecture mailing list Architecture@wso2.org
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
Hi, On Mon, Jul 29, 2013 at 10:53 AM, Harsha Thirimanna hars...@wso2.comwrote: *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/** email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770** twitter: **http://twitter.com/ http://twitter.com/afkham_azeez** harshathirimann linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**// www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* * * *Lean . Enterprise . Middleware* * * On Mon, Jul 29, 2013 at 1:07 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Shamika, On Wed, Jul 24, 2013 at 5:20 PM, Ajanthan Balachandran ajant...@wso2.com wrote: On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. How will this handle the load of a build job? Say that each of these Jenkins server instances can run a number of build jobs(more than 1 build per tenant). How can we handle the concurrent build load? As ajanthan said , if we can deploy jenkins instance for every tenant then we can maintain multiple slave node to every master node to handle the load. This point was mentioned that with previous email thread by ajanthan. Does this means that we have multiple slave nodes that is been used by every master(of each tenant)? Or do we spawn a number of slaves for every tenant? Ideally what we need to do is to spawn a Jenkins slave on demand. Thanks, Janaka Subject : [Architecture] [Appfactory] Tenant Isolation for Jenkins. 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. Jenkins has lots of extension points. Can we have a concept of folders/collections in Jenkins jobs? If so we can group jobs by tenant. This could be done as a part of the Role-Strategy plugin. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. We should be able to work with an existing Jenkins deployment without doing any modifications to the code. If this can be done through a plugin, then I guess it is fine. Otherwise I'm -1 for this. Thanks, Janaka WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 -- ajanthan -- Ajanthan Balachandiran Senior Software Engineer; Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/ email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497 blog: http://bkayts.blogspot.com/ Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com* E-mail: jan...@wso2.com **M: **+94 718370861* * *Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 -- ajanthan -- Ajanthan Balachandiran Senior Software Engineer; Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/ email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497 blog: http://bkayts.blogspot.com/ Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation for Jenkins.
On Tue, Jul 16, 2013 at 6:34 PM, Shiroshica Kulatilake sh...@wso2.comwrote: Hi, Apparently having to find a way to make non multi-tenanted products to work in a multi-tenanted environment is a situation that has been come across before. IBM has had to do this and there are multi-tenanting patternshttp://www.ibm.com/developerworks/cloud/library/cl-tenantconversion/that can be followed. Some of these methods are having a shared user store and operational database thus achieving the shared database model; having tenant specific reports and logs etc.. What we are doing in appfactory for each of the ecosystems may already fall into one of these patterns. For those cases where we are still looking for a solution, viewing the problem from this aspect might shed some light. The different models of multi-tenancy and through that seeing which level can be shared within a given non MT'd tool is also a point to consider. Through this we may have a label to how we multi-tenanted products X, Y and Z. GitBlit - Shared Application Jenkins - Shred Middleware(according to the article) - But I would call it multi-tenancy coming from the PaaS thanks, dimuthu WDYT ? Thank you, Shiro nkins slaves [4][5]. We need to figure out a scheme on how to do this. -- Shiroshica Kulatilake Architect, WSO2, Inc. http://wso2.com/ Phone: +94 776523867 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation for Jenkins.
On Wed, Jul 17, 2013 at 4:33 AM, Dimuthu Leelarathne dimut...@wso2.comwrote: Hi, On Tue, Jul 16, 2013 at 5:33 AM, Samisa Abeysinghe sam...@wso2.comwrote: How about mapping to tiers the tenant has subscribed to? We can have separate build farms for different tiers. We need to discuss aPaaS pricing. How many builds per tier and so on. I think we missed that in our pricing meeting. It is not just a farms or a biliing problem. Tenant isolation needs to take tiers into account. thanks, dimuthu -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks, Samisa... Samisa Abeysinghe VP Engineering WSO2 Inc. http://wso2.com http://wso2.org ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks, Samisa... Samisa Abeysinghe VP Engineering WSO2 Inc. http://wso2.com http://wso2.org ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation for Jenkins.
Hi Samisa,
On Tue, Jul 16, 2013 at 5:33 AM, Samisa Abeysinghe sam...@wso2.com wrote:
How about mapping to tiers the tenant has subscribed to?
We can have separate build farms for different tiers. We need to discuss
aPaaS pricing. How many builds per tier and so on.
thanks,
dimuthu
On Fri, Jul 12, 2013 at 7:06 PM, Ramith Jayasinghe ram...@wso2.comwrote:
Hi,
I have been looking at possible ways to achieve the $subject.
And here's one possible way of achieving this.
Basis of this approach is to deploy jenkins as a web application on our
Application Server ( -- I tested this) [1].
There couple aspects we need to think about :
1. Tenant Creation
2. Authentication and authorization
3. Building Jobs.
4. Load balancing and HA
*
Tenant Creation*
Now suppose user creates a organization/tenant xyz on Appfactory.
Appfactory could either :
1. Deploy jenkins onto the xyz tenant
Or
2. Deploy jenkins on super tenant. With this approach we have to rename
jenkins.war (possibly to xyz.war) to make this jenkins instance different
from others.
With either way each jenkins instance needs to be provided with a its
own JENKINS_HOME.
This can be achieved via adding a context.xml file with content similar
to following to jenkins.war distribution ( specifically into META-INF)
Context
Environment name=JENKINS_HOME value=${J_HOMES}/xyz
type=java.lang.String/
/Context
$J_HOMES refers to a directory in file system and supplied as a system
variable during the server start up etc.
'xyz' sub-directory has to be created and filled with configurations,
plugins etc before deploying the webapp.
Configuration that needs to go into this folder are typically related to
maven,jdks, plugins etc. ( can puppet be of use to automate all this?)
*
**Note:* I already came across an issue where content in conext.xml is
not visible to jenkins.war when deployed onto a tenant ( maybe a bug in AS
?)
*
Authentication Authorization*
We could use Jenkins LDAP (may be with some modifications) plugin based
on the requirement or we might have to change current authentication plugin
[2] we wrote for jenkins.
Further, I think with above approach we still can use the role strategy
plugin [3] that's currently in use.
*
Building Jobs*
Master nodes ( - deployed on AS) should not run build jobs.
Instead these should be delegated to a pool of jenkins slaves [4][5]. We
need to figure out a scheme on how to do this.
*Load balancing and HA *
Deploying and testing jenkins on a AS cluster should be the starting
point to figure out weather this whole approach would scale.
[1] https://wiki.jenkins-ci.org/display/JENKINS/Tomcat
[2]
https://svn.wso2.org/repos/wso2/carbon/platform/branches/4.1.0/products/appfactory/1.0.0/modules/webapps/appfactory-authentication-plugin
[3]
https://svn.wso2.org/repos/wso2/carbon/platform/branches/4.1.0/dependencies/jenkins-ci/role-strategy-plugin/1.1.3-wso2v2
[4] https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds
[5]
https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-RunningMultipleSlavesontheSameMachine
--
Ramith Jayasinghe
Technical Lead
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
E: ram...@wso2.com
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Thanks,
Samisa...
Samisa Abeysinghe
VP Engineering
WSO2 Inc.
http://wso2.com
http://wso2.org
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Dimuthu Leelarathne
Architect Product Lead of App Factory
WSO2, Inc. (http://wso2.com)
email: dimut...@wso2.com
Mobile : 0773661935
Lean . Enterprise . Middleware
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation for Jenkins.
On Wed, Jul 17, 2013 at 6:18 AM, Samisa Abeysinghe sam...@wso2.com wrote: On Wed, Jul 17, 2013 at 4:33 AM, Dimuthu Leelarathne dimut...@wso2.comwrote: Hi, On Tue, Jul 16, 2013 at 5:33 AM, Samisa Abeysinghe sam...@wso2.comwrote: How about mapping to tiers the tenant has subscribed to? We can have separate build farms for different tiers. We need to discuss aPaaS pricing. How many builds per tier and so on. I think we missed that in our pricing meeting. It is not just a farms or a biliing problem. Tenant isolation needs to take tiers into account. Tier X - 3 developers (silver tier) Tier Y - 15 developers (more expensive - gold tier) And there are two build farms - Tier X build farm and Tier Y build farm. Tier Y build farm has more resources. And our BAM things can monitor whether we are meeting SLAs for each build farm and we can start up more slaves on demand as required to meet SLAs. For private Jet Mode we might have to delegate a separate jenkins server. And that should be handled by rules and S2. thanks, dimuthu thanks, dimuthu -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks, Samisa... Samisa Abeysinghe VP Engineering WSO2 Inc. http://wso2.com http://wso2.org ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks, Samisa... Samisa Abeysinghe VP Engineering WSO2 Inc. http://wso2.com http://wso2.org ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation for Jenkins.
On Fri, Jul 12, 2013 at 7:06 PM, Ramith Jayasinghe ram...@wso2.com wrote:
Hi,
I have been looking at possible ways to achieve the $subject.
And here's one possible way of achieving this.
Basis of this approach is to deploy jenkins as a web application on our
Application Server ( -- I tested this) [1].
There couple aspects we need to think about :
1. Tenant Creation
2. Authentication and authorization
3. Building Jobs.
4. Load balancing and HA
*
Tenant Creation*
Now suppose user creates a organization/tenant xyz on Appfactory.
Appfactory could either :
1. Deploy jenkins onto the xyz tenant
Or
This will work as each tenant gets its own classloader.
2. Deploy jenkins on super tenant. With this approach we have to rename
jenkins.war (possibly to xyz.war) to make this jenkins instance different
from others.
This will not work as jenkins uses lot of singletons.So we need seperate
class loader for web app
With either way each jenkins instance needs to be provided with a its own
JENKINS_HOME.
This can be achieved via adding a context.xml file with content similar
to following to jenkins.war distribution ( specifically into META-INF)
Context
Environment name=JENKINS_HOME value=${J_HOMES}/xyz
type=java.lang.String/
/Context
$J_HOMES refers to a directory in file system and supplied as a system
variable during the server start up etc.
'xyz' sub-directory has to be created and filled with configurations,
plugins etc before deploying the webapp.
Configuration that needs to go into this folder are typically related to
maven,jdks, plugins etc. ( can puppet be of use to automate all this?)
It is not good to have modified config for each tenants. Whenever we need
enable failover node we need to get these modified files.But
if we have a one war file(with configs) that can be used regardless of
tenants ,then that is very scalable.I think it is possible because there is
no tenant specific configurations(the config for tenant 1 and
2 ideally same).
*
**Note:* I already came across an issue where content in conext.xml is
not visible to jenkins.war when deployed onto a tenant ( maybe a bug in AS
?)
Did you try to set homepath as jndi variable.It seems that they are using
jndi to supply the value.We have changed the JNDI factory for AS.So we need
to
write a Servlet Context listener and set the Jenkins path as jndi value.
*
Authentication Authorization*
We could use Jenkins LDAP (may be with some modifications) plugin based
on the requirement or we might have to change current authentication plugin
[2] we wrote for jenkins.
Further, I think with above approach we still can use the role strategy
plugin [3] that's currently in use.
Jenkins going to run in carbon environment so you can get user
manager through carbon context.
*
Building Jobs*
Master nodes ( - deployed on AS) should not run build jobs.
Instead these should be delegated to a pool of jenkins slaves [4][5]. We
need to figure out a scheme on how to do this.
*Load balancing and HA *
Deploying and testing jenkins on a AS cluster should be the starting point
to figure out this whole approach would scale.
In jenkins world we can not cluster master.We can only set up failover to
master and offload the build to slaves.There is a one to many connection
between a master and slaves.If
one slave is assigned to a master it can not be assign to other master.As a
solution we can have pool of slaves and assign slave on demand(when ever a
build is trickered ) and return to the pool after finishing job.
Starting point may be jenkins Swarm plugin.We can't use it out of the box
but we have to modify the plugin to support the multi master shared slaves
deployment.
[1] https://wiki.jenkins-ci.org/display/JENKINS/Tomcat
[2]
https://svn.wso2.org/repos/wso2/carbon/platform/branches/4.1.0/products/appfactory/1.0.0/modules/webapps/appfactory-authentication-plugin
[3]
https://svn.wso2.org/repos/wso2/carbon/platform/branches/4.1.0/dependencies/jenkins-ci/role-strategy-plugin/1.1.3-wso2v2
[4] https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds
[5]
https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-RunningMultipleSlavesontheSameMachine
--
Ramith Jayasinghe
Technical Lead
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
E: ram...@wso2.com
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
ajanthan
--
Ajanthan Balachandiran
Senior Software Engineer;
Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/
email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497
blog: http://bkayts.blogspot.com/
Lean . Enterprise . Middleware
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
