Hi, Had a chat with Azeez regarding this. This is how the behavior of the scenario should be.
When a tenant creates an artifact on the Master node, a cluster message is sent asking the slaves to update themselves. However, if the tenant is not loaded on the slave node, it should ignore that message since it does not make sense to load the repo of a tenant which still hasn't been loaded. When a request comes to the slave, it should at that point do a checkout of the artifacts. The problem now is, the slave node does not ignore the cluster message sent from the Master. Therefore, it attempts to engage the deployment synchronizer on a non existent directory. So AFAIU, the fix for this issue should be to ensure the slave node ignores tenant specific cluster messages if the particular tenant is not loaded. Is this something that we can fix before the current AS release? Thanks, NuwanD. On Thu, Jul 26, 2012 at 11:39 AM, Sagara Gunathunga <[email protected]> wrote: > Hi Nuwan/Kasun, > > This seems a critical issue for current AS release too ? Any possibility > to fix this issue ? > > Thanks ! > > On Mon, Jul 23, 2012 at 5:27 PM, Lahiru Sandaruwan <[email protected]>wrote: > >> >> >> On Mon, Jul 23, 2012 at 3:37 PM, Afkham Azeez <[email protected]> wrote: >> >>> >>> >>> On Mon, Jul 23, 2012 at 3:35 PM, Lahiru Sandaruwan <[email protected]>wrote: >>> >>>> >>>> >>>> On Mon, Jul 23, 2012 at 2:55 PM, Nuwan Dias <[email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> Need some help on proceeding with this issue. This is the problem we >>>>> are facing. >>>>> >>>>> In a clustered environment, when we create a tenant on the master >>>>> node, the same tenant does not get created on the slaves nodes >>>>> automatically. Therefore, the relevant tenant artifact directories (ex: >>>>> repository/tenants/1) do not get created on the slave nodes. When a tenant >>>>> creates an artifact on the Master node, the deployment synchronizer >>>>> attempts to sync it on the Slave nodes. This results in an error since the >>>>> relevant directories have not been created on the slave nodes. >>>>> >>>>> One possible solution to this is to create the directories (if they >>>>> are missing) on the slave nodes at deployment synchronization. But I doubt >>>>> that this is the proper solution since this will create deployment >>>>> directories for a non-existent tenant. >>>>> >>>>> The other solution I see is to find a way to synchronize tenant >>>>> accounts across the cluster. This will ensure that deployment directories >>>>> will be created on all instances at the time of tenant creation. >>>>> >>>> In Stratos, if tenant created in one node of cluster and next time >>>> tenant is directed to another node by load balancer, how that is handled? >>>> >>>> >>> When a tenant is loaded on any node, the full depsync (update) is done >>> before the tenant becomes active. >>> >> So, was this tested before and broke later? Or do we have to have >> complete Stratos setup to get this done? >> >>> >>> >>>> IMO if we can find a solution for that, which a must requirement we can >>>> have a solution for this as well. >>>> >>>>> >>>>> Is there any other better way to solve this? >>>>> >>>>> Thanks, >>>>> NuwanD. >>>>> >>>>> On Fri, Jul 20, 2012 at 5:37 PM, Nuwan Dias <[email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> The deployment synchronizer should be engaged to the relevant >>>>>> directory path. Therefore, it will not work by creating the directory by >>>>>> hand. If the same tenant (3) is created on the slave node, then the >>>>>> directory structure will be created and engaged to the deployment >>>>>> synchronizer as well. >>>>>> >>>>>> Will discuss on this further to decide on the proper solution. >>>>>> >>>>>> Thanks, >>>>>> NuwanD. >>>>>> >>>>>> On Fri, Jul 20, 2012 at 4:15 PM, Lahiru Sandaruwan >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Jul 20, 2012 at 3:30 PM, Muhammed Shariq <[email protected]>wrote: >>>>>>> >>>>>>>> I had a look with Lahiru regarding this issue and it seems the >>>>>>>> tenant directory (eg - repository/tenants/3/ ) is not getting created >>>>>>>> in >>>>>>>> the slave node. Slave node is configured with auto commit false which >>>>>>>> is >>>>>>>> correct. So I guess this is a deployment synchronizer issue ?! >>>>>>>> >>>>>>> Yes, it does not sync apps even after i created the directory >>>>>>> "repository/tenants/3/". >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> On Thu, Jul 19, 2012 at 3:32 PM, Lahiru Sandaruwan < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I have a App server cluster of 2 servers (Beta 2 packs) clustered >>>>>>>>> using 'wka' method. >>>>>>>>> >>>>>>>>> For master node, >>>>>>>>> >>>>>>>>> <AutoCommit>true</AutoCommit> >>>>>>>>> <AutoCheckout>true</AutoCheckout> >>>>>>>>> >>>>>>>>> and slave node, >>>>>>>>> >>>>>>>>> <AutoCommit>false</AutoCommit> >>>>>>>>> <AutoCheckout>true</AutoCheckout> >>>>>>>>> >>>>>>>>> When i create a new tenant in master node, it commits all the >>>>>>>>> apps to svn but slave gives following error, >>>>>>>>> >>>>>>>>> [2012-07-19 15:05:50,181] ERROR >>>>>>>>> {org.wso2.carbon.core.deployment.SynchronizeRepositoryRequest} - >>>>>>>>> Repository update failed for tenant 2 >>>>>>>>> java.lang.RuntimeException: >>>>>>>>> org.wso2.carbon.deployment.synchronizer.DeploymentSynchronizerException: >>>>>>>>> A >>>>>>>>> repository synchronizer has not been engaged for the file path: >>>>>>>>> /home/lahiru/work/phpHosting/php/test/wso2as-5.0.0-SNAPSHOT/repository/tenants/2 >>>>>>>>> at >>>>>>>>> org.wso2.carbon.deployment.synchronizer.internal.DeploymentSynchronizerServiceImpl.update(DeploymentSynchronizerServiceImpl.java:78) >>>>>>>>> at >>>>>>>>> org.wso2.carbon.core.deployment.SynchronizeRepositoryRequest.updateDeploymentRepository(SynchronizeRepositoryRequest.java:78) >>>>>>>>> at >>>>>>>>> org.wso2.carbon.core.deployment.SynchronizeRepositoryRequest.execute(SynchronizeRepositoryRequest.java:55) >>>>>>>>> at >>>>>>>>> org.apache.axis2.clustering.tribes.RpcMessagingHandler.replyRequest(RpcMessagingHandler.java:55) >>>>>>>>> at >>>>>>>>> org.apache.catalina.tribes.group.RpcChannel.messageReceived(RpcChannel.java:134) >>>>>>>>> at >>>>>>>>> org.apache.axis2.clustering.tribes.Axis2GroupChannel.messageReceived(Axis2GroupChannel.java:79) >>>>>>>>> at >>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:84) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> When the setup is changed to master-master error changes, >>>>>>>>> >>>>>>>>> [2012-07-19 15:16:16,049] ERROR >>>>>>>>> {org.wso2.carbon.deployment.synchronizer.subversion.SVNBasedArtifactRepository} >>>>>>>>> - Error while committing artifacts to the SVN repository >>>>>>>>> org.tigris.subversion.svnclientadapter.SVNClientException: >>>>>>>>> org.tigris.subversion.svnclientadapter.commandline.CmdLineException: >>>>>>>>> svn: >>>>>>>>> Commit failed (details follow): >>>>>>>>> svn: Server sent unexpected return value (405 Method Not Allowed) >>>>>>>>> in response to MKCOL request for >>>>>>>>> '/svn/!svn/wrk/978da883-294e-4f4c-b930-663d9881a2af/-1234/webapps/example' >>>>>>>>> >>>>>>>>> at >>>>>>>>> org.tigris.subversion.svnclientadapter.SVNClientException.wrapException(SVNClientException.java:93) >>>>>>>>> at >>>>>>>>> org.tigris.subversion.svnclientadapter.commandline.CmdLineClientAdapter.commit(CmdLineClientAdapter.java:583) >>>>>>>>> at >>>>>>>>> org.tigris.subversion.svnclientadapter.commandline.CmdLineClientAdapter.commit(CmdLineClientAdapter.java:547) >>>>>>>>> at >>>>>>>>> org.wso2.carbon.deployment.synchronizer.subversion.SVNBasedArtifactRepository.commit(SVNBasedArtifactRepository.java:264) >>>>>>>>> at >>>>>>>>> org.wso2.carbon.deployment.synchronizer.internal.DeploymentSynchronizer.commit(DeploymentSynchronizer.java:132) >>>>>>>>> >>>>>>>>> Please advice. >>>>>>>>> Thank you. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> -- >>>>>>>>> Lahiru Sandaruwan >>>>>>>>> Software Engineer, >>>>>>>>> Cloud Team, >>>>>>>>> WSO2 Inc., http://wso2.com >>>>>>>>> lean.enterprise.middleware >>>>>>>>> >>>>>>>>> email: [email protected] cell: (+94) 773 325 954 >>>>>>>>> blog: http://lahiruwrites.blogspot.com/ >>>>>>>>> twitter: http://twitter.com/lahirus >>>>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks, >>>>>>>> Shariq. >>>>>>>> Phone: +94 777 202 225 >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> -- >>>>>>> Lahiru Sandaruwan >>>>>>> Software Engineer, >>>>>>> Cloud Team, >>>>>>> WSO2 Inc., http://wso2.com >>>>>>> lean.enterprise.middleware >>>>>>> >>>>>>> email: [email protected] cell: (+94) 773 325 954 >>>>>>> blog: http://lahiruwrites.blogspot.com/ >>>>>>> twitter: http://twitter.com/lahirus >>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Nuwan Dias >>>>>> >>>>>> Software Engineer - WSO2, Inc. >>>>>> Integration Technologies Team >>>>>> email : [email protected] >>>>>> Phone : +94 777 775 729 >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Nuwan Dias >>>>> >>>>> Software Engineer - WSO2, Inc. >>>>> Integration Technologies Team >>>>> email : [email protected] >>>>> Phone : +94 777 775 729 >>>>> >>>>> >>>> >>>> >>>> -- >>>> -- >>>> Lahiru Sandaruwan >>>> Software Engineer, >>>> Cloud Team, >>>> WSO2 Inc., http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> email: [email protected] cell: (+94) 773 325 954 >>>> blog: http://lahiruwrites.blogspot.com/ >>>> twitter: http://twitter.com/lahirus >>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>> >>> >>> >>> >>> -- >>> *Afkham Azeez* >>> Director of Architecture; WSO2, Inc.; http://wso2.com >>> Member; Apache Software Foundation; http://www.apache.org/ >>> * <http://www.apache.org/>** >>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>> twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>> * >>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>> * >>> * >>> *Lean . Enterprise . Middleware* >>> >>> >> >> >> -- >> -- >> Lahiru Sandaruwan >> Software Engineer, >> Cloud Team, >> WSO2 Inc., http://wso2.com >> lean.enterprise.middleware >> >> email: [email protected] cell: (+94) 773 325 954 >> blog: http://lahiruwrites.blogspot.com/ >> twitter: http://twitter.com/lahirus >> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >> >> _______________________________________________ >> Dev mailing list >> [email protected] >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Sagara Gunathunga > > Technical Lead; WSO2, Inc.; http://wso2.com > V.P Apache Web Services ; http://ws.apache.org/ > Blog ; http://ssagara.blogspot.com > > -- Nuwan Dias Software Engineer - WSO2, Inc. Integration Technologies Team email : [email protected] Phone : +94 777 775 729
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
