I have observed this few times even without patch0002. When analyzed deadloak flow it seems hot deployment of main, fault sequences and deleting ESB artifacts from admin services triggers in parallel.
in hot deployment, axis2 thread first get a lock of axis2 configuration and then calls synapse deployer.undeploy which will lock on synapse configuration. during admin service call, it will directly call synapse to undeploy artifacts which will first lock on synapse config and in case of proxy, it will cause related axis2 service to undeploy and try to acquire lock of axis2 config which will become a deadlock. I have modified abstract ESBIntegrationTest.cleanup() method to wait few seconds to let main and fault sequence to hot update. Seems deadlock does not happens after that. On Thu, Nov 14, 2013 at 10:17 PM, Miyuru Wanninayaka <[email protected]>wrote: > Seems patch0002 causing deadlock. Once patch0002 removed from pack, tests > are running without any issue. > > > On Thu, Nov 14, 2013 at 9:31 PM, Krishantha Samaraweera < > [email protected]> wrote: > >> Integration tests hang at the early stage and following dead lock >> situation detected in ESB Alpha pack. >> >> Name: http-nio-9443-exec-19 >> State: BLOCKED on org.apache.axis2.engine.AxisConfiguration@510b02daowned >> by: pool-15-thread-1 >> Total blocked: 2 Total waited: 29 >> >> Stack trace: >> >> >> org.apache.axis2.engine.AxisConfiguration.removeService(AxisConfiguration.java:692) >> >> org.apache.synapse.config.SynapseConfiguration.removeProxyService(SynapseConfiguration.java:1037) >> - locked org.apache.synapse.config.SynapseConfiguration@73e64d34 >> >> org.wso2.carbon.proxyadmin.service.ProxyServiceAdmin.deleteProxyService(ProxyServiceAdmin.java:398) >> sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source) >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> java.lang.reflect.Method.invoke(Method.java:597) >> >> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212) >> >> org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:117) >> >> org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40) >> >> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110) >> org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >> >> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172) >> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146) >> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java: >> >> >> --------------------- >> >> Name: pool-15-thread-1 >> State: BLOCKED on >> org.apache.synapse.config.SynapseConfiguration@73e64d34owned by: >> http-nio-9443-exec-19 >> Total blocked: 1 Total waited: 17 >> >> Stack trace: >> >> >> org.apache.synapse.config.SynapseConfiguration.getDefinedSequences(SynapseConfiguration.java:289) >> >> org.apache.synapse.deployers.SequenceDeployer.updateSynapseArtifact(SequenceDeployer.java:120) >> >> org.wso2.carbon.sequences.SequenceDeploymentInterceptor.updateSynapseArtifact(SequenceDeploymentInterceptor.java:49) >> >> org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.resetDefaultSequence(AbstractSynapseArtifactDeployer.java:584) >> >> org.apache.synapse.deployers.SequenceDeployer.updatingFaultAndMainWithOriginal(SequenceDeployer.java:181) >> >> org.apache.synapse.deployers.SequenceDeployer.undeploy(SequenceDeployer.java:150) >> >> org.apache.axis2.deployment.DeploymentEngine.unDeploy(DeploymentEngine.java:909) >> - locked org.wso2.carbon.core.CarbonAxisConfigurator@5497eb53 >> >> org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:143) >> - locked org.wso2.carbon.core.CarbonAxisConfigurator@5497eb53 >> >> org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377) >> >> org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254) >> >> org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371) >> >> -- >> Krishantha Samaraweera >> Senior Technical Lead - Test Automation >> Mobile: +94 77 7759918 >> WSO2, Inc.; http://wso2.com/ >> lean . enterprise . middlewear. >> > > > > -- > Miyuru Wanninayaka > Technical Lead > WSO2 Inc. : http://wso2.com > > Mobile : +94 77 209 9788 > Blog : http://miyurudw.blogspot.com > Flickr : http://www.flickr.com/photos/miyuru_daminda > -- Miyuru Wanninayaka Technical Lead WSO2 Inc. : http://wso2.com Mobile : +94 77 209 9788 Blog : http://miyurudw.blogspot.com Flickr : http://www.flickr.com/photos/miyuru_daminda
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
