Weird deadlock between CXF and spring on WAS.
---------------------------------------------
Key: CXF-1839
URL: https://issues.apache.org/jira/browse/CXF-1839
Project: CXF
Issue Type: Bug
Components: JAX-WS Runtime
Affects Versions: 2.1.1
Environment: Windows XP, Ubuntu 7.04, jdk 1.5.0_09, CXF 2.1, spring
2.0.2, quartz 1.6.1, WAS 6.1
Reporter: Alton Idowu
We are using quartz to make periodic calls to different web services. Every
once in a while we get a deadlock that eventually crashes our WAS server. The
stack trace below does not make sense to me. it appears that worker-5 has a
lock on the LinkedHashMap containing the spring singleton beans that worker-7
is attempt to access in the synchronized configurebeans method. And worker -5
is waiting for worker-7 to exit the configurebeans method. I took a quick look
at the code and it didn't seem that worker-5 kept a reference. It would be
great if someone point us in the right direction to solve this.
Let me know if you require additional information....
Found one Java-level deadlock:
=============================
"clusteredScheduler_Worker-5":
waiting to lock monitor 0x0000000101f4a6b8 (object 0xfffffffed1c55700, a
org.apache.cxf.configuration.spring.ConfigurerImpl),
which is held by "clusteredScheduler_Worker-7"
"clusteredScheduler_Worker-7":
waiting to lock monitor 0x0000000101f4af40 (object 0xfffffffed1b36d20, a
java.util.LinkedHashMap),
which is held by "clusteredScheduler_Worker-5"
Java stack information for the threads listed above:
===================================================
"clusteredScheduler_Worker-5":
at
org.apache.cxf.configuration.spring.ConfigurerImpl.configureBean(ConfigurerImpl.java:93)
- waiting to lock <0xfffffffed1c55700> (a
org.apache.cxf.configuration.spring.ConfigurerImpl)
at
org.apache.cxf.configuration.spring.ConfigurerImpl.configureBean(ConfigurerImpl.java:88)
at
org.apache.cxf.transport.http.AbstractHTTPTransportFactory.configure(AbstractHTTPTransportFactory.java:223)
at
org.apache.cxf.transport.http.AbstractHTTPTransportFactory.getConduit(AbstractHTTPTransportFactory.java:163)
at
org.apache.cxf.transport.http.AbstractHTTPTransportFactory.getConduit(AbstractHTTPTransportFactory.java:145)
at
org.apache.cxf.endpoint.AbstractConduitSelector.getSelectedConduit(AbstractConduitSelector.java:73)
at
org.apache.cxf.endpoint.UpfrontConduitSelector.selectConduit(UpfrontConduitSelector.java:71)
at org.apache.cxf.endpoint.ClientImpl.getConduit(ClientImpl.java:468)
at
ilss.suidm.dao.ltc.service.LtcClientFactoryBean.create(LtcClientFactoryBean.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:102)
at
org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:275)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:727)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:679)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:388)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:250)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:141)
- locked <0xfffffffed1b36d20> (a java.util.LinkedHashMap)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:247)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:161)
at
org.springframework.aop.target.LazyInitTargetSource.getTarget(LazyInitTargetSource.java:66)
- locked <0xfffffffed22b0bf0> (a
org.springframework.aop.target.LazyInitTargetSource)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:184)
at $Proxy2.processTransaction(Unknown Source)
at ilss.suidm.dao.ltc.LtcDAO.processTransaction(LtcDAO.java:80)
at
ilss.sysmonitor.delegate.SystemMonitoringDelegate.runSystemTestLTC(SystemMonitoringDelegate.java:544)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
ilss.common.reflect.ReflectionUtil.invokeMethod(ReflectionUtil.java:294)
at
ilss.common.reflect.ReflectionUtil.invokeMethodByForce(ReflectionUtil.java:73)
at
ilss.sysmonitor.delegate.SystemMonitoringDelegate.runSystemTests(SystemMonitoringDelegate.java:524)
at
ilss.common.cron.sysmonitor.BaseSysMonitorTestJobBean.executeJob(BaseSysMonitorTestJobBean.java:36)
at ilss.common.cron.BaseJobBean.executeInternal(BaseJobBean.java:37)
at
org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
"clusteredScheduler_Worker-7":
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.containsSingleton(DefaultSingletonBeanRegistry.java:164)
- waiting to lock <0xfffffffed1b36d20> (a java.util.LinkedHashMap)
at
org.springframework.beans.factory.support.AbstractBeanFactory.containsLocalBean(AbstractBeanFactory.java:511)
at
org.springframework.beans.factory.support.AbstractBeanFactory.containsBean(AbstractBeanFactory.java:319)
at
org.springframework.context.support.AbstractApplicationContext.containsBean(AbstractApplicationContext.java:650)
at
org.apache.cxf.configuration.spring.ConfigurerImpl.configureBean(ConfigurerImpl.java:118)
- locked <0xfffffffed1c55700> (a
org.apache.cxf.configuration.spring.ConfigurerImpl)
at
org.apache.cxf.jaxws.ServiceImpl.configureObject(ServiceImpl.java:448)
at
org.apache.cxf.jaxws.ServiceImpl.configureObject(ServiceImpl.java:442)
at org.apache.cxf.jaxws.ServiceImpl.createPort(ServiceImpl.java:382)
at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:247)
at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:234)
at javax.xml.ws.Service.getPort(Service.java:40)
at
mil.dla.ide.glsc_ots.GLSCOTSIdeService.getGLSCOTSIdePort0(GLSCOTSIdeService.java:65)
at
ilss.dla.dao.service.DlaServiceClient.getPort(DlaServiceClient.java:89)
at
ilss.dla.dao.service.DlaServiceClient.getRequisitionStatusByReqNum(DlaServiceClient.java:78)
at ilss.dla.dao.DlaQueryDAO.getRequisitionsByReqNum(DlaQueryDAO.java:98)
at
ilss.sysmonitor.delegate.SystemMonitoringDelegate.runSystemTestDLA_WS(SystemMonitoringDelegate.java:569)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
ilss.common.reflect.ReflectionUtil.invokeMethod(ReflectionUtil.java:294)
at
ilss.common.reflect.ReflectionUtil.invokeMethodByForce(ReflectionUtil.java:73)
at
ilss.sysmonitor.delegate.SystemMonitoringDelegate.runSystemTests(SystemMonitoringDelegate.java:524)
at
ilss.common.cron.sysmonitor.BaseSysMonitorTestJobBean.executeJob(BaseSysMonitorTestJobBean.java:36)
at ilss.common.cron.BaseJobBean.executeInternal(BaseJobBean.java:37)
at
org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Found 1 deadlock.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
