Re: Deadlock in DatabasePersistenceManager

[Nick Stolwijk]

We've found the cause of our problem. Our database connection is being 
dropped by a firewall between the application server and the database 
server. That causes the database driver (oracle) to hang itself by not 
specifying a read time out on java.net.Socket.read. (This feature is 
implemented in a later version of the driver)

However, we also want to use a database pool instead of the one 
connection Jackrabbit uses. For this I'm looking into a one connection 
per session. Are there any known drawbacks in that scenario? Would this 
be a patch the jackrabbit community would be interested in?

With regards,

Nick Stolwijk


Marcel Reutegger wrote:
Hi Nick,

the three thread dumps do not show a deadlock.

thread 20 executes a SQL statement in the persistence manager. threads 
19 and 21 wait until thread 20 returns from the call to Oracle.

regards
 marcel

Nick Stolwijk wrote:
Has someone an idea in which direction we have to search this? Is it 
a bug in our usage, or maybe in jackrabbit?

With regards,

Nick Stolwijk

Nick Stolwijk wrote:
Sorry for the lack of information. I will try to draw a bigger picture.

Our modification on the 1.2.1 tag consists of combining the Oracle 
and JNDI classes to use them both. See [1] Patching the 
OracleFileSystem and OraclePersistenceManager for use with JNDI.

The stacktraces of the threaddump is as following:

<"ExecuteThread: '21' for queue: 'weblogic.kernel.Default'" daemon 
prio=1 tid=0x76888760 nid=0x569a waiting for monitor entry 
[7574f000..7575187c]
<    at 
org.apache.jackrabbit.core.persistence.db.DatabasePersistenceManager.exists(DatabasePersistenceManager.java:645) 

<    - waiting to lock <0x8a7ae690> (a java.lang.String)
<    at 
org.apache.jackrabbit.core.state.SharedItemStateManager.hasNonVirtualItemState(SharedItemStateManager.java:1050) 

<    at 
org.apache.jackrabbit.core.state.SharedItemStateManager.hasItemState(SharedItemStateManager.java:287) 

<    at 
org.apache.jackrabbit.core.state.LocalItemStateManager.hasItemState(LocalItemStateManager.java:180) 

<    at 
org.apache.jackrabbit.core.state.XAItemStateManager.hasItemState(XAItemStateManager.java:252) 

<    at 
org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(SessionItemStateManager.java:163) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.getItemState(HierarchyManagerImpl.java:120) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:267) 

<    at 
org.apache.jackrabbit.core.CachingHierarchyManager.resolvePath(CachingHierarchyManager.java:132) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:203) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:341) 

<    at 
org.apache.jackrabbit.core.CachingHierarchyManager.resolvePath(CachingHierarchyManager.java:193) 

<    at 
org.apache.jackrabbit.core.NodeImpl.resolveRelativeNodePath(NodeImpl.java:243) 

<    at org.apache.jackrabbit.core.NodeImpl.hasNode(NodeImpl.java:2507)
<    at 
nl.anwb.autoportaal.dao.jackrabbit.MediaRepositoryImpl$1.doInJcr(MediaRepositoryImpl.java:44) 

<    at org.springmodules.jcr.JcrTemplate.execute(JcrTemplate.java:76)
<    at org.springmodules.jcr.JcrTemplate.execute(JcrTemplate.java:108)
<    at 
nl.anwb.autoportaal.dao.jackrabbit.MediaRepositoryImpl.getMedia(MediaRepositoryImpl.java:40) 

<    at 
nl.anwb.autoportaal.facade.impl.CommonContextFacadeImpl.getMedia(CommonContextFacadeImpl.java:117) 

<    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 

<    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 

<    at java.lang.reflect.Method.invoke(Method.java:324)
<    at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:280) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:187) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:154) 

<    at 
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176) 

<    at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:210) 

<    at $Proxy19.getMedia(Unknown Source)
<    at 
nl.anwb.autoportaal.web.search.ModelTekstSearchBean.getModelTekst(ModelTekstSearchBean.java:39) 

<    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 

<    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 

<    at java.lang.reflect.Method.invoke(Method.java:324)
<    at 
org.apache.myfaces.el.PropertyResolverImpl.getProperty(PropertyResolverImpl.java:438) 

<    at 
org.apache.myfaces.el.PropertyResolverImpl.getValue(PropertyResolverImpl.java:82) 

<    at 
com.sun.facelets.el.LegacyELContext$LegacyELResolver.getValue(LegacyELContext.java:141) 

<    at com.sun.el.parser.AstValue.getValue(AstValue.java:96)
<    at 
com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:183)
<    at 
com.sun.facelets.el.ELText$ELTextVariable.writeText(ELText.java:184)
<    at 
com.sun.facelets.compiler.TextInstruction.write(TextInstruction.java:45) 

<    at 
com.sun.facelets.compiler.UIInstructions.encodeBegin(UIInstructions.java:39) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:232) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:580) 

<    at 
org.ajax4jsf.framework.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:101) 

<    at 
org.ajax4jsf.framework.ajax.AjaxViewHandler.renderView(AjaxViewHandler.java:222) 

<    at 
org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:384) 

<    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:138)
<    at 
weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1006) 

<    at 
weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419) 

<    at 
weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)
<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:144) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.ajax4jsf.framework.ajax.xmlfilter.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:75) 

<    at 
org.ajax4jsf.framework.ajax.xmlfilter.BaseFilter.doFilter(BaseFilter.java:213) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:119) 

<    at 
com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:55) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:174) 

<    at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264) 

<    at 
org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107) 

<    at 
org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:217) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:191) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148) 

<    at 
org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:61) 

<    at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
nl.anwb.autoportaal.web.filters.SessionExpireFilter.doFilter(SessionExpireFilter.java:74) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6724) 

<    at 
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) 

<    at 
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121) 

<    at 
weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3764) 

<    at 
weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2644) 

<    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
<    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

<"ExecuteThread: '20' for queue: 'weblogic.kernel.Default'" daemon 
prio=1 tid=0x76887c10 nid=0x569a runnable [757d0000..757d287c]
<    at java.net.SocketInputStream.socketRead0(Native Method)
<    at java.net.SocketInputStream.read(SocketInputStream.java:129)
<    at oracle.net.ns.Packet.receive(Unknown Source)
<    at oracle.net.ns.DataPacket.receive(Unknown Source)
<    at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
<    at oracle.net.ns.NetInputStream.read(Unknown Source)
<    at oracle.net.ns.NetInputStream.read(Unknown Source)
<    at oracle.net.ns.NetInputStream.read(Unknown Source)
<    at 
oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:971)
<    at 
oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:941)
<    at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:432)
<    at 
oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:181) 

<    at 
oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows(T4CPreparedStatement.java:543) 

<    at 
oracle.jdbc.driver.OracleStatement.execute_maybe_describe(OracleStatement.java:913) 

<    at 
oracle.jdbc.driver.T4CPreparedStatement.execute_maybe_describe(T4CPreparedStatement.java:452) 

<    at 
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:984) 

<    at 
oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:2885) 

<    at 
oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:2976) 

<    - locked <0x8a7da210> (a oracle.jdbc.driver.T4CPreparedStatement)
<    - locked <0x8a7b5ba8> (a oracle.jdbc.driver.T4CConnection)
<    at 
weblogic.jdbc.wrapper.PreparedStatement.execute(PreparedStatement.java:102) 

<    at 
weblogic.jdbc.wrapper.PreparedStatement_oracle_jdbc_driver_T4CPreparedStatement.execute(Unknown 
Source)
<    at 
org.apache.jackrabbit.core.persistence.db.DatabasePersistenceManager.executeStmt(DatabasePersistenceManager.java:835) 

<    at 
org.apache.jackrabbit.core.persistence.db.DatabasePersistenceManager.exists(DatabasePersistenceManager.java:647) 

<    - locked <0x8a7ae690> (a java.lang.String)
<    at 
org.apache.jackrabbit.core.state.SharedItemStateManager.hasNonVirtualItemState(SharedItemStateManager.java:1050) 

<    at 
org.apache.jackrabbit.core.state.SharedItemStateManager.hasItemState(SharedItemStateManager.java:287) 

<    at 
org.apache.jackrabbit.core.state.LocalItemStateManager.hasItemState(LocalItemStateManager.java:180) 

<    at 
org.apache.jackrabbit.core.state.XAItemStateManager.hasItemState(XAItemStateManager.java:252) 

<    at 
org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(SessionItemStateManager.java:163) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.getItemState(HierarchyManagerImpl.java:120) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:267) 

<    at 
org.apache.jackrabbit.core.CachingHierarchyManager.resolvePath(CachingHierarchyManager.java:132) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:203) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:341) 

<    at 
org.apache.jackrabbit.core.CachingHierarchyManager.resolvePath(CachingHierarchyManager.java:193) 

<    at 
org.apache.jackrabbit.core.NodeImpl.resolveRelativeNodePath(NodeImpl.java:243) 

<    at org.apache.jackrabbit.core.NodeImpl.hasNode(NodeImpl.java:2507)
<    at 
nl.anwb.autoportaal.dao.jackrabbit.MediaRepositoryImpl$1.doInJcr(MediaRepositoryImpl.java:44) 

<    at org.springmodules.jcr.JcrTemplate.execute(JcrTemplate.java:76)
<    at org.springmodules.jcr.JcrTemplate.execute(JcrTemplate.java:108)
<    at 
nl.anwb.autoportaal.dao.jackrabbit.MediaRepositoryImpl.getMedia(MediaRepositoryImpl.java:40) 

<    at 
nl.anwb.autoportaal.facade.impl.CommonContextFacadeImpl.getMedia(CommonContextFacadeImpl.java:117) 

<    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 

<    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 

<    at java.lang.reflect.Method.invoke(Method.java:324)
<    at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:280) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:187) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:154) 

<    at 
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176) 

<    at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:210) 

<    at $Proxy19.getMedia(Unknown Source)
<    at 
nl.anwb.autoportaal.web.search.ModelTekstSearchBean.getModelTekst(ModelTekstSearchBean.java:39) 

<    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 

<    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 

<    at java.lang.reflect.Method.invoke(Method.java:324)
<    at 
org.apache.myfaces.el.PropertyResolverImpl.getProperty(PropertyResolverImpl.java:438) 

<    at 
org.apache.myfaces.el.PropertyResolverImpl.getValue(PropertyResolverImpl.java:82) 

<    at 
com.sun.facelets.el.LegacyELContext$LegacyELResolver.getValue(LegacyELContext.java:141) 

<    at com.sun.el.parser.AstValue.getValue(AstValue.java:96)
<    at 
com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:183)
<    at 
com.sun.facelets.el.ELText$ELTextVariable.writeText(ELText.java:184)
<    at 
com.sun.facelets.compiler.TextInstruction.write(TextInstruction.java:45) 

<    at 
com.sun.facelets.compiler.UIInstructions.encodeBegin(UIInstructions.java:39) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:232) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:580) 

<    at 
org.ajax4jsf.framework.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:101) 

<    at 
org.ajax4jsf.framework.ajax.AjaxViewHandler.renderView(AjaxViewHandler.java:222) 

<    at 
org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:384) 

<    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:138)
<    at 
weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1006) 

<    at 
weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419) 

<    at 
weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)
<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:144) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.ajax4jsf.framework.ajax.xmlfilter.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:75) 

<    at 
org.ajax4jsf.framework.ajax.xmlfilter.BaseFilter.doFilter(BaseFilter.java:213) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:119) 

<    at 
com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:55) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:174) 

<    at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264) 

<    at 
org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107) 

<    at 
org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:217) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:191) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148) 

<    at 
org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:61) 

<    at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
nl.anwb.autoportaal.web.filters.SessionExpireFilter.doFilter(SessionExpireFilter.java:74) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6724) 

<    at 
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) 

<    at 
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121) 

<    at 
weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3764) 

<    at 
weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2644) 

<    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
<    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)


<"ExecuteThread: '19' for queue: 'weblogic.kernel.Default'" daemon 
prio=1 tid=0x768870c0 nid=0x569a waiting for monitor entry 
[75851000..7585387c]
<    at 
org.apache.jackrabbit.core.persistence.db.DatabasePersistenceManager.exists(DatabasePersistenceManager.java:645) 

<    - waiting to lock <0x8a7ae690> (a java.lang.String)
<    at 
org.apache.jackrabbit.core.state.SharedItemStateManager.hasNonVirtualItemState(SharedItemStateManager.java:1050) 

<    at 
org.apache.jackrabbit.core.state.SharedItemStateManager.hasItemState(SharedItemStateManager.java:287) 

<    at 
org.apache.jackrabbit.core.state.LocalItemStateManager.hasItemState(LocalItemStateManager.java:180) 

<    at 
org.apache.jackrabbit.core.state.XAItemStateManager.hasItemState(XAItemStateManager.java:252) 

<    at 
org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(SessionItemStateManager.java:163) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.getItemState(HierarchyManagerImpl.java:120) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:267) 

<    at 
org.apache.jackrabbit.core.CachingHierarchyManager.resolvePath(CachingHierarchyManager.java:132) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:203) 

<    at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:341) 

<    at 
org.apache.jackrabbit.core.CachingHierarchyManager.resolvePath(CachingHierarchyManager.java:193) 

<    at 
org.apache.jackrabbit.core.NodeImpl.resolveRelativeNodePath(NodeImpl.java:243) 

<    at org.apache.jackrabbit.core.NodeImpl.hasNode(NodeImpl.java:2507)
<    at 
nl.anwb.autoportaal.dao.jackrabbit.MediaRepositoryImpl$1.doInJcr(MediaRepositoryImpl.java:44) 

<    at org.springmodules.jcr.JcrTemplate.execute(JcrTemplate.java:76)
<    at org.springmodules.jcr.JcrTemplate.execute(JcrTemplate.java:108)
<    at 
nl.anwb.autoportaal.dao.jackrabbit.MediaRepositoryImpl.getMedia(MediaRepositoryImpl.java:40) 

<    at 
nl.anwb.autoportaal.facade.impl.CommonContextFacadeImpl.getMedia(CommonContextFacadeImpl.java:117) 

<    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 

<    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 

<    at java.lang.reflect.Method.invoke(Method.java:324)
<    at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:280) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:187) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:154) 

<    at 
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107) 

<    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176) 

<    at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:210) 

<    at $Proxy19.getMedia(Unknown Source)
<    at 
nl.anwb.autoportaal.web.search.ModelTekstSearchBean.getModelTekst(ModelTekstSearchBean.java:39) 

<    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 

<    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 

<    at java.lang.reflect.Method.invoke(Method.java:324)
<    at 
org.apache.myfaces.el.PropertyResolverImpl.getProperty(PropertyResolverImpl.java:438) 

<    at 
org.apache.myfaces.el.PropertyResolverImpl.getValue(PropertyResolverImpl.java:82) 

<    at 
com.sun.facelets.el.LegacyELContext$LegacyELResolver.getValue(LegacyELContext.java:141) 

<    at com.sun.el.parser.AstValue.getValue(AstValue.java:96)
<    at 
com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:183)
<    at 
com.sun.facelets.el.ELText$ELTextVariable.writeText(ELText.java:184)
<    at 
com.sun.facelets.compiler.TextInstruction.write(TextInstruction.java:45) 

<    at 
com.sun.facelets.compiler.UIInstructions.encodeBegin(UIInstructions.java:39) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:232) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.tag.jsf.ComponentSupport.encodeRecursive(ComponentSupport.java:239) 

<    at 
com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:580) 

<    at 
org.ajax4jsf.framework.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:101) 

<    at 
org.ajax4jsf.framework.ajax.AjaxViewHandler.renderView(AjaxViewHandler.java:222) 

<    at 
org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:384) 

<    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:138)
<    at 
weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1006) 

<    at 
weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419) 

<    at 
weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)
<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:144) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.ajax4jsf.framework.ajax.xmlfilter.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:75) 

<    at 
org.ajax4jsf.framework.ajax.xmlfilter.BaseFilter.doFilter(BaseFilter.java:213) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:119) 

<    at 
com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:55) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:174) 

<    at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264) 

<    at 
org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107) 

<    at 
org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:217) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:191) 

<    at 
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) 

<    at 
org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148) 

<    at 
org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:61) 

<    at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
nl.anwb.autoportaal.web.filters.SessionExpireFilter.doFilter(SessionExpireFilter.java:74) 

<    at 
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:27) 

<    at 
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6724) 

<    at 
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) 

<    at 
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121) 

<    at 
weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3764) 

<    at 
weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2644) 

<    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
<    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)


We first used jackrabbit to only query existing nodes. A while ago 
we also started to query non-existing nodes (Files with text) The 
method 
nl.anwb.autoportaal.web.search.ModelTekstSearchBean.getModelTekst(ModelTekstSearchBean.java:39 
is used for this.

It is hard to test, because the only place we have been able to 
reproduce this is the acceptance server at our customers site.

I hope this is enough information. If not, please ask.

Nick Stolwijk

[1] http://wiki.apache.org/jackrabbit/ExamplesPage

Stefan Guggisberg wrote:
hi nick,

On 5/21/07, Nick Stolwijk <[EMAIL PROTECTED]> wrote:
We have a problem with a deadlock in Jackrabbit. We are using version
1.2.1 with some modifications to use a Oracle connection through 
JNDI.

Now we have a deadlock in the DatabasePersistenceManager. Can someone
explain to me what we're doing wrong.

well that's going to be very difficult without knowing what exactly
you're doing
and what exactly your modifcations consist of ;) we'd also need a 
jvm thread
dump including all threads involved in the dead lock.

btw: are you also experiencing dead locks with an out-of-the-box 
jackrabbit
release?

cheers
stefan


Stacktrace:

"ExecuteThread: '21' for queue: 'weblogic.kernel.Default'" daemon 
prio=1 tid=0x76888760 nid=0x569a waiting for monitor entry 
[7574f000..7575187c]
<       at 
org.apache.jackrabbit.core.persistence.db.DatabasePersistenceManager.exists(DatabasePersistenceManager.java:645) 

<       - waiting to lock <0x8a7ae690> (a java.lang.String)
<       at 
org.apache.jackrabbit.core.state.SharedItemStateManager.hasNonVirtualItemState(SharedItemStateManager.java:1050) 

<       at 
org.apache.jackrabbit.core.state.SharedItemStateManager.hasItemState(SharedItemStateManager.java:287) 

<       at 
org.apache.jackrabbit.core.state.LocalItemStateManager.hasItemState(LocalItemStateManager.java:180) 

<       at 
org.apache.jackrabbit.core.state.XAItemStateManager.hasItemState(XAItemStateManager.java:252) 

<       at 
org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(SessionItemStateManager.java:163) 

<       at 
org.apache.jackrabbit.core.HierarchyManagerImpl.getItemState(HierarchyManagerImpl.java:120) 

<       at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:267) 

<       at 
org.apache.jackrabbit.core.CachingHierarchyManager.resolvePath(CachingHierarchyManager.java:132) 

<       at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:203) 

<       at 
org.apache.jackrabbit.core.HierarchyManagerImpl.resolvePath(HierarchyManagerImpl.java:341) 

<       at 
org.apache.jackrabbit.core.CachingHierarchyManager.resolvePath(CachingHierarchyManager.java:193) 

<       at 
org.apache.jackrabbit.core.NodeImpl.resolveRelativeNodePath(NodeImpl.java:243) 

<       at 
org.apache.jackrabbit.core.NodeImpl.hasNode(NodeImpl.java:2507)
<       at 
nl.anwb.autoportaal.dao.jackrabbit.MediaRepositoryImpl$1.doInJcr(MediaRepositoryImpl.java:44) 

<       at 
org.springmodules.jcr.JcrTemplate.execute(JcrTemplate.java:76)
<       at 
org.springmodules.jcr.JcrTemplate.execute(JcrTemplate.java:108)
<       at 
nl.anwb.autoportaal.dao.jackrabbit.MediaRepositoryImpl.getMedia(MediaRepositoryImpl.java:40) 



Environment:
Weblogic 8.1
Oracle 9
Weblogic connection pool with 15 connections
Jackrabbit 1.2.1

I hope someone can help me.

Nick Stolwijk