Hi Richard, I have a small patch for this which seems to work. Can provide it in an issue tomorrow.
Regards Felix "Richard S. Hall" <[email protected]> wrote: Felix, This situation looks like this to me: 1. First thread has the global lock and is trying to acquire a bundle lock. 2. Second thread wants the global lock. 3. Third thread has two bundle locks (via registerService()) and is blocked trying to get a third. It is not clear to me who is holding the bundle lock that is blocking the third thread. As far as I can guess, the first two threads do not hold a bundle lock. Are you sure you have all of the threads that are involved? Regarding the comment about firing events in the code, it is relevant, but I'd have to think about whether there was a way to refactor the service registry to make it possible to fire the event externally. -> richard On 6/9/11 15:19, Felix Meschberger wrote: > Hi all, > > We have a strange situation on a Framework 3.0.7 based system here which > is not reproducible on all platforms. > > We can track down a system freeze/deadlock to three threads all > contending for bundle locks and the global lock: > > >> "127.0.0.1 [1305717927375] GET /xxx.html HTTP/1.0" daemon prio=3 >> tid=0x0000000106203000 nid=0x2c01b in Object.wait() [0x00007ffe939f7000] >> java.lang.Thread.State: WAITING (on object monitor) >> at java.lang.Object.wait(Native Method) >> at java.lang.Object.wait(Object.java:485) >> at org.apache.felix.framework.Felix.acquireBundleLock(Felix.java:4712) >> - locked<0x00007ffeabca9908> (a [Ljava.lang.Object;) >> at >> org.apache.felix.framework.Felix$FelixResolver.markBundleResolved(Felix.java:4220) >> at >> org.apache.felix.framework.Felix$FelixResolver.markResolvedModules(Felix.java:4200) >> at >> org.apache.felix.framework.Felix$FelixResolver.resolve(Felix.java:3999) >> at org.apache.felix.framework.Felix.resolveBundle(Felix.java:3402) >> at org.apache.felix.framework.Felix.loadBundleClass(Felix.java:1594) >> at org.apache.felix.framework.BundleImpl.loadClass(BundleImpl.java:904) >> at >> org.apache.sling.commons.classloader.impl.PackageAdminClassLoader.findClass(PackageAdminClassLoader.java:150) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> - locked<0x00007ffecc57d040> (a >> org.apache.sling.commons.classloader.impl.PackageAdminClassLoader) >> at >> org.apache.sling.commons.classloader.impl.PackageAdminClassLoader.loadClass(PackageAdminClassLoader.java:174) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:296) >> - locked<0x00007ffecc58b260> (a >> org.apache.sling.jcr.classloader.internal.DynamicRepositoryClassLoader) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> at >> org.apache.jsp.apps.wci.components.page.page.page_jsp._jspService(page_jsp.java:170) >> at >> org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) >> at >> org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:419) >> at >> org.apache.sling.scripting.jsp.JspServletWrapperAdapter.service(JspServletWrapperAdapter.java:59) >> at >> org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:173) >> at >> org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:84) >> ... >> >> "ObservationManager" daemon prio=3 tid=0x000000010232e000 nid=0x30 in >> Object.wait() [0x00007ffef24fe000] >> java.lang.Thread.State: WAITING (on object monitor) >> at java.lang.Object.wait(Native Method) >> at java.lang.Object.wait(Object.java:485) >> at org.apache.felix.framework.Felix.acquireGlobalLock(Felix.java:4785) >> - locked<0x00007ffeabca9908> (a [Ljava.lang.Object;) >> at org.apache.felix.framework.Felix.access$400(Felix.java:80) >> at >> org.apache.felix.framework.Felix$FelixResolver.resolve(Felix.java:4043) >> at >> org.apache.felix.framework.ModuleImpl.searchDynamicImports(ModuleImpl.java:1412) >> at >> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:734) >> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71) >> at >> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> at >> org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645) >> at >> org.apache.felix.framework.ServiceRegistrationImpl$ServiceReferenceImpl.isAssignableTo(ServiceRegistrationImpl.java:544) >> at >> org.apache.felix.framework.util.Util.isServiceAssignable(Util.java:275) >> at >> org.apache.felix.framework.Felix.getServiceReferences(Felix.java:2920) >> at >> org.apache.felix.framework.Felix.getAllowedServiceReferences(Felix.java:2970) >> at >> org.apache.felix.framework.BundleContextImpl.getServiceReferences(BundleContextImpl.java:309) >> at >> org.apache.felix.eventadmin.impl.handler.BlacklistingHandlerTasks.createHandlerTasks(BlacklistingHandlerTasks.java:102) >> at >> org.apache.felix.eventadmin.impl.EventAdminImpl.postEvent(EventAdminImpl.java:92) >> at >> org.apache.felix.eventadmin.impl.security.EventAdminSecurityDecorator.postEvent(EventAdminSecurityDecorator.java:77) >> at >> org.apache.sling.jcr.resource.internal.JcrResourceListener.sendOsgiEvent(JcrResourceListener.java:224) >> at >> org.apache.sling.jcr.resource.internal.JcrResourceListener.onEvent(JcrResourceListener.java:176) >> at >> org.apache.jackrabbit.core.observation.EventConsumer.consumeEvents(EventConsumer.java:248) >> at >> org.apache.jackrabbit.core.observation.ObservationDispatcher.run(ObservationDispatcher.java:161) >> at java.lang.Thread.run(Thread.java:662) >> >> "OsgiInstallerImpl" daemon prio=3 tid=0x0000000104f71800 nid=0x39 in >> Object.wait() [0x00007ffef12fc000] >> java.lang.Thread.State: WAITING (on object monitor) >> at java.lang.Object.wait(Native Method) >> at java.lang.Object.wait(Object.java:485) >> at org.apache.felix.framework.Felix.acquireBundleLock(Felix.java:4712) >> - locked<0x00007ffeabca9908> (a [Ljava.lang.Object;) >> at org.apache.felix.framework.Felix.registerService(Felix.java:2813) >> at >> org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:251) >> at >> org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:449) >> at >> org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:460) >> at >> org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1009) >> at >> org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:333) >> at >> org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:283) >> at >> org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:170) >> at >> org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:871) >> at >> org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:733) >> at >> org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:662) >> at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:3735) >> at org.apache.felix.framework.Felix.access$000(Felix.java:80) >> at org.apache.felix.framework.Felix$2.serviceChanged(Felix.java:722) >> at >> org.apache.felix.framework.ServiceRegistry.registerService(ServiceRegistry.java:107) >> at org.apache.felix.framework.Felix.registerService(Felix.java:2847) >> at >> org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:251) >> at >> org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:449) >> at >> org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:460) >> at >> org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1009) >> at >> org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:333) >> at >> org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:283) >> at >> org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:170) >> at >> org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:871) >> at >> org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:733) >> at >> org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:662) >> at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:3735) >> at org.apache.felix.framework.Felix.access$000(Felix.java:80) >> at org.apache.felix.framework.Felix$2.serviceChanged(Felix.java:722) >> at >> org.apache.felix.framework.ServiceRegistry.registerService(ServiceRegistry.java:107) >> at org.apache.felix.framework.Felix.registerService(Felix.java:2847) >> at >> org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:251) >> at >> org.apache.sling.commons.classloader.impl.Activator.registerManagerFactory(Activator.java:78) >> at >> org.apache.sling.commons.classloader.impl.Activator.bundleChanged(Activator.java:122) >> at >> org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:807) >> at >> org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:729) >> at >> org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:610) >> at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:3724) >> at org.apache.felix.framework.Felix.updateBundle(Felix.java:2030) >> at org.apache.felix.framework.BundleImpl.update(BundleImpl.java:940) >> at >> org.apache.sling.installer.core.impl.tasks.BundleUpdateTask.execute(BundleUpdateTask.java:78) >> at >> org.apache.sling.installer.core.impl.OsgiInstallerImpl.executeTasks(OsgiInstallerImpl.java:522) >> at >> org.apache.sling.installer.core.impl.OsgiInstallerImpl.run(OsgiInstallerImpl.java:175) > Looking at the Framework source, particularly acquireBundleLock and > acquireGlobalLock I > cannot see where this "deadlock" can occur. > > The only hint I have is a note in the Felix.registerService: > >> // TODO: CONCURRENCY - Reconsider firing event here, outside of the >> // bundle lock. > I wonder whether this situation can be fixed with moving the service > registration > event ? > > Regards > Felix >
