I'm not that familiar with blueprint's activation model, does it have an "all or nothing" model that doesn't respond to service changes after initialization?
Yes, I was thinking of changing the felix code to emit a thread dump when an acquire lock timeout occurs. I've found this very very helpful in understanding and fixing DS deadlocks. thanks! david jencks On Sep 27, 2013, at 11:04 PM, "Rudolph, Dirk" <[email protected]> wrote: > Hi David, > > for now we fixed the issue by starting blueprint container synchronusly > (-Dorg.apache.aries.blueprint.synchronous=true). Afterwards the similar > behavior occurred because a service registered by the blueprint container had > an mandatory reference to a service registered using DS. This could be solved > by setting the dependency to optional (very ugly ...). > > Do you mean creating thread dumps in the timeout code of acquireLock in > Felix.java? Yes this would be possible. I will give it a try. > > Thanks so far, > Dirk > > > -----Ursprüngliche Nachricht----- > Von: David Jencks [mailto:[email protected]] > Gesendet: Donnerstag, 26. September 2013 20:12 > An: [email protected] > Betreff: Re: Are bundles containing declerative services and blueprint > possible? > > I've looked but must be missing something, I don't see the deadlock in the > thread dump. In DS I added some code to do a thread dump on lock acquisition > timeout (AbstractComponentManager obtainLock/dumpThreads) Is there any > chance you could add something like this to the felix service registry and > see what comes out? > > thanks > david jencks > > On Sep 25, 2013, at 12:21 AM, "Rudolph, Dirk" <[email protected]> > wrote: > >> Thanks for your response. >> >> The state of the bundle is ACTIVE. Unfortunately the original Exception >> causing the previously attacked ISE isn't passed to the ISE itself and >> therefore not in the stacktrace: >> >> ERROR: [Thread[FelixDispatchQueue,5,main]] Waited too long to acquire the >> global lock own by Thread[FelixPackageAdmin,5,main]; giving up. >> java.lang.IllegalStateException: [Thread[Blueprint Extender: 3,5,main]] >> Waited too long to acquire lock for bundle ******* [516] owned by null >> (lockcount=0) >> at org.apache.felix.framework.Felix.acquireBundleLock(Felix.java:4756) >> at org.apache.felix.framework.Felix.registerService(Felix.java:2820) >> at >> org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:251) >> at >> org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:229) >> at >> org.apache.aries.jpa.container.context.impl.PersistenceContextManager.registerEM(PersistenceContextManager.java:286) >> at >> org.apache.aries.jpa.container.context.impl.PersistenceContextManager.registerContext(PersistenceContextManager.java:200) >> at >> org.apache.aries.jpa.container.context.impl.GlobalPersistenceManager.registerContext(GlobalPersistenceManager.java:123) >> at sun.reflect.GeneratedMethodAccessor398.invoke(Unknown Source) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) >> at java.lang.reflect.Method.invoke(Unknown Source) >> at >> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:54) >> at >> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:119) >> at $Proxy12.registerContext(Unknown Source) >> at >> org.apache.aries.jpa.blueprint.aries.impl.NSHandler.decorate(NSHandler.java:235) >> at sun.reflect.GeneratedMethodAccessor136.invoke(Unknown Source) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) >> at java.lang.reflect.Method.invoke(Unknown Source) >> at >> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:54) >> at >> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:119) >> at $Proxy9.decorate(Unknown Source) >> at >> org.apache.aries.blueprint.parser.Parser.decorateCustomNode(Parser.java:1273) >> at >> org.apache.aries.blueprint.parser.Parser.handleCustomElements(Parser.java:1263) >> at >> org.apache.aries.blueprint.parser.Parser.parseBeanMetadata(Parser.java:601) >> at >> org.apache.aries.blueprint.parser.Parser.parseBlueprintElement(Parser.java:393) >> at >> org.apache.aries.blueprint.parser.Parser.loadComponents(Parser.java:326) >> at org.apache.aries.blueprint.parser.Parser.populate(Parser.java:277) >> at >> org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:315) >> at >> org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:261) >> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) >> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) >> at java.util.concurrent.FutureTask.run(Unknown Source) >> at >> org.apache.aries.blueprint.container.ExecutorServiceWrapper.run(ExecutorServiceWrapper.java:106) >> at >> org.apache.aries.blueprint.utils.threading.impl.DiscardableRunnable.run(DiscardableRunnable.java:48) >> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) >> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) >> at java.util.concurrent.FutureTask.run(Unknown Source) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown >> Source) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown >> Source) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >> at java.lang.Thread.run(Unknown Source) >> >> This happens because m_globalLockThread isn't null. Just for my >> understanding: Why is it necessary for the FelixPackageAdmin Thread to hold >> a global lock and not a bundle lock only? >> >> Just for summary: A quick solution would be to split the bundle in one >> containing blueprint and one containing SCR. Another solution would be >> trying to update org.apache.felix.scr to the latest release (with >> adobe CQ this can cause "some" of problems ;-)) >> >> Attached you will find the full thread dump, captured on ISE breakpoint in >> registerService(). >> >> Thanks a lot, >> Dirk >> >> >> -----Ursprüngliche Nachricht----- >> Von: David Jencks [mailto:[email protected]] >> Gesendet: Mittwoch, 25. September 2013 00:23 >> An: [email protected] >> Betreff: Re: Are bundles containing declerative services and blueprint >> possible? >> >> I can't see any reason all known osgi component frameworks shouldn't >> all work at once in one bundle. (e.g DS, IPOJO, Blueprint) >> >> Is there a thread dump for when this problem originally occurs? Do you know >> what state the bundle is in when blueprint gets in trouble? What got the >> bundle into this state? >> >> If there's a deadlock between package admin and something else we should fix >> it. (I recall running into some deadlocks like this a long time ago, but >> don't recall the details). Please also try with DS trunk as there are a >> _lot_ of changes even from 1.6.2 and any fix would be made against trunk. >> >> thanks >> david jencks >> >> On Sep 24, 2013, at 3:06 PM, Michael Täschner <[email protected]> wrote: >> >>> Hi Dirk, >>> >>>>> So we assume that the scr bundle activation cannot coexists with >>>>> the >>> blueprint container for the same bundle. Is this right? >>> >>> Yes, your assumption is correct. One bundle should be managed by one >>> dependency injection implementation only. The DI is managed by the DI >>> frameworks via extender pattern on startup and depending on which >>> extender gets notified first (via bundlelistener) will try to initialize >>> the bundle. >>> Therefore you will run into locks or IllegalStateException depending >>> on the current order of extenders (blueprint, scr, ...) which runs first. >>> >>> Your approach might work - but I don't think it is good style - if >>> the "link" between the beans/components is realized via OSGi service >>> registry (e.g. registering a service using blueprint and a reference >>> in DS and vice >>> versa) still I would strongly suggest sticking to one framework >>> inside one bundle. >>> >>> The benefit of OSGi is the central service registry broker which >>> allows providers and consumers to use their preferred DI approach >>> inside their implementations independently from each other. >>> >>> Best Regards, >>> Michael >>> >>> >>> 2013/9/24 Rudolph, Dirk <[email protected]> >>> >>>> Hi all, >>>> >>>> >>>> >>>> in our project we make use of blueprint and declarative services. >>>> Both of them are part of one bundle running on Apache Felix >>>> (3.0.8.B006, CQ >>>> Version) with Apache Aries 1.0.1 (blueprint, jpa) >>>> >>>> >>>> >>>> Now we noticed, that there are some non-deterministic >>>> IllegalStateExceptions when we redeploy bundles and sometime when >>>> we deploy bundles the first time. (stacktrace attached) >>>> >>>> >>>> >>>> 24.09.2013 12:12:12.990 *ERROR* [Blueprint Extender: 1] >>>> org.apache.aries.blueprint.container.BlueprintContainerImpl Unable >>>> to start blueprint container for bundle ******************* >>>> java.lang.IllegalStateException: Can only register services while >>>> bundle is active or activating. >>>> >>>> at >>>> org.apache.felix.framework.Felix.registerService(Felix.java:2824) >>>> >>>> at >>>> org.apache.felix.framework.BundleContextImpl.registerService(BundleC >>>> o >>>> ntextImpl.java:251) >>>> >>>> at >>>> org.apache.felix.framework.BundleContextImpl.registerService(BundleC >>>> o >>>> ntextImpl.java:229) >>>> >>>> at >>>> org.apache.aries.jpa.container.context.impl.PersistenceContextManage >>>> r >>>> .registerEM(PersistenceContextManager.java:286) >>>> >>>> at >>>> org.apache.aries.jpa.container.context.impl.PersistenceContextManage >>>> r >>>> .registerContext(PersistenceContextManager.java:200) >>>> >>>> at >>>> org.apache.aries.jpa.container.context.impl.GlobalPersistenceManager. >>>> registerContext(GlobalPersistenceManager.java:123) >>>> >>>> at >>>> sun.reflect.GeneratedMethodAccessor398.invoke(Unknown >>>> Source) >>>> >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown >>>> Source) >>>> >>>> at java.lang.reflect.Method.invoke(Unknown Source) >>>> >>>> at >>>> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java: >>>> 5 >>>> 4) >>>> >>>> at >>>> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:11 >>>> 9 >>>> ) >>>> >>>> at $Proxy12.registerContext(Unknown Source) >>>> >>>> at >>>> org.apache.aries.jpa.blueprint.aries.impl.NSHandler.decorate(NSHandl >>>> e >>>> r.java:235) >>>> >>>> at >>>> sun.reflect.GeneratedMethodAccessor136.invoke(Unknown >>>> Source) >>>> >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown >>>> Source) >>>> >>>> at java.lang.reflect.Method.invoke(Unknown Source) >>>> >>>> at >>>> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java: >>>> 5 >>>> 4) >>>> >>>> at >>>> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:11 >>>> 9 >>>> ) >>>> >>>> at $Proxy9.decorate(Unknown Source) >>>> >>>> at >>>> org.apache.aries.blueprint.parser.Parser.decorateCustomNode(Parser.j >>>> a >>>> va:1273) >>>> >>>> at >>>> org.apache.aries.blueprint.parser.Parser.handleCustomElements(Parser. >>>> java:1263) >>>> >>>> at >>>> org.apache.aries.blueprint.parser.Parser.parseBeanMetadata(Parser.ja >>>> v >>>> a:601) >>>> >>>> at >>>> org.apache.aries.blueprint.parser.Parser.parseBlueprintElement(Parse >>>> r >>>> .java:393) >>>> >>>> at >>>> org.apache.aries.blueprint.parser.Parser.loadComponents(Parser.java: >>>> 3 >>>> 26) >>>> >>>> at >>>> org.apache.aries.blueprint.parser.Parser.populate(Parser.java:277) >>>> >>>> at >>>> org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(Bl >>>> u >>>> eprintContainerImpl.java:315) >>>> >>>> at >>>> org.apache.aries.blueprint.container.BlueprintContainerImpl.run(Blue >>>> p >>>> rintContainerImpl.java:261) >>>> >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) >>>> >>>> at >>>> java.util.concurrent.FutureTask$Sync.innerRun(Unknown >>>> Source) >>>> >>>> at java.util.concurrent.FutureTask.run(Unknown Source) >>>> >>>> at >>>> org.apache.aries.blueprint.container.ExecutorServiceWrapper.run(Exec >>>> u >>>> torServiceWrapper.java:106) >>>> >>>> at >>>> org.apache.aries.blueprint.utils.threading.impl.DiscardableRunnable. >>>> r >>>> un(DiscardableRunnable.java:48) >>>> >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) >>>> >>>> at >>>> java.util.concurrent.FutureTask$Sync.innerRun(Unknown >>>> Source) >>>> >>>> at java.util.concurrent.FutureTask.run(Unknown Source) >>>> >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask. >>>> access$201(Unknown >>>> Source) >>>> >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask. >>>> run(Unknown >>>> Source) >>>> >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) >>>> >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >>>> >>>> at java.lang.Thread.run(Unknown Source) >>>> >>>> >>>> >>>> After a while of debugging we figured out that the service cannot be >>>> registered because Felix fails to obtain the lock on the bundle. The >>>> lock is hold by the Thread running the Apache Felix >>>> BundleComponentActivator (FelixPackageAdmin). So we assume that the >>>> scr bundle activation cannot coexists with the blueprint container for the >>>> same bundle. Is this right? >>>> Would it be possible to trigger the blueprint container creation >>>> after scr has been processed or vice versa? >>>> >>>> >>>> >>>> Thanks so far, >>>> >>>> >>>> Dirk Rudolph >>>> >>>> >>>> >>>> >>>> T-Systems Multimedia Solutions GmbH >>>> Organisationseinheit CCS >>>> Dirk Rudolph >>>> Software-Entwicklung, OCJP >>>> >>>> Hausanschrift: Riesaer Straße 5, 01129 Dresden >>>> Postanschrift: Postfach 10 02 24, 01072 Dresden >>>> +49 351 2820-5363 (Tel) >>>> E-Mail: [email protected] >>>> <mailto:[email protected] >>>>> >>>> Internet: http://www.t-systems-mms.com >>>> <http://www.t-systems-mms.de/> >>>> >>>> T-Systems Multimedia Solutions GmbH >>>> >>>> Aufsichtsrat: Klaus Werner (Vorsitzender) >>>> >>>> Geschäftsführung: Peter Klingenburg, Susanne Heger >>>> >>>> Handelsregister: Amtsgericht Dresden HRB 11433 >>>> >>>> Sitz der Gesellschaft: Dresden >>>> >>>> Ust-IdNr.: DE 811 807 949 >>>> >>>> >>>> >>>> >>>> >>>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> <threaddump-bp.txt> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
