Looks like ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIds is involved in some way. I've replaced it with ec.faultForGlobalID(gid, ec) and so far no deadlock (finger crossed).
Alex 2012/2/13 Alexis Tual <[email protected]> > Thanks for your quick answer. > None of the two threads are launched by ERXExecutorService. One is managed > by Quartz, the other one is in fact a TimerTask so launched in a periodical > manner... > There's hope thought, as thanks to the tracing > (er.extensions.ERXApplication.traceOpenEditingContextLocks...) I've found a > suspect : > > Outstanding at @Thread[QuartzScheduler_Worker-1,5,main] > java.lang.Exception: Locked > at er.extensions.eof.ERXEC.traceLock(ERXEC.java:541) > at er.extensions.eof.ERXEC.lock(ERXEC.java:511) > at > org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.switchEcForValidationAndPurge(JobValidationReport.java:143) > at > org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.executeJob(JobValidationReport.java:81) > at > org.cocktail.fwkcktlevenement.serveur.quartz.job.impl.JobEvenementImpl.execute(JobEvenementImpl.java:116) > at org.quartz.core.JobRunShell.run(JobRunShell.java:216) > at > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549) > java.lang.Exception: Locked > at er.extensions.eof.ERXEC.traceLock(ERXEC.java:541) > at er.extensions.eof.ERXEC.lock(ERXEC.java:511) > at > er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:243) > at > er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:229) > at > er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectWithGlobalID(ERXEOGlobalIDUtilities.java:209) > at > org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.executeJob(JobValidationReport.java:68) > at > org.cocktail.fwkcktlevenement.serveur.quartz.job.impl.JobEvenementImpl.execute(JobEvenementImpl.java:116) > at org.quartz.core.JobRunShell.run(JobRunShell.java:216) > at > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549) > > ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIds is still locking the ec... > The first stack is normal as I recycle the ec's (create a new one, unlock > the previous, lock the new one) > > Alex > > > 2012/2/13 Kieran Kelleher <[email protected]> > >> How are you dispatching the background threads? Are you >> using ERXExecutorService.executorService() to execute/submit >> Runnable/Callable? >> >> On Feb 12, 2012, at 9:17 PM, Alexis Tual wrote: >> >> Hi Kieran and others, >> related to this topic, I have a deadlock situation which I can't explain. >> I have 2 background threads doing EOF stuff, the first one is doing a lot >> of fetches and can take several minutes. The second one is doing periodical >> short work (a fetch every 30sec). >> For these two threads I use manual locking : locking at the begining of >> the processing and unlocking at the end. I don't share ec's or eo's between >> threads. However they use the same OSC. >> >> Here's the stack trace, any help is welcomed ! >> >> Found one Java-level deadlock: >> ============================= >> "Timer-1": >> waiting for ownable synchronizer 7de742608, (a >> java.util.concurrent.locks.ReentrantLock$NonfairSync), >> which is held by "QuartzScheduler_Worker-1" >> "QuartzScheduler_Worker-1": >> waiting for ownable synchronizer 7de40c8e8, (a >> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync), >> which is held by "Timer-1" >> >> Java stack information for the threads listed above: >> =================================================== >> "Timer-1": >> at sun.misc.Unsafe.park(Native Method) >> - parking to wait for <7de742608> (a >> java.util.concurrent.locks.ReentrantLock$NonfairSync) >> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) >> at >> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) >> at >> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) >> at >> com.webobjects.eocontrol.EOObjectStoreCoordinator.lock(EOObjectStoreCoordinator.java:420) >> at >> com.webobjects.eocontrol.EOEditingContext.lockObjectStore(EOEditingContext.java:4666) >> at er.extensions.eof.ERXEC.lockObjectStore(ERXEC.java:724) >> at >> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4067) >> at >> er.extensions.eof.ERXEC.objectsWithFetchSpecification(ERXEC.java:1215) >> at >> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4444) >> at >> er.extensions.eof.ERXFetchSpecification.fetchObjects(ERXFetchSpecification.java:125) >> at >> org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$DefaultDelegate.fetchEvents(FwkCktlEvenementPrincipal.java:321) >> at >> org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$SchedulingEvenementTask.rescheduleAllEvents(FwkCktlEvenementPrincipal.java:374) >> at >> org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$SchedulingEvenementTask.run(FwkCktlEvenementPrincipal.java:364) >> at java.util.TimerThread.mainLoop(Timer.java:512) >> at java.util.TimerThread.run(Timer.java:462) >> "QuartzScheduler_Worker-1": >> at sun.misc.Unsafe.park(Native Method) >> - parking to wait for <7de40c8e8> (a >> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) >> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) >> at >> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807) >> at >> com.webobjects.eocontrol.EOSharedEditingContext.lock(EOSharedEditingContext.java:736) >> at >> com.webobjects.eocontrol.EOEditingContext.lockObjectStore(EOEditingContext.java:4664) >> at er.extensions.eof.ERXEC.lockObjectStore(ERXEC.java:724) >> at >> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4067) >> at >> er.extensions.eof.ERXEC.objectsWithFetchSpecification(ERXEC.java:1215) >> at >> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4444) >> at >> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:290) >> at >> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:229) >> at >> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectWithGlobalID(ERXEOGlobalIDUtilities.java:209) >> at >> org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.executeJob(JobValidationReport.java:67) >> at >> org.cocktail.fwkcktlevenement.serveur.quartz.job.impl.JobEvenementImpl.execute(JobEvenementImpl.java:116) >> at org.quartz.core.JobRunShell.run(JobRunShell.java:216) >> at >> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549) >> >> Found 1 deadlock. >> >> >> >> >> >> 2012/2/13 Kieran Kelleher <[email protected]> >> >>> Just to follow up..... you can use the task OSC pool as follows: >>> >>> osc = ERXTaskObjectStoreCoordinatorPool.objectStoreCoordinator() >>> >>> You can configure how many OSC's in the pool using this property which >>> defaults to 1 (which is still better than using the default OSC for >>> background EOF tasks): >>> >>> >>> er.extensions.concurrency.ERXTaskObjectStoreCoordinatorPool.maxCoordinators >>> = 1 >>> >>> As a convenience, there is an abstract "task" class,ERXAbstractTask, you >>> can subclass which has a newEditingContext() method that incorporates the >>> OSC pool for you and adjusts EC timestamp to ensure fresh EOs in background >>> tasks. IIRC, this is all explained in the WOWODc presentation too. >>> >>> >>> On Feb 11, 2012, at 6:00 PM, Kieran Kelleher wrote: >>> >>> > Don't worry about the OSC unless you manipulating the OSC directly. >>> > >>> > BTW intensive background EOF activity can impact regular user EOF >>> performance. One approach is to use a dedicated OSC pool for background >>> tasks. Such a pool exists in Wonder. IIRC it is used in the WOWODC example >>> app. >>> > >>> > Regards, Kieran. >>> > (Sent from my iPhone) >>> > >>> > >>> > On Feb 11, 2012, at 7:56 AM, Giles Palmer <[email protected]> >>> wrote: >>> > >>> >> Hi >>> >> >>> >> Just to extend this... when (if at all) should we be locking the >>> EOObjectStoreCoordinator as well, in the context of background threads? >>> >> >>> >> I lock and unlock my ec in a try, catch, finally, is that enough or >>> do I also need to worry about the OSC? >>> >> >>> >> Thanks >>> >> >>> >> >>> >> Giles >>> >> >>> >> >>> >>> Also if you use ERXExecutorService to execute any Plain Old Java >>> Callable (or Runnable), your editing contexts will be auto unlocked by >>> safety-net unlocker at the end of execution if you haven't done so ....... >>> however it is highly recommend that you follow the ec >>> lock/try/finally/unlock pattern in in your Callable.call() or >>> Runnable.run() methods anyway which will be better for long running tasks, >>> recycling ec's if needed, etc. it doesn't hurt. >>> >>> >>> >>> EOEditingContext ec = ERXEC.newEditingContext(); >>> >>> ec.lock(); >>> >>> try { >>> >>> ........ >>> >>> } finally { >>> >>> ec.unlock(); >>> >>> } >>> >>> >>> >>> And yeah, do what Ramsey said .... save yourself time figuring out >>> concurrency management of EC's, etc. by watching the WOWODC presentation >>> from 2011. There is a bunch of stuff related to this in Wonder to make life >>> in the background easy and totally painless for you and that WOWODC session >>> explains it's usage. >>> >>> >>> >>> >>> >>> On Feb 10, 2012, at 11:34 AM, Ramsey Gurley wrote: >>> >>> >>> >>>> If you use ERXRunnable, then you still get autolocking. Just don't >>> try to pass an existing EC or EOs to a background thread. Pass EOs by >>> global id and create the EC on the thread. >>> >>>> >>> >>>> See also Kieran's most recent WOWODC presentation on ERXExecutor >>> stuff and background thread processing. >>> >>>> >>> >>>> Ramsey >>> >>>> >>> >>>> On Feb 10, 2012, at 9:30 AM, Michael Gargano wrote: >>> >>>> >>> >>>>> Hi everyone, >>> >>>>> >>> >>>>> I just want to get clarification on something before I get >>> myself into trouble later. If I have a headless WO app (or potential just >>> a spawned worker thread), and I'm using ERXEC, I need to manually lock and >>> unlock the context, correct? I'm assuming that ERXEC does the >>> autolock/autounlock in the RR loop, which I won't have in this situation. >>> >>>>> >>> >>>>> Thanks. >>> >>>>> -Mike >>> >>>>> >>> >>>>> >>> >>>>> _______________________________________________ >>> >>>>> Do not post admin requests to the list. They will be ignored. >>> >>>>> Webobjects-dev mailing list ([email protected]) >>> >>>>> Help/Unsubscribe/Update your Subscription: >>> >>>>> >>> https://lists.apple.com/mailman/options/webobjects-dev/rgurley%40smarthealth.com >>> >>>>> >>> >>>>> This email sent to [email protected] >>> >>>> >>> >>>> >>> >>>> _______________________________________________ >>> >>>> Do not post admin requests to the list. They will be ignored. >>> >>>> Webobjects-dev mailing list ([email protected]) >>> >>>> Help/Unsubscribe/Update your Subscription: >>> >>>> >>> https://lists.apple.com/mailman/options/webobjects-dev/kelleherk%40gmail.com >>> >>>> >>> >>>> This email sent to [email protected] >>> >>> >>> >>> >>> >>> _______________________________________________ >>> >>> Do not post admin requests to the list. They will be ignored. >>> >>> Webobjects-dev mailing list ([email protected]) >>> >>> Help/Unsubscribe/Update your Subscription: >>> >>> >>> https://lists.apple.com/mailman/options/webobjects-dev/lists%40cedarstone.co.uk >>> >>> >>> >>> This email sent to [email protected] >>> >> >>> >>> >>> _______________________________________________ >>> Do not post admin requests to the list. They will be ignored. >>> Webobjects-dev mailing list ([email protected]) >>> Help/Unsubscribe/Update your Subscription: >>> >>> https://lists.apple.com/mailman/options/webobjects-dev/alexis.tual%40gmail.com >>> >>> This email sent to [email protected] >>> >> >> >> >
_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list ([email protected]) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [email protected]
