Hi Danny, Thanks for the thoughtful responses. Florent was inquiring about CPF on my behalf (we're working together here in London).
We noticed this condition after running several thousand updates overnight to test our new CPF approach to asynchronously changing collection membership on hierarchies of related documents. A small document is created that contains information about which hierarchy to modify. The creation of that small temp document fires CPF. Most updates worked as expected but a few appeared to just be ignored. After some investigation, we found evidence that some CPF actions were dying because of EX-TIME. Taking a closer look, we discovered that these appeared to be attempting to change the collection membership of several thousand documents at once. Upon further investigation, we saw that these unprocessed documents (the ones that are the target of a CPF action) did not have any CPF state property, but they did have a "created" processing-status. This indicated that CPF noticed creation of the document, but the document apparently never transitioned into "initial" state. So it would appear that if a CPF action dies with EX-TIME that the CPF framework code does not have a chance to catch that exception and send the document to an error state. As you point out, on a restart CPF will try to transition to initial state again and ultimately fail in the same way. We're running 4.1-5 on the dev systems, but I believe the production system where this was observed is still running 4.1-3. So CPF might be able to handle the condition better in the future. We've implemented an improved solution with an action module that works through the list of documents to be updated and does them in small batches, looping until all are processed. On Mar 24, 2010, at 9:58 PM, Danny Sokolsky wrote: > And one other thing I recently found out, Florent: > > As of 4.1-4, XDMP-EXTIME is a caught exception. So if you are running 4.1-4 > or later, it should just work. > > -Danny > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Geert Josten > Sent: Wednesday, March 24, 2010 2:40 PM > To: General Mark Logic Developer Discussion > Subject: RE: [MarkLogic Dev General] Timeout in a CPF pipeline? > > Hi Florent, > > If you wrap your code in an xdmp:eval, you can set a time-limit that is > smaller than the default task limit. That should result the doc being given > the error state. You could also wrap the eval in a try-catch to do some > additional work before rethrowing the error.. > > Kind regards, > Geert > >> > > > drs. G.P.H. (Geert) Josten > Consultant > > > Daidalos BV > Hoekeindsehof 1-4 > 2665 JZ Bleiswijk > > T +31 (0)10 850 1200 > F +31 (0)10 850 1199 > > mailto:[email protected] > http://www.daidalos.nl/ > > KvK 27164984 > > P Please consider the environment before printing this mail. > De informatie - verzonden in of met dit e-mailbericht - is afkomstig van > Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit > bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit > bericht kunnen geen rechten worden ontleend. > >> From: [email protected] >> [mailto:[email protected]] On Behalf Of >> Florent Georges >> Sent: woensdag 24 maart 2010 22:34 >> To: General Mark Logic Developer Discussion >> Subject: Re: [MarkLogic Dev General] Timeout in a CPF pipeline? >> >> Danny Sokolsky wrote: >> >> Thanks for your fast response! >> >>> Ah, I see the problem now. The timeout exception is not caught, so >>> the document never enters the error state, nor does it >> advance to the >>> next state (because it is not successful). >> >> Exactly... >> >>> This is a tricky situation. I can think of a few things to >> counteract >>> it. >> >> Yes, this is quite a corner case, but we did run into such >> a problem a few days ago and we found it only by chance (if I >> understood correctly, I don't have the full story). >> >> A nice solution would have been to be able to, say, send an >> email or any kind of alert for manual inspection, i.e to be >> notified in some way. Maybe a batch on the error log could >> achieve that. >> >> At least that confirm what I thought. Thanks again, >> >> -- >> Florent Georges >> http://www.fgeorges.org/ >> >> >> >> >> _______________________________________________ >> General mailing list >> [email protected] >> http://xqzone.com/mailman/listinfo/general >> > _______________________________________________ > General mailing list > [email protected] > http://xqzone.com/mailman/listinfo/general > _______________________________________________ > General mailing list > [email protected] > http://xqzone.com/mailman/listinfo/general --- Ron Hitchens {mailto:[email protected]} Ronsoft Technologies (650) 766-2355 (Home Office) http://www.ronsoft.com (707) 924-3878 (fax) Bit Twiddling At Its Finest "No amount of belief establishes any fact." -Unknown _______________________________________________ General mailing list [email protected] http://xqzone.com/mailman/listinfo/general
