Hi Tim, The app servers should have threads of their own, but if they again depend on background processing, you might be in trouble indeed. But you could also easily be blocked by write-locks, not much you can do to prevent that either, in your situation I'm afraid..
Unfortunately, CPF, triggers, scheduled tasks and spawned processes all rely on the task server queue which is managed fully automatically. Easiest way out would be to use two hosts (or VM's perhaps), and make sure you point to the other one from your pipelines. Kind regards, Geert > drs. G.P.H. (Geert) Josten Consultant Daidalos BV Hoekeindsehof 1-4 2665 JZ Bleiswijk T +31 (0)10 850 1200 F +31 (0)10 850 1199 mailto:geert.jos...@daidalos.nl http://www.daidalos.nl/ KvK 27164984 P Please consider the environment before printing this mail. De informatie - verzonden in of met dit e-mailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend. > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of > Tim Meagher > Sent: woensdag 28 april 2010 6:22 > To: 'General Mark Logic Developer Discussion' > Subject: [MarkLogic Dev General] Pipeline processing > saturating the ML server > > Hi folks, > > > > I have a fairly intensive operation that is processing > datasets with anywhere between 1 and 2,000 records. Each > dataset is inserted into a directory URI that kicks off a > pipeline that includes a number of steps. One of those steps > includes making a call to a MarkLogic HTTP server (which for > now is served from the same server as the request). > > > > The request server has 4 CPUs and therefore 4 taskserver > threads. As records get loaded, then requests get queued to > the HTTP server, but at some point I think I'm running into a > deadlock situation in which all 4 task server threads are > waiting on a response form the HTTP server. I cannot > collocate the two datasets on the same database for a variety > of reasons, so I must use the HTTP server. > > > > I'm wondering if there is a way to specify the maximum number > of threads to use in the pipeline to avoid starving the HTTP > service from responding? BTW, the HTTP service will be > moving to a 2 CPU (hence 2 task server thread) box. > > > > Thank you! > > > > Tim Meagher > > > > _______________________________________________ General mailing list General@developer.marklogic.com http://xqzone.com/mailman/listinfo/general
