Danny, thanks very much for the info. My pipeline calls out to a 3rd-party text processing tool. When I submit each document to ML as a separate load, the 3rd-party tool marks up the text correctly, but when I submit multiple documents in one load, it only marks up the first one. I'm thinking maybe the tool isn't able to handle multithreaded requests, though I haven't been able to confirm that yet. In light of that, is there a config way to get CPF / the task server to stop multithreading and process each document sequentially? If not, maybe I'll have to write something to submit each document separately.
Thanks again, John On 6/7/2011 7:46 PM, Danny Sokolsky wrote: > Hi John, > > CPF spawns tasks onto the task server, and the task server processes with > multiple threads. So that is normal to have interleaved log messages. > > So I guess the question is what is not working correctly when you have > multiple documents? > > 2 tips for debugging CPF apps are: > > 1) use the CPF trace event (which throws many log messages into the log) and > 2) look at the properties of documents after processing to see if any errors > occurred (any errors are logged into the properties). > > -Danny > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of John Munson > Sent: Tuesday, June 07, 2011 4:15 PM > To: [email protected] > Subject: [MarkLogic Dev General] CPF and multithreading > > Hi -- > > I'm very new to MarkLogic and to XQuery. Didn't see anything about this > in the archives... > > I've set up a Content Processing Framework pipeline using one of the > samples supplied in the installation. It works fine when I load a > single document, but not when I load more than one. I used xdmp:log() > to trace what's going on via the error log, and I was surprised to find > that my debug output was interleaved between the two documents I was > loading. This suggests that CPF handed the documents off to my > processing in separate threads. Does anyone know if that is how CPF is > designed to work? If so, then maybe the bad output I'm getting is due > to some other problem, but I'm wondering if it might be due to a > multithreading problem, given that it shows up only with multiple documents. > > Thanks, > John > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
