Tim, it seems to me that you have two high-level options:
(a) identify and fix the root cause of the contention or deadlock, whichever it turns out to be, or...
(b) serialize the requests in order to keep the contention or deadlock from causing problems.
My own preference would be (a) - it's probably the harder problem, but I dislike leaving known serialization points in an architecture.
However, you seem to be leaning toward (b). If so, you could simply reduce the number of threads on the task server and/or the http server. If either or both servers can only run one thread, then it will be impossible for multiple requests to interfere with each other.
-- Mike On 2010-04-27 21:21, Tim Meagher wrote:
Hi folks, I have a fairly intensive operation that is processing datasets with anywhere between 1 and 2,000 records. Each dataset is inserted into a directory URI that kicks off a pipeline that includes a number of steps. One of those steps includes making a call to a MarkLogic HTTP server (which for now is served from the same server as the request). The request server has 4 CPUs and therefore 4 taskserver threads. As records get loaded, then requests get queued to the HTTP server, but at some point I think I'm running into a deadlock situation in which all 4 task server threads are waiting on a response form the HTTP server. I cannot collocate the two datasets on the same database for a variety of reasons, so I must use the HTTP server. I'm wondering if there is a way to specify the maximum number of threads to use in the pipeline to avoid starving the HTTP service from responding? BTW, the HTTP service will be moving to a 2 CPU (hence 2 task server thread) box. Thank you! Tim Meagher
_______________________________________________ General mailing list General@developer.marklogic.com http://xqzone.com/mailman/listinfo/general
