Hi Folks,
I have a workflow for processing documents of various priorities using the Content Processing Framework. The problem I'm running into is that I might get 10,000 documents that need to be processed at a low priority which get submitted to the task server queue, but then maybe 10 documents come in that are of a higher priority (I'm using these counts for purposes of discussion). What I would like to be able to do is to insert the 10 high priority items in the queue so that they are processed before any outstanding low priority items in the task server queue, in other words I want to interrupt FIFO processing. I'm not concerned about the high priority processing starving low priority processing as the volume of the high priority items is relatively low, but nonetheless an elegant solution would allow me to fine-tune the process so that low priority starvation does not occur. There was some previous discussion about using tail-recursion with xdmp:spawn. That way I would hopefully be able to select the next document to process based on its relative priority. In that case I would probably want to revise the CPF process to merely fill customized priority queues, e.g. high, mid, and low priority queues and to use tail recursion to examine the queues and decide which document to process next. I get the impression that clustering could be a useful way to create task servers that are dedicated to higher and lower priority processing for the needs of an entire organization, but it seems to me that allowing for pre-emption in a given task server could be a really useful feature. Perhaps there are some existing features that are provided to deal with just this problem. There are times when I've submitted more docs to be processed by the task server and would like to be able to dequeue them - I suppose that a prioritization solution would also allow for dequeuing tasks. Thanks ahead of time for any help! Tim Meagher
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
