Dec, A trigger on the node that gets the eventual document update will fire to start CPF processing. That CPF processing task will then go into the Task Server queue on that node. You can monitor the tasks in process and total number queued by looking at the task server status page of the admin console on 8001.
If you want to increase ingest rates, you may want to step back to be sure the task server configuration can help. Is the machine CPU bound, or I/O bound? (check cpu utilization and iowait in top if it is Linux, e.g.). What processes are running for a long time per the status pages? Are those processes properly coded, and can you profile them in queryConsole? Might there be lock contention issues or DEBUG-level retry messages showing up in the logs? Yours, Damon -- Damon Feldman Principal Consultant, MarkLogic From: [email protected] [mailto:[email protected]] On Behalf Of Declan Newman Sent: Monday, January 07, 2013 11:40 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] How a Task Server works in cluster Hello, I've been trying to find some docs on how the Task Server works in a cluster. We're using v6.0-2. The reason being that we need to speed up content loading times. We're using content pump along with CPF - there are some parameters that we can play with (fastload, threads etc) but wondering if there's anything else we need to do to capitalise on utilising the cluster to its full potential? The CPF pipelines are fairly trivial. The most expensive task is adding docs to multiple collections. Any links to docs, insight and experience greatly appreciated. Thanks, Dec ---------------------------------------------------------------------------- Declan Newman, Development Team Lead, Semantico, Floor 1, 21-23 Dyke Road, Brighton BN1 3FE <mailto:[email protected]> <tel:+44-1273-358247>
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
