Hi Mike
We got a solution from Norman which improved performance a lot.
The solution we got is to compile the Schematron before in hand and supply it
to Validation which actually reduced the overhead of compiling the Schematron
every time.
Thanks and Regards,
Gnanaprakash Bodireddy
Date: Wed, 28 Nov 2012 08:48:45 -0800
From: Michael Blakeley <m...@blakeley.com>
Subject: Re: [MarkLogic Dev General] Performance Issue with Schematron
Validation
To: general@developer.marklogic.com
Message-ID: <624b2fc1-3f83-45e3-a1d5-a4824a6b7...@blakeley.com>
Content-Type: text/plain; charset=us-ascii
Modern CPUs are fast, but they are still subject to physics. Each CPU core can
only run one thread at a time. The OS will try to switch between threads so
that they get equal time, but more threads means less CPU time per thread.
Sometimes threads are waiting for disk I/O, so I usually configure threads = 2
* cores, or sometimes 4 * cores. Try 4-8 threads.
Given 0.4-s for one task * 50 threads = 20-sec of work to do. Divide that over
2 cores and it will take 10-sec. It could be even longer if context switching
becomes significant or if the larger workload swamps caches or working memory,
increasing disk I/O.
-- Mike
On 28 Nov 2012, at 00:18 , <gnanaprakash.bodire...@cognizant.com> wrote:
> 50 threads could be too many. How many CPU cores does the host have? How much
> RAM?
> Gnana:
> As MarkLogic supports 256 threads I thought 50 threads are good to use. I am
> just performing the validation and if not valid quarantining the files.
> CPU Cores: 2
> RAM: 2GB
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged information.
If you are not the intended recipient(s), please reply to the sender and
destroy all copies of the original message. Any unauthorized review, use,
disclosure, dissemination, forwarding, printing or copying of this email,
and/or any action taken in reliance on the contents of this e-mail is strictly
prohibited and may be unlawful.
_______________________________________________
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
