Changing APIs may not get you anywhere: I can match your ingestion rate
on my laptop, using either API. Start by analyzing the performance of
every subsystem, so that you know where the bottleneck is. Measure the
utilization of the client CPU, server CPU, disk, and network.
If the subsystems seem to be largely idle, I'd ask if you are using
enough client threads? With a multithreaded loader, such as RecordLoader
or XQSync, I'd start with THREADS = 2 * ServerCpuCores, as a rule of thumb.
If the network is your bottleneck, and you can't improve its capacity,
then xdmp:document-load() could be somewhat more efficient than XCC,
since it would avoid some network traffic. But xdmp:document-load
requires you to make the content available on the server filesystem,
which may not be practical. But we don't know if the network is your
bottleneck or not, so start by measuring subsystem utilization.
thanks,
-- Mike
Jain, Kokil wrote:
Hello,
From a efficiency standpoint, is it better to insert content using
session.insertContent(content) or via Xquery using xdmp:document-load?
We have a high volume system with 10k docs per min. Is there a faster
way of uploading or inserting documents as we receive them?
Thanks for the help,
Kokil Jain
This communication, along with any attachments, is covered by federal and state law governing electronic communications and may contain company proprietary and legally privileged information.
If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, use or copying of this message is strictly prohibited.
If you have received this in error, please reply immediately to the sender and delete this message. Thank you.
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
