I am working on a bulk document copy right now - I will let you know
if it improves things much.
I doubt it, because I already configure the streams to use fairly
large input and output buffers during a merge - but the memory index
merge may see additional benefits due to lower CPU calls.
On Nov 2, 2007, at 3:39 PM, robert engels (JIRA) wrote:
[ https://issues.apache.org/jira/browse/LUCENE-1043?
page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
tabpanel#action_12539696 ]
robert engels commented on LUCENE-1043:
---------------------------------------
When bulk copying the documents, I think you need to:
read array of long from index (8 * (ndocs+1)) in long[ndocs+1]
offsets;
calculate length = offset[ndocs]-offset[0];
read bytes of length from document file
startoffset = current output document stream position
write bytes to output document
modify offset[] adding startoffset-offset[0] to each entry
write offset[] in bulk to index output
Speedup merging of stored fields when field mapping "matches"
-------------------------------------------------------------
Key: LUCENE-1043
URL: https://issues.apache.org/jira/browse/
LUCENE-1043
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Affects Versions: 2.2
Reporter: Michael McCandless
Assignee: Michael McCandless
Priority: Minor
Fix For: 2.3
Attachments: LUCENE-1043.patch
Robert Engels suggested the following idea, here:
http://www.gossamer-threads.com/lists/lucene/java-dev/54217
When merging in the stored fields from a segment, if the field
name ->
number mapping is identical then we can simply bulk copy the entire
entry for the document rather than re-interpreting and then re-
writing
the actual stored fields.
I've pulled the code from the above thread and got it working on the
current trunk.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
