Re: Merge performance

2007-04-19 Thread Michael D. Curtin
david m wrote: A couple of reasons that lead to the merge approach: - Source documents are written to archive media and retrieval is relatively slow. Add to that our processing pipeline (including text extraction)... Retrieving and merging minis is faster than re-processing and re-indexing f

Re: Merge performance

2007-04-18 Thread david m
Michael, Our application includes indexing and archiving documents to meet compliance requirements. A couple of reasons that lead to the merge approach: - Source documents are written to archive media and retrieval is relatively slow. Add to that our processing pipeline (including text extrac

Re: Merge performance

2007-04-18 Thread david m
so that we'll be able to test multiple merge policies. -Original Message- From: Erick Erickson [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 18, 2007 9:17 AM To: java-user@lucene.apache.org Subject: Re: Merge performance This *may* be relevant, I haven't needed to investigate

Re: Merge performance

2007-04-18 Thread Michael D. Curtin
d m wrote: I'd like to share index merge performance data and have a couple of questions about it... We (AXS-One, www.axsone.com) build one "master" index per day. For backup and recovery purposes, we also build many individual "mini" indexes from the docs added to the master index. Should one

RE: Merge performance

2007-04-18 Thread Steven Parkes
007 9:17 AM To: java-user@lucene.apache.org Subject: Re: Merge performance This *may* be relevant, I haven't needed to investigate it yet... http://issues.apache.org/jira/browse/LUCENE-845 Also, see the thread titled "MergeFactor and MaxBufferedDocs value should ...?" for an intere

Re: Merge performance

2007-04-18 Thread Erick Erickson
This *may* be relevant, I haven't needed to investigate it yet... http://issues.apache.org/jira/browse/LUCENE-845 Also, see the thread titled "MergeFactor and MaxBufferedDocs value should ...?" for an interesting discussion of how to optimize indexing, although I'm not sure the notion of using I