I had the same problem with large documents causing memory problems. I
solved this problem by introducing a new setting in IndexWriter
setMaxBufferSize(long). Now a merge is either triggered when
bufferedDocs==maxBufferedDocs *or* the size of the bufferedDocs >=
maxBufferSize. I made these changes based on the new merge policy Yonik
mentioned, so if anyone is interested I could open a Jira issue and
submit a patch.
- Michael
Yonik Seeley wrote:
On 11/9/06, Chuck Williams <[EMAIL PROTECTED]> wrote:
Thanks Yonik! Poor wording on my part. I won't vary maxBufferedDocs,
just am making flushRamSegments() public and calling it externally
(properly synchronized), earlier than it would otherwise be called from
ongoing addDocument-driven merging.
Sounds like this should work.
Yep.
For best behavior, you probably want to be using the current
(svn-trunk) version of Lucene with the new merge policy. It ensures
there are mergeFactor segments with size <= maxBufferedDocs before
triggering a merge. This makes for faster indexing in the presence of
deleted docs or partially full segments.
-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search
server
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
