[
https://issues.apache.org/jira/browse/LUCENE-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983798#action_12983798
]
Michael McCandless commented on LUCENE-2871:
--------------------------------------------
I attempted to benchmark this patch, testing indexing time for Wikipedia docs,
but I hit this exception:
{noformat}
Exception in thread "Lucene Merge Thread #49"
org.apache.lucene.index.MergePolicy$MergeException:
java.nio.BufferOverflowException
at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:507)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:472)
Caused by: java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:501)
at java.nio.DirectByteBuffer.putLong(DirectByteBuffer.java:745)
at
org.apache.lucene.store.FSDirectory$FSChannelIndexOutput.writeLong(FSDirectory.java:706)
at
org.apache.lucene.index.codecs.PrefixCodedTermsWriter.writeTrailer(PrefixCodedTermsWriter.java:146)
at
org.apache.lucene.index.codecs.PrefixCodedTermsWriter.close(PrefixCodedTermsWriter.java:129)
at
org.apache.lucene.index.PerFieldCodecWrapper$FieldsWriter.close(PerFieldCodecWrapper.java:80)
at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:559)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:120)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3084)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2768)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:379)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:447)
{noformat}
This was after adding ~6 M docs... the 100K doc index built just fine. Shay
any ideas?
> Use FileChannel in FSDirectory
> ------------------------------
>
> Key: LUCENE-2871
> URL: https://issues.apache.org/jira/browse/LUCENE-2871
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Store
> Reporter: Shay Banon
> Attachments: LUCENE-2871.patch
>
>
> Explore using FileChannel in FSDirectory to see if it improves write
> operations performance
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
