[
https://issues.apache.org/jira/browse/LUCENE-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732915#comment-16732915
]
Dawid Weiss commented on LUCENE-8628:
-------------------------------------
A duplicate of LUCENE-8624 and LUCENE-8625, already fixed?
> ByteBuffersDirectory merge exception
> ------------------------------------
>
> Key: LUCENE-8628
> URL: https://issues.apache.org/jira/browse/LUCENE-8628
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
> Affects Versions: 7.5
> Reporter: Toke Eskildsen
> Priority: Major
>
> An exception during merge for {{ByteBufferDirectory}} was [reported on the
> lucene-dev
> mailinglist|https://mail-archives.apache.org/mod_mbox/lucene-dev/201812.mbox/raw/%3CCAL%3D08JvGioNvfy8xKbb7%2B999YnNnU4iQv6F6ktRkudrkr-6jGg%40mail.gmail.com%3E]:
>
> {{Exception in thread "Lucene Merge Thread #879"
> org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.store.AlreadyClosedException: refusing to delete any files:
> this IndexWriter hit an unrecoverable exception}}{{ at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)}}{{
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:685)}}{{Caused
> by: org.apache.lucene.store.AlreadyClosedException: refusing to delete any
> files: this IndexWriter hit an unrecoverable exception}}{{ at
> org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)}}{{
> at
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)}}{{
> at
> org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:664)}}{{
> at
> org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:5024)}}{{
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4539)}}{{
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4075)}}{{
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)}}{{
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:663)}}{{Caused
> by: java.lang.IllegalArgumentException: cannot write negative vLong (got:
> -4294878395)}}{{ at
> org.apache.lucene.store.DataOutput.writeVLong(DataOutput.java:225)}}{{
> at
> org.apache.lucene.codecs.lucene50.Lucene50SkipWriter.writeSkipData(Lucene50SkipWriter.java:180)}}{{
> at
> org.apache.lucene.codecs.MultiLevelSkipListWriter.bufferSkip(MultiLevelSkipListWriter.java:143)}}{{
> at
> org.apache.lucene.codecs.lucene50.Lucene50SkipWriter.bufferSkip(Lucene50SkipWriter.java:162)}}{{
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsWriter.startDoc(Lucene50PostingsWriter.java:228)}}{{
> at
> org.apache.lucene.codecs.PushPostingsWriterBase.writeTerm(PushPostingsWriterBase.java:148)}}{{
> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.write(BlockTreeTermsWriter.java:865)}}{{
> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter.write(BlockTreeTermsWriter.java:344)}}{{
> at
> org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)}}{{
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:169)}}{{
> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:244)}}{{
> at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:139)}}{{
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4453)}}{{
> ... 3 more}}
> Following the trace shows no apparent problems with the
> {{Lucene50SkipWripter}}. However, it uses {{getFilePointer}} to resolve the
> skip lengths, which for {{ByteBuffersDirectory}} ends in
> {{ByteBuffersDataOutput.size()}} with the code
> {{ public long size() {}}
> {{ long size = 0;}}
> {{ int blockCount = blocks.size();}}
> {{ if (blockCount >= 1) {}}
> {{ int fullBlockSize = (blockCount - 1) * blockSize();}}
> {{ int lastBlockSize = blocks.getLast().position();}}
> {{ size = fullBlockSize + lastBlockSize;}}
> {{ }}}
> {{ return size;}}
> {{ }}}
> One problem here is that
> {{int fullBlockSize = (blockCount - 1) * blockSize();}}
> can overflow. This should be changed to something like
> {{long fullBlockSize = (blockCount - 1) * (long)blockSize();}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
