[
https://issues.apache.org/jira/browse/LUCENE-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993133#comment-13993133
]
Uwe Schindler commented on LUCENE-5658:
---------------------------------------
bq. This can only happen if the underlying JVM has a bug (J9???) or the
underlying filesystem reports another file length after opening than it really
is (maybe file was truncated).
The first is more obviously the problem. If the file length() reported by the
OS is incorrect, the original mmap should have failed already.
openFullSlice/clone just clones the ByteBuffers. And this cannot change the
length, unless the JVM corrumpts the clones.
> IllegalArgumentException from ByteBufferIndexInput.buildSlice
> -------------------------------------------------------------
>
> Key: LUCENE-5658
> URL: https://issues.apache.org/jira/browse/LUCENE-5658
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
> Affects Versions: 4.8
> Reporter: Shai Erera
>
> I've received an email with the following stacktrace:
> {noformat}
> Exception in thread "Lucene Merge Thread #73"
> org.apache.lucene.index.MergePolicy$MergeException:
> java.lang.IllegalArgumentException
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:545)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:278)
> at
> org.apache.lucene.store.ByteBufferIndexInput.buildSlice(ByteBufferIndexInput.java:259)
> at
> org.apache.lucene.store.ByteBufferIndexInput.buildSlice(ByteBufferIndexInput.java:230)
> at
> org.apache.lucene.store.ByteBufferIndexInput.clone(ByteBufferIndexInput.java:187)
> at
> org.apache.lucene.store.MMapDirectory$1.openFullSlice(MMapDirectory.java:211)
> at
> org.apache.lucene.store.CompoundFileDirectory.readEntries(CompoundFileDirectory.java:138)
> at
> org.apache.lucene.store.CompoundFileDirectory.<init>(CompoundFileDirectory.java:105)
> at
> org.apache.lucene.index.SegmentReader.readFieldInfos(SegmentReader.java:209)
> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:99)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:142)
> at
> org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:624)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3728)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
> {noformat}
> According to the email, it happens randomly while indexing Wikipedia, on
> 4.8.0. As far as I understood, the indexer creates 4 indexes in parallel, by
> a total of 48 threads. Each index is created in a separate directory, and
> there's no sharing of MP or MS instances between the writers (in fact,
> default settings are used). This could explain the {{Lucene Merge Thread
> #73}}. The indexing process ends w/ a {{forceMerge(1)}}. If that call is
> omitted, the exception doesn't reproduce. Also, since it doesn't happen
> always, there's no simple testcase which reproduces.
> I've asked the reporter to add more info to the issue, but opening the issue
> now so we could check and hopefully fix before 4.8.1.
> I checked the stacktrace against trunk, but not all the lines align (e.g.
> {{at
> org.apache.lucene.store.MMapDirectory$1.openFullSlice(MMapDirectory.java:211)}}
> is only in 4.8), but I haven't dug deeper yet...
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
