[
https://issues.apache.org/jira/browse/LUCENE-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated LUCENE-5121:
-----------------------------
Attachment: disk_docvals_bug.tgz
the attached disk_docvals_bug.tgz contains a simple solr home dir containing
some very minimal config files and a sample index demonstrating this problem.
the same index was built using solr 4.2.1 by indexing the included
inputdata.xml file using post.jar and confirming that a basic search for "*:*"
returned the indexed document.
after shutting down solr 4.2.1, the 4.2.1 version of CheckIndex was run against
hte index directory, and no errors were reported...
{noformat}
hossman@frisbee:~/lucene/lucene-4.2.1_tag$ java -ea:org.apache.lucene... -cp
'./lucene/build/codecs/*:lucene/build/core/*'
org.apache.lucene.index.CheckIndex
/home/hossman/tmp/disk_docvals_bug/collection1/data/index/
Opening index @ /home/hossman/tmp/disk_docvals_bug/collection1/data/index/
Segments file=segments_3 numSegments=1 version=4.2.1 format=
userData={commitTimeMSec=1374187354148}
1 of 1: name=_0 docCount=1
codec=Lucene42
compound=false
numFiles=16
size (MB)=0.003
diagnostics = {timestamp=1374187354189, os=Linux,
os.version=3.2.0-50-generic, source=flush, lucene.version=4.2.1-SNAPSHOT
1487037 - hossman - 2013-05-28 11:55:58, os.arch=amd64, java.version=1.7.0_25,
java.vendor=Oracle Corporation}
no deletions
test: open reader.........OK
test: fields..............OK [9 fields]
test: field norms.........OK [5 fields]
test: terms, freq, prox...OK [11 terms; 11 terms/docs pairs; 0 tokens]
test: stored fields.......OK [5 total field count; avg 5 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector
fields per doc]
test: docvalues...........OK [0 total doc count; 5 docvalues fields]
No problems were detected with this index.
{noformat}
At that point, I attempted to run the 4.4.0 RC0 version of CheckIndex against
the same inex directory, and got an EOF error...
{noformat}
hossman@frisbee:~/tmp/4.4/RC0-rev1503555/hoss/solr-4.4.0$ java
-ea:org.apache.lucene... -cp './lucene/build/codecs/*:lucene/build/core/*'
org.apache.lucene.index.CheckIndex
/home/hossman/tmp/disk_docvals_bug/collection1/data/index/
Opening index @ /home/hossman/tmp/disk_docvals_bug/collection1/data/index/
Segments file=segments_3 numSegments=1 version=4.2.1 format=
userData={commitTimeMSec=1374187354148}
1 of 1: name=_0 docCount=1
codec=Lucene42
compound=false
numFiles=16
size (MB)=0.003
diagnostics = {timestamp=1374187354189, os=Linux,
os.version=3.2.0-50-generic, source=flush, lucene.version=4.2.1-SNAPSHOT
1487037 - hossman - 2013-05-28 11:55:58, os.arch=amd64, java.version=1.7.0_25,
java.vendor=Oracle Corporation}
no deletions
test: open reader.........FAILED
WARNING: fixIndex() would remove reference to this segment; full exception:
java.io.EOFException: read past EOF:
MMapIndexInput(path="/home/hossman/tmp/disk_docvals_bug/collection1/data/index/_0_Disk_0.dvdm")
at
org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:78)
at
org.apache.lucene.codecs.diskdv.DiskDocValuesProducer.readFields(DiskDocValuesProducer.java:105)
at
org.apache.lucene.codecs.diskdv.DiskDocValuesProducer.<init>(DiskDocValuesProducer.java:72)
at
org.apache.lucene.codecs.diskdv.DiskDocValuesFormat.fieldsProducer(DiskDocValuesFormat.java:49)
at
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:213)
at
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:282)
at
org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:134)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:56)
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:543)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:1854)
WARNING: 1 broken segments (containing 1 documents) detected
WARNING: would write new segments file, and 1 documents would be lost, if -fix
were specified
{noformat}
The same error occurs if you attempt to start up Solr 4.4.0 RC0 pointed at this
solr home dir....
{noformat}
org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:835)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:629)
at
org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:622)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:657)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:364)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:356)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1522)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1634)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:810)
... 13 more
Caused by: org.apache.solr.common.SolrException: Error opening Reader
at
org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:177)
at
org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:188)
at
org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:184)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1497)
... 15 more
Caused by: java.io.EOFException: read past EOF:
MMapIndexInput(path="/home/hossman/tmp/disk_docvals_bug/collection1/data/index/_0_Disk_0.dvdm")
at
org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:78)
at
org.apache.lucene.codecs.diskdv.DiskDocValuesProducer.readFields(DiskDocValuesProducer.java:105)
at
org.apache.lucene.codecs.diskdv.DiskDocValuesProducer.<init>(DiskDocValuesProducer.java:72)
at
org.apache.lucene.codecs.diskdv.DiskDocValuesFormat.fieldsProducer(DiskDocValuesFormat.java:49)
at
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:213)
at
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:282)
at
org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:134)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:56)
at
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:812)
at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88)
at
org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
at
org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:174)
... 18 more
{noformat}
> 4.4 RC0 can not read indexes created with 4.2.1 using Disk based docvals
> ------------------------------------------------------------------------
>
> Key: LUCENE-5121
> URL: https://issues.apache.org/jira/browse/LUCENE-5121
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Hoss Man
> Priority: Blocker
> Fix For: 4.4
>
> Attachments: disk_docvals_bug.tgz
>
>
> On the #solr irc channel, user adityab reported a problem trying to upgrade
> an existing solr instance from 4.2.1 to the 4.4 RC0 code. The specific error
> he reported was a "CorruptIndexException: invalid type: 65" from
> DiskDocValuesProducer.readFields using an MMapIndexInput. While waiting for
> more details from adityab on reproducible configs, i attempted to do a
> trivial test using simple solr configs based on the docvals test configs, and
> was able to trigger a slightly different error in which
> DiskDocValuesProducer.readFields caused an EOFException.
> The problem reproduces on both solr init, and when using CheckIndex --
> details to reproduce to follow in an attachment/comment
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
