[
https://issues.apache.org/jira/browse/LUCENE-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13713238#comment-13713238
]
Aditya commented on LUCENE-5121:
--------------------------------
Some additional information as Hoss Man asked on solr IRC after running
CheckIndex with 4.4 and 4.2.1
*With CheckIndex (4.4)*
{code}
[root@solrperfmv002 storage]# java -cp
lucene-core-4.4.0.jar:lucene-codecs-4.4.0.jar
org.apache.lucene.index.CheckIndex ./solrdata/index/
NOTE: testing will be more thorough if you run java with
'-ea:org.apache.lucene...', so assertions are enabled
Opening index @ ./solrdata/index/
Segments file=segments_q numSegments=1 version=4.2.1 format=
userData={commitTimeMSec=1374020371815}
1 of 1: name=_fd docCount=24429799
codec=Lucene42
compound=false
numFiles=16
size (MB)=25,112.699
diagnostics = {timestamp=1374016842452, os=Linux,
os.version=2.6.18-164.11.1.el5, mergeFactor=3, source=merge,
lucene.version=4.2.1-SNAPSHOT 1490837M - greg - 2013-06-07 14:10:44,
os.arch=amd64, mergeMaxNumSegments=1, java.version=1.6.0_45, java.vendor=Sun
Microsystems Inc.}
no deletions
test: open reader.........FAILED
WARNING: fixIndex() would remove reference to this segment; full exception:
org.apache.lucene.index.CorruptIndexException: invalid type: 65,
resource=MMapIndexInput(path="/storage/solrdata/index/_fd_Disk_0.dvdm")
at
org.apache.lucene.codecs.diskdv.DiskDocValuesProducer.readFields(DiskDocValuesProducer.java:159)
at
org.apache.lucene.codecs.diskdv.DiskDocValuesProducer.<init>(DiskDocValuesProducer.java:72)
at
org.apache.lucene.codecs.diskdv.DiskDocValuesFormat.fieldsProducer(DiskDocValuesFormat.java:49)
at
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:213)
at
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:282)
at
org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:134)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:56)
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:543)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:1854)
WARNING: 1 broken segments (containing 24429799 documents) detected
WARNING: would write new segments file, and 24429799 documents would be lost,
if -fix were specified
{code}
*With CheckIndex (4.2.1)*
{code}
[root@solrperfmv002 storage]# java -d64 -Xmx16g -cp
lucene-codecs-4.2.1-SNAPSHOT.jar:lucene-core-4.2.1-SNAPSHOT.jar
org.apache.lucene.index.CheckIndex ./solrdata/index/
NOTE: testing will be more thorough if you run java with
'-ea:org.apache.lucene...', so assertions are enabled
Opening index @ ./solrdata/index/
Segments file=segments_q numSegments=1 version=4.2.1 format=
userData={commitTimeMSec=1374020371815}
1 of 1: name=_fd docCount=24429799
codec=Lucene42
compound=false
numFiles=16
size (MB)=25,112.699
diagnostics = {timestamp=1374016842452, os=Linux,
os.version=2.6.18-164.11.1.el5, mergeFactor=3, source=merge,
lucene.version=4.2.1-SNAPSHOT 1490837M - greg - 2013-06-07 14:10:44,
os.arch=amd64, mergeMaxNumSegments=1, java.version=1.6.0_45, java.vendor=Sun
Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [12367 fields]
test: field norms.........OK [515 fields]
test: terms, freq, prox...OK [139455881 terms; 2613497338 terms/docs pairs;
803255554 tokens]
test: stored fields.......OK [2351712224 total field count; avg 96.264
fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector
fields per doc]
test: docvalues...........OK [0 total doc count; 335 docvalues fields]
No problems were detected with this index.
{code}
> 4.4 RC0 can not read indexes created with 4.2.1 using Disk based docvals
> ------------------------------------------------------------------------
>
> Key: LUCENE-5121
> URL: https://issues.apache.org/jira/browse/LUCENE-5121
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Hoss Man
> Priority: Blocker
> Fix For: 4.4
>
> Attachments: disk_docvals_bug.tgz
>
>
> On the #solr irc channel, user adityab reported a problem trying to upgrade
> an existing solr instance from 4.2.1 to the 4.4 RC0 code. The specific error
> he reported was a "CorruptIndexException: invalid type: 65" from
> DiskDocValuesProducer.readFields using an MMapIndexInput. While waiting for
> more details from adityab on reproducible configs, i attempted to do a
> trivial test using simple solr configs based on the docvals test configs, and
> was able to trigger a slightly different error in which
> DiskDocValuesProducer.readFields caused an EOFException.
> The problem reproduces on both solr init, and when using CheckIndex --
> details to reproduce to follow in an attachment/comment
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
