[
https://issues.apache.org/jira/browse/LUCENE-5934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128412#comment-14128412
]
Uwe Schindler commented on LUCENE-5934:
---------------------------------------
>From user list, my final analysis:
{quote}
{noformat}
Hi,
we looked into earlier releases:
The index version number of 4.0-ALPHA was "4.0"
The index version number of 4.0-BETA was "4.0.0.1"
The index version number of 4.0 final was "4.0.0.2"
Ian's index is there fore a real official 4.0 index.
Unfortunately the version comparison logic in Lucene 4.10 is wrong, as it has a
special case for ALPHA and BETA indexes, which does not fit reality. Also, the
constants are wrong:
/**
* Match settings and bugs in Lucene's 4.0.0-ALPHA release.
* @deprecated (4.1) Use latest
*/
@Deprecated
public static final Version LUCENE_4_0_0_ALPHA = new Version(4, 0, 0, 1);
/**
* Match settings and bugs in Lucene's 4.0.0-BETA release.
* @deprecated (4.1) Use latest
*/
@Deprecated
public static final Version LUCENE_4_0_0_BETA = new Version(4, 0, 0, 2);
/**
* Match settings and bugs in Lucene's 4.0.0 release.
* @deprecated (4.1) Use latest
*/
@Deprecated
public static final Version LUCENE_4_0_0 = new Version(4, 0, 0);
Because of this and the special case, 4.0.0.2 orders before "4.0.0" (see
encodedVersionNumber). This causes IndexReader/IndexWriter to think it was
created in 3.x.
TestBackwards compatibility did not find that bug, because the backwards index
in the tests directory was created with the Alpha version :(
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [email protected]
> -----Original Message-----
> From: Uwe Schindler [mailto:[email protected]]
> Sent: Wednesday, September 10, 2014 1:42 PM
> To: [email protected]
> Subject: RE: 4.10.0: java.lang.IllegalStateException: cannot write 3x
> SegmentInfo unless codec is Lucene3x (got: Lucene40)
>
> Hi Ian,
>
> this index was created with the BETA version of Lucene 4.0:
>
> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> 1 of 1: name=_0 docCount=15730
>
> "4.0.0.2" was the index version number of Lucene 4.0-BETA. This is not
> a supported version and may not open correctly. In Lucene 4.10 we
> changed version handling and parsing version numbers a bit, so this
> may be the cause for the error.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [email protected]
>
>
> > -----Original Message-----
> > From: Ian Lea [mailto:[email protected]]
> > Sent: Wednesday, September 10, 2014 1:01 PM
> > To: [email protected]
> > Subject: 4.10.0: java.lang.IllegalStateException: cannot write 3x
> > SegmentInfo unless codec is Lucene3x (got: Lucene40)
> >
> > Hi
> >
> >
> > On running a quick test after a handful of minor code changes to
> > deal with
> > 4.10 deprecations, a program that updates an existing index failed
> > with
> >
> > Exception in thread "main" java.lang.IllegalStateException: cannot
> > write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40) at
> >
> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607
> > )
> >
> > and along the way did something to the index to make it unusable.
> >
> > Digging a bit deeper and working on a different old test index that
> > was lying around, and taking a backup first this time, this is reproducible.
> >
> > The working index:
> >
> > total 1036
> > -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> > -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> > -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> > -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> > -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> > -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> > -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
> > -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> > -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
> >
> > and output from 4.10 CheckIndex
> >
> > Opening index @ index/
> >
> > Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
> > 1 of 1: name=_0 docCount=15730
> > version=4.0.0.2
> > codec=Lucene40
> > compound=false
> > numFiles=7
> > size (MB)=0.987
> > diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
> > source=flush,
> > lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12,
> > os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [13 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
> > pairs; 139382 tokens]
> > test: stored fields.......OK [15730 total field count; avg 1 fields per
> > doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > No problems were detected with this index.
> >
> >
> > Now run this little program
> >
> > public static void main(final String[] _args) throws Exception {
> > File index = new File(_args[0]); IndexWriterConfig iwcfg = new
> > IndexWriterConfig(Version.LUCENE_4_10_0,
> > new StandardAnalyzer());
> >
> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
> > Directory d = FSDirectory.open(index, new
> > SimpleFSLockFactory(index)); IndexWriter iw = new IndexWriter(d,
> > iwcfg); Document doc1 = new Document(); doc1.add(new
> > StringField("type", "test", Field.Store.NO)); iw.addDocument(doc1);
> > iw.close();
> > }
> >
> > and it fails with
> >
> > Exception in thread "main" java.lang.IllegalStateException: cannot
> > write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40) at
> >
> org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607
> > ) at
> > org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
> > at
> >
> org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:
> > 1017)
> > at
> > org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:454
> > 9)
> > at
> >
> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.
> > j
> > ava:3062)
> > at
> > org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:
> > 31
> > 69
> > )
> > at
> > org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
> > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
> > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
> > at t.main(t.java:25)
> >
> > and when run CheckIndex again get
> >
> >
> > Opening index @ index/
> >
> > ERROR: could not read any segments file in directory
> > java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si at
> > sun.nio.fs.UnixException.translateToIOException(UnixException.java:8
> > 6)
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102
> > )
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107
> > )
> > at
> >
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.
> > java:176)
> > at java.nio.channels.FileChannel.open(FileChannel.java:287)
> > at java.nio.channels.FileChannel.open(FileChannel.java:334)
> > at
> >
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:19
> > 6)
> > at
> >
> org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Luce
> > ne40SegmentInfoReader.java:52)
> > at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
> > at
> > org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
> > at
> >
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> > s.java:913)
> > at
> >
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfo
> > s.java:759)
> > at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
> > at
> > org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
> > at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
> >
> > which is true
> >
> > total 1032
> > -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
> > -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
> > -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
> > -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
> > -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
> > -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
> > -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
> > -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
> >
> >
> > I don't recall the origins of this index but it may well have been
> > created in the distant past and been upgraded, explicitly or
> > automatically,
> along the way.
> > Although evidently not for a while.
> >
> >
> > Running the same test with lucene 4.9.0 and minimal mods to the
> > program runs to successful completion. Here's the CheckIndex output:
> >
> >
> > Opening index @ index-4.9.updated/
> >
> > Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9]
> format=
> > 1 of 2: name=_0 docCount=15730
> > codec=Lucene40
> > compound=false
> > numFiles=7
> > size (MB)=0.987
> > diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
> > source=flush,
> > lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 02:58:12,
> > os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [13 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
> > pairs; 139382 tokens]
> > test: stored fields.......OK [15730 total field count; avg 1 fields per
> > doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > 2 of 2: name=_1 docCount=1
> > codec=Lucene49
> > compound=true
> > numFiles=3
> > size (MB)=0.001
> > diagnostics = {timestamp=1410281698360, os=Linux,
> > os.version=3.1.0-1.2- desktop, source=flush, lucene.version=4.9.0
> > 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64,
> > java.version=1.7.0_10, java.vendor=Oracle Corporation}
> > no deletions
> > test: open reader.........OK
> > test: check integrity.....OK
> > test: check live docs.....OK
> > test: fields..............OK [1 fields]
> > test: field norms.........OK [0 fields]
> > test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
> > test: stored fields.......OK [0 total field count; avg 0 fields per doc]
> > test: term vectors........OK [0 total vector count; avg 0
> > term/freq vector fields per doc]
> > test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
> > NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
> >
> > No problems were detected with this index.
> >
> >
> >
> >
> > --
> > Ian.
{noformat}
{quote}
> 4.10 broke backwards compatibility for 4.0 indexes
> --------------------------------------------------
>
> Key: LUCENE-5934
> URL: https://issues.apache.org/jira/browse/LUCENE-5934
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
>
> As reported by Ian on the user list:
> Its trying to treat them as 3.x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
