Strictly speaking, we could have opened an older index using Lucene 8 (say one that was created using Lucene 7, or 6) that would no longer be valid in Lucene 9, at least according to the policy? I agree we should try to fix this, just want to clarify the policy
On Tue, Dec 14, 2021 at 8:54 AM Adrien Grand <jpou...@gmail.com> wrote: > > This looks related to the new changes around schema validation. Lucene > now requires a field to either be absent from a document or be indexed > with the exact same options (index options, points dimensions, norms, > doc values type, etc.) as already indexed documents that also have > this field. > > However it's a bug that Lucene fails to open an index that was legal > in Lucene 8. Can you file a JIRA issue? > > On Mon, Dec 13, 2021 at 4:23 PM Ian Lea <ian....@gmail.com> wrote: > > > > Hi > > > > > > We have a long-standing index with some mandatory fields and some optional > > fields that has been through multiple lucene upgrades without a full > > rebuild and on testing out an upgrade from version 8.11.0 to 9.0.0, when > > open an IndexWriter we are hitting the exception > > > > Exception in thread "main" java.lang.IllegalArgumentException: cannot > > change field "language" from index options=NONE to inconsistent index > > options=DOCS > > at > > org.apache.lucene.index.FieldInfo.verifySameIndexOptions(FieldInfo.java:245) > > at > > org.apache.lucene.index.FieldInfos$FieldNumbers.verifySameSchema(FieldInfos.java:421) > > at > > org.apache.lucene.index.FieldInfos$FieldNumbers.addOrGet(FieldInfos.java:357) > > at > > org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:1263) > > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1116) > > > > Where language is one of our optional fields. > > > > Presumably this is at least somewhat related to "Index options can no > > longer be changed dynamically" as mentioned at > > https://lucene.apache.org/core/9_0_0/MIGRATE.html although it fails before > > our code attempts to update the index, and we are not trying to change any > > index options. > > > > Adding some displays to IndexWriter and FieldInfos and logging rather than > > throwing the exception I see > > > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=DOCS > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > > > where there is one line per segment. It logs the exception whenever > > other=DOCS. Subset with segment info: > > > > segment _x8(8.2.0):c31753/-1:[diagnostics={timestamp=1565623850605, > > lucene.version=8.2.0, java.vm.version=11.0.3+7, java.version=11.0.3, > > mergeMaxNumSegments=-1, os.version=3.1.0-1.2-desktop, > > java.vendor=AdoptOpenJDK, source=merge, os.arch=amd64, mergeFactor=10, > > java.runtime.version=11.0.3+7, > > os=Linux}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}] > > > > language curr=NONE, other=NONE > > > > segment _y9(8.7.0):c43531/-1:[diagnostics={timestamp=1604597581562, > > lucene.version=8.7.0, java.vm.version=11.0.3+7, java.version=11.0.3, > > mergeMaxNumSegments=-1, os.version=3.1.0-1.2-desktop, > > java.vendor=AdoptOpenJDK, source=merge, os.arch=amd64, mergeFactor=10, > > java.runtime.version=11.0.3+7, > > os=Linux}]:[attributes={Lucene87StoredFieldsFormat.mode=BEST_SPEED}] > > > > language curr=NONE, other=DOCS > > > > NOT throwing java.lang.IllegalArgumentException: cannot change field > > "language" from index options=NONE to inconsistent index options=DOCS > > > > > > Some variation on an old-fashioned not set versus not present bug perhaps? > > > > > > -- > > Ian. > > > > -- > Adrien > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
