Thanks for the response. https://issues.apache.org/jira/browse/LUCENE-10314
Will we still be able to decide, maybe years down the line, that we do want to search on fieldX after all, and be able to change the code and reindex the maybe small proportion of documents that have a value for fieldX without having to create a new index from scratch? That happens. -- Ian. On Tue, Dec 14, 2021 at 1:54 PM Adrien Grand <jpou...@gmail.com> wrote: > This looks related to the new changes around schema validation. Lucene > now requires a field to either be absent from a document or be indexed > with the exact same options (index options, points dimensions, norms, > doc values type, etc.) as already indexed documents that also have > this field. > > However it's a bug that Lucene fails to open an index that was legal > in Lucene 8. Can you file a JIRA issue? > > On Mon, Dec 13, 2021 at 4:23 PM Ian Lea <ian....@gmail.com> wrote: > > > > Hi > > > > > > We have a long-standing index with some mandatory fields and some > optional > > fields that has been through multiple lucene upgrades without a full > > rebuild and on testing out an upgrade from version 8.11.0 to 9.0.0, when > > open an IndexWriter we are hitting the exception > > > > Exception in thread "main" java.lang.IllegalArgumentException: cannot > > change field "language" from index options=NONE to inconsistent index > > options=DOCS > > at > > > org.apache.lucene.index.FieldInfo.verifySameIndexOptions(FieldInfo.java:245) > > at > > > org.apache.lucene.index.FieldInfos$FieldNumbers.verifySameSchema(FieldInfos.java:421) > > at > > > org.apache.lucene.index.FieldInfos$FieldNumbers.addOrGet(FieldInfos.java:357) > > at > > > org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:1263) > > at > org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1116) > > > > Where language is one of our optional fields. > > > > Presumably this is at least somewhat related to "Index options can no > > longer be changed dynamically" as mentioned at > > https://lucene.apache.org/core/9_0_0/MIGRATE.html although it fails > before > > our code attempts to update the index, and we are not trying to change > any > > index options. > > > > Adding some displays to IndexWriter and FieldInfos and logging rather > than > > throwing the exception I see > > > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=DOCS > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=NONE > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > language curr=NONE, other=DOCS > > > > where there is one line per segment. It logs the exception whenever > > other=DOCS. Subset with segment info: > > > > segment _x8(8.2.0):c31753/-1:[diagnostics={timestamp=1565623850605, > > lucene.version=8.2.0, java.vm.version=11.0.3+7, java.version=11.0.3, > > mergeMaxNumSegments=-1, os.version=3.1.0-1.2-desktop, > > java.vendor=AdoptOpenJDK, source=merge, os.arch=amd64, mergeFactor=10, > > java.runtime.version=11.0.3+7, > > os=Linux}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}] > > > > language curr=NONE, other=NONE > > > > segment _y9(8.7.0):c43531/-1:[diagnostics={timestamp=1604597581562, > > lucene.version=8.7.0, java.vm.version=11.0.3+7, java.version=11.0.3, > > mergeMaxNumSegments=-1, os.version=3.1.0-1.2-desktop, > > java.vendor=AdoptOpenJDK, source=merge, os.arch=amd64, mergeFactor=10, > > java.runtime.version=11.0.3+7, > > os=Linux}]:[attributes={Lucene87StoredFieldsFormat.mode=BEST_SPEED}] > > > > language curr=NONE, other=DOCS > > > > NOT throwing java.lang.IllegalArgumentException: cannot change field > > "language" from index options=NONE to inconsistent index options=DOCS > > > > > > Some variation on an old-fashioned not set versus not present bug > perhaps? > > > > > > -- > > Ian. > > > > -- > Adrien > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
