Hi, In the early days (I mean in the time when it was already read only until we refactored the IndexReader.delete()/Codec stuff), this was working, because the LiveDocs were always handled in a special way. Making it now 100% read-only is in my opinion very bad, as it does not allow to update documents in a 3.x index anymore, so you have no chance, you must run IndexUpgrader.
The usual step like opening old Index and adding documents works (because the new documents are added always to new segment), but the much more usual IW.updateDocument() which is commonly used also to add documents fails on old Indexes. This is a no-go, we have to fix this. If we allow the trick with updating LiveDocs on 3.x codec, for the end-user the "read-only" stuff in Lucene3x codec would be completely invisible, as he can do everything IndexWriter provides. The other horrible things like changing norms is no longer possible, so deletes are the only thing that affects here. The read-only ness of Lucene3x codec would only be visible to the user when someone tries to explicitly create an index with Lucene3x codec. And I understood the CHANGES/MIGRATE.txt exactly as that. I was not aware, that at some point this UOE was added, so I suggest that Robert's inlined patch should be applied! PLEASE FIX! PLEASE FIX! PLEASE FIX! :-) Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Robert Muir [mailto:[email protected]] > Sent: Wednesday, August 29, 2012 5:02 AM > To: [email protected] > Subject: Re: Lucene3xCodec doesn't allow deletions in 4x? > > On Tue, Aug 28, 2012 at 10:06 PM, Chris Hostetter > <[email protected]> wrote: > > On the Solr side of things, i think we should even want to consider > > automaticly running IndexUpgrader on startup if we detect that the > > Lucene3xCodec is in use to simplify things -- we can't even suggest > > running "optimize" as a quick/easy way to force and index format > > upgrade because if the 3x index as already optimized then it's a no-op > > and the index stays in the 3x format. > > > > By the way: i don't really agree with that. If we want to be able to make > deletes > against 3.x segments, thats a trivial patch (hint: > uncomment out the UOE) > > But we just need to decide thats what we want to do (and if should set any > precedent for future backwards compat). > > Personally I wouldn't be against it here because of how we do sneaky stuff for > 3.x segments on commit already (e.g. 'upgrade the SI'), so we already support > 2 > codecs (preflex and preflex-upgraded). this wouldn't make the situation any > worse. > > Index: lucene/test- > framework/src/java/org/apache/lucene/codecs/lucene3x/PreFlexRWCodec.jav > a > ================================================================ > === > --- lucene/test- > framework/src/java/org/apache/lucene/codecs/lucene3x/PreFlexRWCodec.jav > a (revision > 1378430) > +++ lucene/test- > framework/src/java/org/apache/lucene/codecs/lucene3x/PreFlexRWCodec.jav > a (working > copy) > @@ -38,8 +38,6 @@ > private final TermVectorsFormat termVectors = new > PreFlexRWTermVectorsFormat(); > private final SegmentInfoFormat segmentInfos = new > PreFlexRWSegmentInfoFormat(); > private final StoredFieldsFormat storedFields = new > PreFlexRWStoredFieldsFormat(); > - // TODO: this should really be a different impl > - private final LiveDocsFormat liveDocs = new Lucene40LiveDocsFormat(); > > @Override > public PostingsFormat postingsFormat() { @@ -87,15 +85,6 @@ > } > > @Override > - public LiveDocsFormat liveDocsFormat() { > - if (LuceneTestCase.PREFLEX_IMPERSONATION_IS_ACTIVE) { > - return liveDocs; > - } else { > - return super.liveDocsFormat(); > - } > - } > - > - @Override > public StoredFieldsFormat storedFieldsFormat() { > if (LuceneTestCase.PREFLEX_IMPERSONATION_IS_ACTIVE) { > return storedFields; > Index: > lucene/core/src/test/org/apache/lucene/index/TestBackwardsCompatibility.jav > a > ================================================================ > === > --- > lucene/core/src/test/org/apache/lucene/index/TestBackwardsCompatibility.jav > a (revision > 1378430) > +++ > lucene/core/src/test/org/apache/lucene/index/TestBackwardsCompatibility.jav > a (working > copy) > @@ -301,6 +301,22 @@ > dir.close(); > } > } > + > + public void testDeleteOldIndex() throws IOException { > + for (String name : oldNames) { > + if (VERBOSE) { > + System.out.println("TEST: oldName=" + name); > + } > + Directory dir = newDirectory(oldIndexDirs.get(name)); > + IndexWriter iw = new IndexWriter(dir, new > IndexWriterConfig(TEST_VERSION_CURRENT, null)); > + iw.deleteDocuments(new Term("id", "3")); > + iw.close(); > + IndexReader ir = DirectoryReader.open(dir); > + assertEquals(34, ir.numDocs()); > + ir.close(); > + dir.close(); > + } > + } > > private void doTestHits(ScoreDoc[] hits, int expectedCount, IndexReader > reader) throws IOException { > final int hitCount = hits.length; > Index: > lucene/core/src/java/org/apache/lucene/codecs/lucene3x/Lucene3xCodec.java > ================================================================ > === > --- > lucene/core/src/java/org/apache/lucene/codecs/lucene3x/Lucene3xCodec.java > (revision > 1378430) > +++ > lucene/core/src/java/org/apache/lucene/codecs/lucene3x/Lucene3xCodec.java > (working > copy) > @@ -68,12 +68,7 @@ > static final String COMPOUND_FILE_STORE_EXTENSION = "cfx"; > > // TODO: this should really be a different impl > - private final LiveDocsFormat liveDocsFormat = new > Lucene40LiveDocsFormat() { > - @Override > - public void writeLiveDocs(MutableBits bits, Directory dir, > SegmentInfoPerCommit info, int newDelCount, IOContext context) throws > IOException { > - throw new UnsupportedOperationException("this codec can only be > used for reading"); > - } > - }; > + private final LiveDocsFormat liveDocsFormat = new > + Lucene40LiveDocsFormat(); > > // 3.x doesn't support docvalues > private final DocValuesFormat docValuesFormat = new DocValuesFormat() { > > -- > lucidworks.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] For additional > commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
