I think I know, what is different than earlier versions: The close of the Reader is done by Tokenizer.close(). As in 2.9.0 we did much work to get all the Tokenizers reusable, there is the following problem:
The Reader is only closed, if you call Tokenizer.close(), but not, if you call Tokenizer.reset(newReader). This was always be so (and has nothing to do with the new TokenStream API), but in earlier version not all analyzers were able to reuse the TokenStreams. Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Sunday, September 27, 2009 10:42 AM > To: java-dev@lucene.apache.org > Subject: RE: Lucene 2.9.0-rc5 : Reader stays open after > IndexWriter.updateDocument(), is that possible? > > How does your test perform with 2.4.1 ? > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > -----Original Message----- > > From: Chris Hostetter [mailto:hossman_luc...@fucit.org] > > Sent: Sunday, September 27, 2009 3:34 AM > > To: Lucene Dev > > Subject: Re: Lucene 2.9.0-rc5 : Reader stays open after > > IndexWriter.updateDocument(), is that possible? > > > > > > : Thanks Mark for the pointer, I thought somehow that lucene closed them > > as a > > : convenience, I don't know if it did that in previous releases (aka > > 2.4.1) but > > : I'll close them myself from now on. > > > > FWIW: As far as i know, Lucene has always closed the Reader for you when > > calling addDocument/updateDocument -- BUT -- the docs never promized > > that Lucene would close any Readers used in Fields. In fact the Field > > constructor docs say "you may not close the Reader until addDocument has > > been called" suggesting that you should close it yourself. > > (Reader.close() is very clear that there should be no effect on closing > a > > Reader multiple times, so this is safe no matter what Lucene does) > > > > That said: If the behavior has changed in 2.9, this could easily bite > lots > > of people in the ass if they haven't been closing their readers and now > > they run out of file handles. I wrote a quick test to try and reproduce > > the problem you're describing, but as far as i can tell 2.9.0 > > (final) still seems to close the Reader for you. > > > > Can anyone else reproduce this problem of Reader's in Field's not > getting > > closed? (my test is below) > > > > --BEGIN-- > > package org.apache.lucene; > > > > /** > > * Licensed to the Apache Software Foundation (ASF) under one or more > > * contributor license agreements. See the NOTICE file distributed with > > * this work for additional information regarding copyright ownership. > > * The ASF licenses this file to You under the Apache License, Version > 2.0 > > * (the "License"); you may not use this file except in compliance with > > * the License. You may obtain a copy of the License at > > * > > * http://www.apache.org/licenses/LICENSE-2.0 > > * > > * Unless required by applicable law or agreed to in writing, software > > * distributed under the License is distributed on an "AS IS" BASIS, > > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or > > implied. > > * See the License for the specific language governing permissions and > > * limitations under the License. > > */ > > > > import org.apache.lucene.analysis.KeywordAnalyzer; > > import org.apache.lucene.index.*; > > import org.apache.lucene.document.*; > > import org.apache.lucene.util.LuceneTestCase; > > import org.apache.lucene.store.RAMDirectory; > > > > import java.io.*; > > > > public class TestFieldWithReaderClosing extends LuceneTestCase { > > > > IndexWriter writer = null; > > Document d = null; > > CloseStateReader reader; > > public void setUp() throws Exception { > > writer = new IndexWriter(new RAMDirectory(), > > new KeywordAnalyzer(), true, > > IndexWriter.MaxFieldLength.LIMITED); > > d = new Document(); > > d.add(new Field("id", "x", Field.Store.YES, Field.Index.ANALYZED)); > > reader = new CloseStateReader("foo"); > > d.add(new Field("contents", reader)); > > } > > public void tearDown() throws Exception { > > writer.close(); > > writer = null; > > reader.close(); > > reader = null; > > } > > > > public void testAdd() throws Exception { > > writer.addDocument(d); > > assertEquals("close count should be 1", 1, reader.getCloseCount()); > > writer.close(); > > assertEquals("close count should still be 1", 1, > > reader.getCloseCount()); > > } > > public void testEmptyUpdate() throws Exception { > > writer.updateDocument(new Term("id","x"), d); > > assertEquals("close count should be 1", 1, reader.getCloseCount()); > > writer.close(); > > assertEquals("close count should still be 1", 1, > > reader.getCloseCount()); > > } > > public void testAddAndUpdate() throws Exception { > > writer.addDocument(d); > > assertEquals("close count should be 1", 1, reader.getCloseCount()); > > d = new Document(); > > d.add(new Field("id", "x", Field.Store.YES, Field.Index.ANALYZED)); > > reader = new CloseStateReader("foo"); > > d.add(new Field("contents", reader)); > > writer.updateDocument(new Term("id","x"), d); > > assertEquals("new close count should be 1", 1, > > reader.getCloseCount()); > > writer.close(); > > assertEquals("new close count should still be 1", 1, > > reader.getCloseCount()); > > } > > > > > > static class CloseStateReader extends StringReader { > > private int closeCount = 0; > > public CloseStateReader(String s) { > > super(s); > > } > > public synchronized void close() { > > closeCount++; > > super.close(); > > } > > public int getCloseCount() { > > return closeCount; > > } > > } > > } > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-dev-h...@lucene.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
