Maybe you can contribute the code you have for managing multiple indices and we can iterate/debug from there?
Somehow we need to expose this failure in a standalone test case so we can isolate it. Mike McCandless http://blog.mikemccandless.com On Thu, Oct 20, 2016 at 1:57 AM, Trejkaz <trej...@trypticon.org> wrote: > Hi all. > > I seem to have a situation where ReaderManager is reducing a refCount > to 0 before it actually releases all its references. > > It's difficult because it's all mixed up in our framework for multiple > ReaderManagers, which I'm still not convinced works because the > concurrency is impossible to figure out, and probably won't be allowed > to publish in order to have anyone at Lucene look at it either. (Which > is why I hope that someone at Lucene figures out how to manage more > than one index reliably one day...) > > The stack trace trying to close the directory is just trying to > refresh the reader, but I guess this reader was the last one using a > Directory, so now we're closing that as well: > > java.lang.RuntimeException: Resources inside the directory did not > get closed before closing the directory > at > com.acme.storage.textindex.store.CloseCheckingDirectory.close(CloseCheckingDirectory.java:109) > at > com.acme.storage.textindex.index.DefaultIndexReaderSharer$IndexReaderWrapper.release(DefaultIndexReaderSharer.java:146) > at > com.acme.storage.textindex.index.DefaultIndexReaderSharer$IndexReaderWrapper.access$100(DefaultIndexReaderSharer.java:77) > at > com.acme.storage.textindex.index.DefaultIndexReaderSharer.release(DefaultIndexReaderSharer.java:45) > at > com.acme.storage.textindex.DefaultTextIndex$WrappingReaderManager$1.doClose(DefaultTextIndex.java:370) > at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:253) > at > com.acme.storage.textindex.DefaultTextIndex$WrappingReaderManager.decRef(DefaultTextIndex.java:331) > at > com.acme.storage.textindex.DefaultTextIndex$WrappingReaderManager.decRef(DefaultTextIndex.java:306) > at > org.apache.lucene.search.ReferenceManager.release(ReferenceManager.java:274) > at > org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:189) > at > org.apache.lucene.search.ReferenceManager.maybeRefreshBlocking(ReferenceManager.java:253) > > The stack trace which opened the resource and didn't close it is > apparently the first reader which ReaderManager: > > Caused by: java.lang.RuntimeException: unclosed IndexInput: _7d.tvd > at > com.acme.storage.textindex.store.CloseCheckingDirectory.addOpenResource(CloseCheckingDirectory.java:82) > at > com.acme.storage.textindex.store.CloseCheckingDirectory.openInput(CloseCheckingDirectory.java:57) > at > org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.<init>(CompressingTermVectorsReader.java:144) > at > org.apache.lucene.codecs.compressing.CompressingTermVectorsFormat.vectorsReader(CompressingTermVectorsFormat.java:91) > at > org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:120) > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65) > at > org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58) > at > org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:50) > at > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731) > at > org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:50) > at > org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63) > at > com.acme.storage.textindex.index.DefaultIndexReaderSharer$CustomReaderManager.<init>(DefaultIndexReaderSharer.java:164) > > But if it's the first reader held by the ReaderManager, I wouldn't > expect the refCount to be 0, so it shouldn't be closing the directory. > > I can't reproduce this myself, so I can't just dump out conveniently > placed messages to figure out how it's happening... > > But has anyone else seen something like this? > > CustomReaderManager is probably shareable, it just does this: > > private static class CustomReaderManager extends > ReferenceManager<DirectoryReader> { > private CustomReaderManager(Directory directory) throws IOException { > current = > UnInvertingDirectoryReader.wrap(DirectoryReader.open(directory)); > } > > @Override > protected void decRef(DirectoryReader reference) throws IOException { > reference.decRef(); > } > > @Override > protected DirectoryReader refreshIfNeeded(DirectoryReader > referenceToRefresh) throws IOException { > return DirectoryReader.openIfChanged(referenceToRefresh); > } > > @Override > protected boolean tryIncRef(DirectoryReader reference) { > return reference.tryIncRef(); > } > > @Override > protected int getRefCount(DirectoryReader reference) { > return reference.getRefCount(); > } > } > > So basically the same as the normal one, except that it wraps the > reader in an UnInvertingDirectoryReader. The only reason we're forced > to subclass the manager to do this is that if we don't, each > UnInvertingDirectoryReader becomes a new instance, and basic caching > stuff stops working in some way. > > DefaultIndexReaderSharer#release() is like this: > > private synchronized void release(DirectoryReader reader) { > try { > if (readerManager == null) { > reader.decRef(); // assumes we don't own it anymore > return; > } > > readerManager.release(reader); > if (reader.getRefCount() == 1) { > readerManager.close(); // 👈 close > happens from here > readerManager = null; > > reader.directory().close(); > } > } catch (IOException e) { > Logger.getLogger(getClass()).warn("Error closing reader", e); > } > } > > So it's releasing the reader and then checking the ref count. If the > count is now 1, it closes it, but who knows, maybe this can race, > despite the synchronized keyword put here explicitly to prevent that, > based on the advice last time. > > Basically, it is still essentially impossible to manage indexes > without getting weird errors like this. I can't even tell whether > we're doing something wrong or whether Lucene is simply not closing > the files it's opening. :/ > > TX > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
