[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[ https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946528#comment-13946528 ] Robert Muir commented on LUCENE-5553: - This stuff essentially needs the pattern of IOUtils.close, close everything, deliver the first exception. IndexReader#ReaderClosedListener is not always called on IndexReader#close() Key: LUCENE-5553 URL: https://issues.apache.org/jira/browse/LUCENE-5553 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.7, 5.0 Reporter: Simon Willnauer Priority: Blocker Fix For: 4.7.1 Today IndexReader#ReaderClosedListener might not be called if the last IndexReader#decRef() call runs into an exception on IndexReader#doClose(). Today we just reset the refCount and never go and call the listeners. There seem to be a bunch of problems here along the same lines but IMO if we close a reader it should close all resources no matter what exception it runs into. What this should do is call the close listeners in a finally block and then rethrow the exception. The real problem here for apps relying on the listener to release resources is that you might leak memory or file handles or whatnot which I think is a bug how we handle closing the IR. As a side-note I think we should never reset the reference here to be honest. Along the same lines I think we need to fix the loop in IndexReader#notifyReaderClosedListeners() to make sure we call all of them in the case any of them throws an exception. It also seems that SegmentCoreReaders#decRef() has a similar problem where for instance a fieldsReader can throw an exception on close and we never call the core listeners. IMO we need to fix this for 4.7.1 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[
https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946554#comment-13946554
]
Shai Erera commented on LUCENE-5553:
bq. if the last IndexReader#decRef() call runs into an exception on
IndexReader#doClose(). Today we just reset the refCount and never go and call
the listeners.
As I read the code, if doClose() hits an exception, we put the reference back,
then the exception is thrown further and therefore {{closed}} isn't set to
true. So the reader is in fact not closed, and therefore you can attempt to
{{r.close()}} is again? And also that's probably why we don't notify the
listeners?
bq. I think we need to fix the loop in
IndexReader#notifyReaderClosedListeners() to make sure we call all of them
+1!
bq. It also seems that SegmentCoreReaders#decRef() has a similar problem where
for instance a fieldsReader can throw an exception on close and we never call
the core listeners.
Hmm ... this is trickier. Here, unlike IndexReader.close(), we don't put the
reference back. So if that's a bug, we should put back the reference if any of
the IOUtils.close() failed. But if that's ok, then +1 to notify the listeners
irregardless if close hit an exception.
IndexReader#ReaderClosedListener is not always called on IndexReader#close()
Key: LUCENE-5553
URL: https://issues.apache.org/jira/browse/LUCENE-5553
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Affects Versions: 4.7, 5.0
Reporter: Simon Willnauer
Priority: Blocker
Fix For: 4.7.1
Today IndexReader#ReaderClosedListener might not be called if the last
IndexReader#decRef() call runs into an exception on IndexReader#doClose().
Today we just reset the refCount and never go and call the listeners. There
seem to be a bunch of problems here along the same lines but IMO if we close
a reader it should close all resources no matter what exception it runs into.
What this should do is call the close listeners in a finally block and then
rethrow the exception. The real problem here for apps relying on the listener
to release resources is that you might leak memory or file handles or whatnot
which I think is a bug how we handle closing the IR. As a side-note I think
we should never reset the reference here to be honest.
Along the same lines I think we need to fix the loop in
IndexReader#notifyReaderClosedListeners() to make sure we call all of them in
the case any of them throws an exception. It also seems that
SegmentCoreReaders#decRef() has a similar problem where for instance a
fieldsReader can throw an exception on close and we never call the core
listeners.
IMO we need to fix this for 4.7.1
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[
https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946578#comment-13946578
]
Shai Erera commented on LUCENE-5553:
bq. As a side-note I think we should never reset the reference here to be
honest.
Didn't see that -- if you say that close() should render the IR closed, no
matter what exception it hit, then you're right -- we should fix the code to
notify the listeners but also mark {{closed=true}}. And I think that's not a
bad idea!
E.g., I looked at SegmentReader.doClose() and I think it too isn't
exception-safe. If any of the CloseableThreadLocals throw an exception (very
unlikely), we fail to decRef SegDocValues.
But what's worse is the whole chain -- IR.close() - SR.doClose() -- if an
exception is thrown from SegDocValues.decRef(), it happens after
{{core.decRef()}} succeeded. IR.close() hits the exception and doesn't mark
itself as closed. So it doesn't notify the listeners cause it thinks it isn't
closed, but in fact I think if you try to use it you will hit an exception ..
it's in a weird state?
IndexReader#ReaderClosedListener is not always called on IndexReader#close()
Key: LUCENE-5553
URL: https://issues.apache.org/jira/browse/LUCENE-5553
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Affects Versions: 4.7, 5.0
Reporter: Simon Willnauer
Priority: Blocker
Fix For: 4.7.1
Today IndexReader#ReaderClosedListener might not be called if the last
IndexReader#decRef() call runs into an exception on IndexReader#doClose().
Today we just reset the refCount and never go and call the listeners. There
seem to be a bunch of problems here along the same lines but IMO if we close
a reader it should close all resources no matter what exception it runs into.
What this should do is call the close listeners in a finally block and then
rethrow the exception. The real problem here for apps relying on the listener
to release resources is that you might leak memory or file handles or whatnot
which I think is a bug how we handle closing the IR. As a side-note I think
we should never reset the reference here to be honest.
Along the same lines I think we need to fix the loop in
IndexReader#notifyReaderClosedListeners() to make sure we call all of them in
the case any of them throws an exception. It also seems that
SegmentCoreReaders#decRef() has a similar problem where for instance a
fieldsReader can throw an exception on close and we never call the core
listeners.
IMO we need to fix this for 4.7.1
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[ https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946693#comment-13946693 ] Shai Erera commented on LUCENE-5553: Looks good. I'd maybe rename reThrowSilent to reThrowUnchecked cause it's not silent - it sure will hit someone very loudly! :) Otherwise, +1! IndexReader#ReaderClosedListener is not always called on IndexReader#close() Key: LUCENE-5553 URL: https://issues.apache.org/jira/browse/LUCENE-5553 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.7, 5.0 Reporter: Simon Willnauer Priority: Blocker Fix For: 4.7.1 Attachments: LUCENE-5553.patch, LUCENE-5553.patch Today IndexReader#ReaderClosedListener might not be called if the last IndexReader#decRef() call runs into an exception on IndexReader#doClose(). Today we just reset the refCount and never go and call the listeners. There seem to be a bunch of problems here along the same lines but IMO if we close a reader it should close all resources no matter what exception it runs into. What this should do is call the close listeners in a finally block and then rethrow the exception. The real problem here for apps relying on the listener to release resources is that you might leak memory or file handles or whatnot which I think is a bug how we handle closing the IR. As a side-note I think we should never reset the reference here to be honest. Along the same lines I think we need to fix the loop in IndexReader#notifyReaderClosedListeners() to make sure we call all of them in the case any of them throws an exception. It also seems that SegmentCoreReaders#decRef() has a similar problem where for instance a fieldsReader can throw an exception on close and we never call the core listeners. IMO we need to fix this for 4.7.1 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[ https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946720#comment-13946720 ] Michael McCandless commented on LUCENE-5553: +1 for the patch and to change to close really closes and throws first exc behavior. IndexReader#ReaderClosedListener is not always called on IndexReader#close() Key: LUCENE-5553 URL: https://issues.apache.org/jira/browse/LUCENE-5553 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.7, 5.0 Reporter: Simon Willnauer Priority: Blocker Fix For: 4.7.1 Attachments: LUCENE-5553.patch, LUCENE-5553.patch, LUCENE-5553.patch Today IndexReader#ReaderClosedListener might not be called if the last IndexReader#decRef() call runs into an exception on IndexReader#doClose(). Today we just reset the refCount and never go and call the listeners. There seem to be a bunch of problems here along the same lines but IMO if we close a reader it should close all resources no matter what exception it runs into. What this should do is call the close listeners in a finally block and then rethrow the exception. The real problem here for apps relying on the listener to release resources is that you might leak memory or file handles or whatnot which I think is a bug how we handle closing the IR. As a side-note I think we should never reset the reference here to be honest. Along the same lines I think we need to fix the loop in IndexReader#notifyReaderClosedListeners() to make sure we call all of them in the case any of them throws an exception. It also seems that SegmentCoreReaders#decRef() has a similar problem where for instance a fieldsReader can throw an exception on close and we never call the core listeners. IMO we need to fix this for 4.7.1 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[ https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946721#comment-13946721 ] Robert Muir commented on LUCENE-5553: - +1 IndexReader#ReaderClosedListener is not always called on IndexReader#close() Key: LUCENE-5553 URL: https://issues.apache.org/jira/browse/LUCENE-5553 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.7, 5.0 Reporter: Simon Willnauer Priority: Blocker Fix For: 4.7.1 Attachments: LUCENE-5553.patch, LUCENE-5553.patch, LUCENE-5553.patch Today IndexReader#ReaderClosedListener might not be called if the last IndexReader#decRef() call runs into an exception on IndexReader#doClose(). Today we just reset the refCount and never go and call the listeners. There seem to be a bunch of problems here along the same lines but IMO if we close a reader it should close all resources no matter what exception it runs into. What this should do is call the close listeners in a finally block and then rethrow the exception. The real problem here for apps relying on the listener to release resources is that you might leak memory or file handles or whatnot which I think is a bug how we handle closing the IR. As a side-note I think we should never reset the reference here to be honest. Along the same lines I think we need to fix the loop in IndexReader#notifyReaderClosedListeners() to make sure we call all of them in the case any of them throws an exception. It also seems that SegmentCoreReaders#decRef() has a similar problem where for instance a fieldsReader can throw an exception on close and we never call the core listeners. IMO we need to fix this for 4.7.1 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[ https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946730#comment-13946730 ] ASF subversion and git services commented on LUCENE-5553: - Commit 1581400 from [~simonw] in branch 'dev/trunk' [ https://svn.apache.org/r1581400 ] LUCENE-5553: IndexReader#ReaderClosedListener is not always called on IndexReader#close() IndexReader#ReaderClosedListener is not always called on IndexReader#close() Key: LUCENE-5553 URL: https://issues.apache.org/jira/browse/LUCENE-5553 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.7, 5.0 Reporter: Simon Willnauer Priority: Blocker Fix For: 4.7.1 Attachments: LUCENE-5553.patch, LUCENE-5553.patch, LUCENE-5553.patch Today IndexReader#ReaderClosedListener might not be called if the last IndexReader#decRef() call runs into an exception on IndexReader#doClose(). Today we just reset the refCount and never go and call the listeners. There seem to be a bunch of problems here along the same lines but IMO if we close a reader it should close all resources no matter what exception it runs into. What this should do is call the close listeners in a finally block and then rethrow the exception. The real problem here for apps relying on the listener to release resources is that you might leak memory or file handles or whatnot which I think is a bug how we handle closing the IR. As a side-note I think we should never reset the reference here to be honest. Along the same lines I think we need to fix the loop in IndexReader#notifyReaderClosedListeners() to make sure we call all of them in the case any of them throws an exception. It also seems that SegmentCoreReaders#decRef() has a similar problem where for instance a fieldsReader can throw an exception on close and we never call the core listeners. IMO we need to fix this for 4.7.1 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[ https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946743#comment-13946743 ] ASF subversion and git services commented on LUCENE-5553: - Commit 1581404 from [~simonw] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1581404 ] LUCENE-5553: IndexReader#ReaderClosedListener is not always called on IndexReader#close() IndexReader#ReaderClosedListener is not always called on IndexReader#close() Key: LUCENE-5553 URL: https://issues.apache.org/jira/browse/LUCENE-5553 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.7, 5.0 Reporter: Simon Willnauer Priority: Blocker Fix For: 4.7.1 Attachments: LUCENE-5553.patch, LUCENE-5553.patch, LUCENE-5553.patch Today IndexReader#ReaderClosedListener might not be called if the last IndexReader#decRef() call runs into an exception on IndexReader#doClose(). Today we just reset the refCount and never go and call the listeners. There seem to be a bunch of problems here along the same lines but IMO if we close a reader it should close all resources no matter what exception it runs into. What this should do is call the close listeners in a finally block and then rethrow the exception. The real problem here for apps relying on the listener to release resources is that you might leak memory or file handles or whatnot which I think is a bug how we handle closing the IR. As a side-note I think we should never reset the reference here to be honest. Along the same lines I think we need to fix the loop in IndexReader#notifyReaderClosedListeners() to make sure we call all of them in the case any of them throws an exception. It also seems that SegmentCoreReaders#decRef() has a similar problem where for instance a fieldsReader can throw an exception on close and we never call the core listeners. IMO we need to fix this for 4.7.1 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[ https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946766#comment-13946766 ] ASF subversion and git services commented on LUCENE-5553: - Commit 1581408 from [~simonw] in branch 'dev/branches/lucene_solr_4_7' [ https://svn.apache.org/r1581408 ] LUCENE-5553: IndexReader#ReaderClosedListener is not always called on IndexReader#close() IndexReader#ReaderClosedListener is not always called on IndexReader#close() Key: LUCENE-5553 URL: https://issues.apache.org/jira/browse/LUCENE-5553 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.7, 5.0 Reporter: Simon Willnauer Priority: Blocker Fix For: 4.7.1 Attachments: LUCENE-5553.patch, LUCENE-5553.patch, LUCENE-5553.patch Today IndexReader#ReaderClosedListener might not be called if the last IndexReader#decRef() call runs into an exception on IndexReader#doClose(). Today we just reset the refCount and never go and call the listeners. There seem to be a bunch of problems here along the same lines but IMO if we close a reader it should close all resources no matter what exception it runs into. What this should do is call the close listeners in a finally block and then rethrow the exception. The real problem here for apps relying on the listener to release resources is that you might leak memory or file handles or whatnot which I think is a bug how we handle closing the IR. As a side-note I think we should never reset the reference here to be honest. Along the same lines I think we need to fix the loop in IndexReader#notifyReaderClosedListeners() to make sure we call all of them in the case any of them throws an exception. It also seems that SegmentCoreReaders#decRef() has a similar problem where for instance a fieldsReader can throw an exception on close and we never call the core listeners. IMO we need to fix this for 4.7.1 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5553) IndexReader#ReaderClosedListener is not always called on IndexReader#close()
[ https://issues.apache.org/jira/browse/LUCENE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947080#comment-13947080 ] Uwe Schindler commented on LUCENE-5553: --- Thanks, cool. I knew about this problem, but I was not aware that it is so serious. Thanks for adding suppressed Exceptions using Throwable#addSupressed(). IndexReader#ReaderClosedListener is not always called on IndexReader#close() Key: LUCENE-5553 URL: https://issues.apache.org/jira/browse/LUCENE-5553 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.7, 5.0 Reporter: Simon Willnauer Assignee: Simon Willnauer Priority: Blocker Fix For: 4.8, 5.0, 4.7.1 Attachments: LUCENE-5553.patch, LUCENE-5553.patch, LUCENE-5553.patch Today IndexReader#ReaderClosedListener might not be called if the last IndexReader#decRef() call runs into an exception on IndexReader#doClose(). Today we just reset the refCount and never go and call the listeners. There seem to be a bunch of problems here along the same lines but IMO if we close a reader it should close all resources no matter what exception it runs into. What this should do is call the close listeners in a finally block and then rethrow the exception. The real problem here for apps relying on the listener to release resources is that you might leak memory or file handles or whatnot which I think is a bug how we handle closing the IR. As a side-note I think we should never reset the reference here to be honest. Along the same lines I think we need to fix the loop in IndexReader#notifyReaderClosedListeners() to make sure we call all of them in the case any of them throws an exception. It also seems that SegmentCoreReaders#decRef() has a similar problem where for instance a fieldsReader can throw an exception on close and we never call the core listeners. IMO we need to fix this for 4.7.1 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
