[ https://issues.apache.org/jira/browse/LUCENE-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785625#comment-16785625 ]
Simon Willnauer commented on LUCENE-8692: ----------------------------------------- {quote} For now I've updated the patch to take the simplest possible approach to checking for MergeAbortedException {quote} +1 {quote} Well, to flip your question around: is there an example of a Throwable you can think of bubbling up out of IndexWriter.startCommit() that should NOT be considered fatal? {quote} I think we need to be careful here. From my perspective there are 3 types of exceptions here: * unrecoverable exceptions aka. VirtualMachineErrors * exceptions that happen during indexing and are not recoverable (these are handled in DocumentsWriter) * exceptions that cause dataloss or inconsistencies (we didn't handle those as fatal yet at least not consistently) but we only catch VirtualMachineError. Those are in particular: * getReader() * deleteAll() * addIndexes() * flushNextBuffer() * prepareCommitInternal() * doFlush() * startCommit() Those methods might cause documents go missing etc. but we treated them not as fatal or tragic events since a user could always call rollback() to go back the the last known safe-point / previous commit. Now we can debate if we want to change this and we can, in-fact I am all for making it even more strict especially since it's inconsistent with what we do if addDocument fails with an aborting exception. If we do that we need to see if rollback still has a purpose and maybe remove it? now speaking of maybeMerge I don't see why we need to close the index writer with a tragic event, there is no dataloss nor an inconsistency? From that logic I don't think we need to handle these exceptions in such a drastic way? {quote} I don't use github for lucene development – I track all contributions as patches in the official issue tracker for the project as recommended by our official guidelines : ) ... but i'll go ahead and create a jira/LUCENE-8692 branch if that will help you review. {quote} Bummer, I am not sure branches help. Working like it's still 1999 is a pain we should fix our guidelines. > IndexWriter.getTragicException() nay not reflect all corrupting exceptions > (notably: NoSuchFileException) > --------------------------------------------------------------------------------------------------------- > > Key: LUCENE-8692 > URL: https://issues.apache.org/jira/browse/LUCENE-8692 > Project: Lucene - Core > Issue Type: Bug > Reporter: Hoss Man > Priority: Major > Attachments: LUCENE-8692.patch, LUCENE-8692.patch, LUCENE-8692.patch, > LUCENE-8692_test.patch > > > Backstory... > Solr has a "LeaderTragicEventTest" which uses MockDirectoryWrapper's > {{corruptFiles}} to introduce corruption into the "leader" node's index and > then assert that this solr node gives up it's leadership of the shard and > another replica takes over. > This can currently fail sporadically (but usually reproducibly - > seeSOLR-13237) due to the leader not giving up it's leadership even after the > corruption causes an update/commit to fail. Solr's leadership code makes > this decision after encountering an exception from the IndexWriter based on > wether {{IndexWriter.getTragicException()}} is (non-)null. > ---- > While investigating this, I created an isolated Lucene-Core equivilent test > that demonstrates the same basic situation: > * Gradually cause corruption on an index untill (otherwise) valid execution > of IW.add() + IW.commit() calls throw an exception to the IW client. > * assert that if an exception is thrown to the IW client, > {{getTragicException()}} is now non-null. > It's fairly easy to make my new test fail reproducibly -- in every situation > I've seen the underlying exception is a {{NoSuchFileException}} (ie: the > randomly introduced corruption was to delete some file). -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org