[
https://issues.apache.org/jira/browse/SOLR-14677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178047#comment-17178047
]
ASF subversion and git services commented on SOLR-14677:
--------------------------------------------------------
Commit 8bc6688e6a846451f7b94052290588c70e4dfd99 in lucene-solr's branch
refs/heads/branch_8x from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8bc6688 ]
SOLR-14677: Always close DIH EntityProcessor/DataSource (#1741)
Prior to this commit, the wrapup logic at the end of
DocBuilder.execute() closed out a series of DIH objects, but did so
in a way that an exception closing any of them resulted in the remainder
staying open. This is especially problematic since Writer.close()
throws exceptions that DIH uses to determine the success/failure of the
run.
In practice this caused network errors sending DIH data to other Solr
nodes to result in leaked JDBC connections.
This commit changes DocBuilder's termination logic to handle exceptions
more gracefully, ensuring that errors closing a DIHWriter (for example)
don't prevent the closure of entity-processor and DataSource objects.
> DIH doesnt close DataSource when import encounters errors
> ---------------------------------------------------------
>
> Key: SOLR-14677
> URL: https://issues.apache.org/jira/browse/SOLR-14677
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - DataImportHandler
> Affects Versions: 7.5, master (9.0)
> Reporter: Jason Gerlowski
> Assignee: Jason Gerlowski
> Priority: Minor
> Attachments: error-solr.log, no-error-solr.log
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> DIH imports don't close DataSource's (which can hold db connections, etc.) in
> all cases. Specifically, if an import runs into an unexpected error
> forwarding processed docs to other nodes, it will neglect to close the
> DataSource's when it finishes.
> This problem goes back to at least 7.5. This is partially mitigated in older
> versions of some DataSource implementations (e.g. JdbcDataSource) by means of
> a "finalize" hook which invokes "close()" when the DataSource object is
> garbage-collected. In practice, this means that resources might be held open
> longer than necessary but will be closed within a few seconds or minutes by
> GC. This only helps JdbcDataSource though - all other DataSource impl's risk
> leaking resources.
> In master/9.0, which requires a minimum of Java 11 and doesn't have the
> finalize-hook, the connections are never cleaned up when an error is
> encountered during DIH. DIH will likely be removed for the 9.0 release, but
> if it isn't this bug should be fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
