[
https://issues.apache.org/jira/browse/SOLR-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Rafalovitch resolved SOLR-6644.
-----------------------------------------
Resolution: Won't Fix
> DataImportHandler holds on to each DB connection until the end
> --------------------------------------------------------------
>
> Key: SOLR-6644
> URL: https://issues.apache.org/jira/browse/SOLR-6644
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 4.7, 4.10.1
> Reporter: Gunnlaugur Thor Briem
> Priority: Major
> Labels: dataimport, jdbc
>
> DataImportHandler with a JDBC data source opens one DB connection per entity,
> and then holds on to that DB connection with an open transaction after it's
> finished processing that entity ... right until the whole DataImportHandler
> operation is finished.
> So this can mean dozens of DB connections tied up for hours, unnecessarily
> --- with each connection staying in "idle in transaction" state, holding (in
> PostgreSQL) an AccessShareLock on each relation it has looked at. Not ideal
> for production operations, of course.
> Here are the connections from Solr to the DB when a large import has been
> running for a while:
> {code}
> backend_start | xact_start | query_start | state | minutes
> idle
> ---------------+------------+-------------+---------------------+--------------
> 20:03:20 | 20:03:20 | 20:03:21 | idle in transaction | 32
> 20:03:22 | 20:03:22 | 20:03:22 | idle in transaction | 32
> 20:03:22 | 20:03:22 | 20:03:22 | idle in transaction | 32
> 20:03:22 | 20:03:22 | 20:03:23 | idle in transaction | 32
> 20:03:21 | 20:03:21 | 20:16:35 | idle in transaction | 19
> 20:03:21 | 20:03:21 | 20:16:35 | idle in transaction | 19
> 20:03:22 | 20:03:22 | 20:16:35 | idle in transaction | 19
> 20:03:22 | 20:03:22 | 20:16:35 | idle in transaction | 19
> 20:03:22 | 20:03:22 | 20:16:35 | idle in transaction | 19
> 20:16:37 | 20:16:37 | 20:16:38 | idle in transaction | 19
> 20:03:21 | 20:03:21 | 20:16:35 | idle in transaction | 19
> 20:03:21 | 20:03:21 | 20:16:35 | idle in transaction | 19
> 20:03:21 | 20:03:21 | 20:16:35 | idle in transaction | 19
> 20:16:36 | 20:16:36 | 20:16:37 | idle in transaction | 19
> 20:03:20 | 20:03:20 | 20:16:35 | idle in transaction | 19
> 20:16:36 | 20:16:36 | 20:35:49 | idle in transaction | 0
> 20:16:36 | 20:16:36 | 20:35:49 | idle in transaction | 0
> 20:16:37 | 20:16:37 | 20:35:49 | idle in transaction | 0
> 20:16:35 | 20:16:35 | 20:35:41 | idle in transaction | 0
> 20:16:36 | 20:16:36 | 20:35:49 | idle in transaction | 0
> 20:16:37 | 20:16:37 | 20:35:49 | active | 0
> {code}
> Most of these haven't been touched for a long time, and will not be needed
> again (because DataImportHandler is done with that top-level entity). They
> should be released as soon as possible.
> Noticed in production in Solr 4.7.0, then reproduced in 4.10.1 (so probably
> also true of all versions inbetween).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
