[
https://issues.apache.org/jira/browse/OPENJPA-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17820042#comment-17820042
]
Romain Manni-Bucau edited comment on OPENJPA-2919 at 2/23/24 12:26 PM:
-----------------------------------------------------------------------
I don't use dbcp since some years but using tomcat-dbcp the keys are 1. enforce
a global default query/statement timeout (validation + runtime) + 2. ensure
validation runs often enough for your infra (can be from every 10mn to every
30s depending the perfs and instability).
was (Author: romain.manni-bucau):
I don't use dbcp since some years but using tomcat-dbcp the keys are 1. enforce
a global default query/statement timeout (validation + runtime) + ensure
validation runs often enough for your infra (can be from every 10mn to every
30s depending the perfs and instability).
> Connection pool can be exhausted when connections are killed on the DB side
> ---------------------------------------------------------------------------
>
> Key: OPENJPA-2919
> URL: https://issues.apache.org/jira/browse/OPENJPA-2919
> Project: OpenJPA
> Issue Type: Bug
> Components: jdbc
> Affects Versions: 3.2.2
> Reporter: Dénes Bodó
> Priority: Critical
> Labels: deadlock, robustness
> Attachments: Screenshot 2024-02-23 at 09.02.19.png
>
>
> Apache Oozie 5.2.1 uses OpenJPA 2.4.2 and commons-dbcp 1.4 and commons-pool
> 1.5.4. These are ancient versions, I know.
> h1. Description
> The issue is that when due to some network issues or "maintenance work" on
> the DB side (especially PostgreSQL) which causes the DB connection to be
> closed, it results exhausted Pool on the client side. Many threads are
> waiting at this point:
> {noformat}
> "pool-2-thread-4" #20 prio=5 os_prio=31 tid=0x00007faf7903b800 nid=0x8603
> waiting on condition [0x000000030f3e7000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000000066aca8e70> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:1324)
> {noformat}
> According to my observation this is because the JDBC driver does not get
> closed on the client side, nor the abstract DBCP connection
> _org.apache.commons.dbcp2.PoolableConnection_ .
> h1. Repro
> (Un)Fortunately I can reproduce the issue using the latest and greatest
> commons-dbcp 2.11.0 and commons-pool 2.12.0 along with OpenJPA 3.2.2.
> I've just created a Java application to reproduce the issue:
> [https://github.com/dionusos/pool_exhausted_repro] . See README.md for
> detailed repro steps.
> h1. What we tried so far
> I got in touch with DBCP team who confirmed that in case of an error in the
> connection the client (in this case OpenJPA is the client of DBCP) should
> handle the exception like closing the connection: DBCP-595. I agree with them
> as based on the investigation I did I can also confirm that DBCP is really
> robust when the client releases the broken connection object after catching
> SQLException. Please check the 4 comments on DBCP-595 for extra details.
> h1. Ask
> OpenJPA team!
> * Could you please confirm that my findings are valid?
> * Did I do anything wrong in my repro program?
> * Oozie has retry logic implemented:
> [https://github.com/apache/oozie/blob/318fac5/core/src/main/java/org/apache/oozie/service/JPAService.java#L397L427]
> but this cannot avoid the reported dead lock.
> * Do you have any questions I can answer to help in the investigation?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
