from:"JIRA"

[jira] [Updated] (OPENJPA-2920) CriteriaBuilder generating suboptimal code

2024-04-12 Thread Rik Schaaf (Jira)



 [ 
https://issues.apache.org/jira/browse/OPENJPA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rik Schaaf updated OPENJPA-2920:

Description: 
*Background:*

I have the following entity:
{code:java}
package com.github.cc007.headsplugin.integration.database.entities;

import lombok.AccessLevel;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import lombok.ToString;

import javax.persistence.CascadeType;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.FetchType;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Index;
import javax.persistence.ManyToMany;
import javax.persistence.Table;
import javax.persistence.Version;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;

@Entity
@Table(name = "heads",
indexes = {
@Index(name = "heads_headowner_index", columnList = 
"headOwner"),
@Index(name = "heads_name_index", columnList = "name")
}
)
@Getter
@Setter
@ToString
@NoArgsConstructor
public class HeadEntity {

@Id
@Column(name = "id", nullable = false)
@GeneratedValue(strategy = GenerationType.AUTO)
@Setter(AccessLevel.NONE)
private long id;

@Version
@Column(name = "version", nullable = false)
private long version;

@Column(name = "headOwner", unique = true, nullable = false)
private String headOwner;

@Column(name = "name", nullable = false)
private String name;

@Column(name = "value", length = 1023, nullable = false)
private String value;

@ToString.Exclude
@ManyToMany(
fetch = FetchType.EAGER,
cascade = {CascadeType.PERSIST, CascadeType.MERGE},
mappedBy = "heads"
)
@Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE)
private Set databases = new HashSet<>();

@ToString.Exclude
@ManyToMany(
fetch = FetchType.EAGER,
cascade = {CascadeType.PERSIST, CascadeType.MERGE},
mappedBy = "heads"
)
@Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE)
private Set tags = new HashSet<>();

@ToString.Exclude
@ManyToMany(
fetch = FetchType.EAGER,
cascade = {CascadeType.PERSIST, CascadeType.MERGE},
mappedBy = "heads"
)
@Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE)
private Set categories = new HashSet<>();

@ToString.Exclude
@ManyToMany(
fetch = FetchType.LAZY,
cascade = {CascadeType.PERSIST, CascadeType.MERGE},
mappedBy = "heads"
)
@Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE)
private Set searches = new HashSet<>();

public Set getDatabases() {
return Collections.unmodifiableSet(databases);
}

public Set getTags() {
return Collections.unmodifiableSet(tags);
}

public Set getCategories() {
return Collections.unmodifiableSet(categories);
}

public Set getSearches() {
return Collections.unmodifiableSet(searches);
}
} {code}
Note the  {{{}unique = true{}}}, {{nullable = false}} (and the 
{{{}heads_headowner_index{}}})

I have created a couple of functions using a CriteriaBuilder:
{code:java}
public List findAllHeadOwnersByHeadOwnerIn(Collection 
headOwners) {
final var resultList = new ArrayList();
final var headOwnerGroups = CollectionUtils.partitionCollection(headOwners, 
configProperties.getDatabase().getChunkSize());
for (final var headOwnerGroup : headOwnerGroups) {
final var query = querySelectionByCondition(HeadEntity.class,
root -> root.get("headOwner"), String.class,
(criteriaBuilder, root) -> 
root.get("headOwner").in(headOwnerGroup)
);
resultList.addAll(query.getResultList());
}
return resultList;
}

public  TypedQuery querySelectionByCondition(
Class entityType,
Function, Path> selection,
Class selectPropertyType,
BiFunction, Predicate> whereCondition
) {
final var criteriaBuilder = entityManager.getCriteriaBuilder();

final var criteriaQuery = criteriaBuilder
.createQuery(selectPropertyType);

final var root = criteriaQuery.from(entityType);


criteriaQuery.select(selection.apply(root)).where(whereCondition.apply(criteriaBuilder,
 root));

return entityManager.createQuery(criteriaQuery);
}

public  Optional getSingleResult(TypedQuery query) {
try {
return Optional.of(query.getSingleResult());
} catch (NoResultException e) {
return Optional.empty();
}
}

public  List getMutableResultList(TypedQuery

[jira] [Created] (OPENJPA-2920) CriteriaBuilder generating suboptimal code

2024-04-12 Thread Rik Schaaf (Jira)

Rik Schaaf created OPENJPA-2920:
---

 Summary: CriteriaBuilder generating suboptimal code
 Key: OPENJPA-2920
 URL: https://issues.apache.org/jira/browse/OPENJPA-2920
 Project: OpenJPA
  Issue Type: Bug
  Components: criteria
Affects Versions: 3.2.0
Reporter: Rik Schaaf


*Background:*

I have the following entity:


{code:java}
package com.github.cc007.headsplugin.integration.database.entities;

import lombok.AccessLevel;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import lombok.ToString;

import javax.persistence.CascadeType;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.FetchType;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Index;
import javax.persistence.ManyToMany;
import javax.persistence.Table;
import javax.persistence.Version;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;

@Entity
@Table(name = "heads",
indexes = {
@Index(name = "heads_headowner_index", columnList = 
"headOwner"),
@Index(name = "heads_name_index", columnList = "name")
}
)
@Getter
@Setter
@ToString
@NoArgsConstructor
public class HeadEntity {

@Id
@Column(name = "id", nullable = false)
@GeneratedValue(strategy = GenerationType.AUTO)
@Setter(AccessLevel.NONE)
private long id;

@Version
@Column(name = "version", nullable = false)
private long version;

@Column(name = "headOwner", unique = true, nullable = false)
private String headOwner;

@Column(name = "name", nullable = false)
private String name;

@Column(name = "value", length = 1023, nullable = false)
private String value;

@ToString.Exclude
@ManyToMany(
fetch = FetchType.EAGER,
cascade = {CascadeType.PERSIST, CascadeType.MERGE},
mappedBy = "heads"
)
@Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE)
private Set databases = new HashSet<>();

@ToString.Exclude
@ManyToMany(
fetch = FetchType.EAGER,
cascade = {CascadeType.PERSIST, CascadeType.MERGE},
mappedBy = "heads"
)
@Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE)
private Set tags = new HashSet<>();

@ToString.Exclude
@ManyToMany(
fetch = FetchType.EAGER,
cascade = {CascadeType.PERSIST, CascadeType.MERGE},
mappedBy = "heads"
)
@Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE)
private Set categories = new HashSet<>();

@ToString.Exclude
@ManyToMany(
fetch = FetchType.LAZY,
cascade = {CascadeType.PERSIST, CascadeType.MERGE},
mappedBy = "heads"
)
@Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE)
private Set searches = new HashSet<>();

public Set getDatabases() {
return Collections.unmodifiableSet(databases);
}

public Set getTags() {
return Collections.unmodifiableSet(tags);
}

public Set getCategories() {
return Collections.unmodifiableSet(categories);
}

public Set getSearches() {
return Collections.unmodifiableSet(searches);
}
} {code}
Note the  {{{}unique = true{}}}, {{nullable = false}} (and the 
{{{}heads_headowner_index{}}})

I have created a couple of functions using a CriteriaBuilder:


{code:java}
public List findAllHeadOwnersByHeadOwnerIn(Collection 
headOwners) {
final var resultList = new ArrayList();
final var headOwnerGroups = CollectionUtils.partitionCollection(headOwners, 
configProperties.getDatabase().getChunkSize());
for (final var headOwnerGroup : headOwnerGroups) {
final var query = querySelectionByCondition(HeadEntity.class,
root -> root.get("headOwner"), String.class,
(criteriaBuilder, root) -> 
root.get("headOwner").in(headOwnerGroup)
);
resultList.addAll(query.getResultList());
}
return resultList;
}

public  TypedQuery querySelectionByCondition(
Class entityType,
Function, Path> selection,
Class selectPropertyType,
BiFunction, Predicate> whereCondition
) {
final var criteriaBuilder = entityManager.getCriteriaBuilder();

final var criteriaQuery = criteriaBuilder
.createQuery(selectPropertyType);

final var root = criteriaQuery.from(entityType);


criteriaQuery.select(selection.apply(root)).where(whereCondition.apply(criteriaBuilder,
 root));

return entityManager.createQuery(criteriaQuery);
}

public  Optional getSingleResult(TypedQuery quer

[jira] [Commented] (OPENJPA-2642) Entity Loading issue in open-jpa

2024-02-28 Thread Jira



[ 
https://issues.apache.org/jira/browse/OPENJPA-2642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821581#comment-17821581
 ] 

Jan Würthner commented on OPENJPA-2642:
---

Hi Senthil! I encountered the very same issue, running the runtime enhancer in 
tomee-plus 7.1.1 with open-jdk 17: In some cases suddenly one of the entities 
is not available, while all others are loaded and available (same wording for 
the error: "...  is not a recognized entity or identifier." and the list of 
entities is not empty ). Have you been able to solve this or find a workaround? 
Unfortunately there is not much on the internet about this issue. Any insight 
is highly appreciated!

> Entity Loading issue in open-jpa
> 
>
> Key: OPENJPA-2642
> URL: https://issues.apache.org/jira/browse/OPENJPA-2642
> Project: OpenJPA
>  Issue Type: Bug
>Reporter: Senthil Kumar Sekar
>Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Hello Team,
>   We have been openjpa – 2.4.0 for our application data access layer 
> which will be deployed in Jboss Fuse 6.2.1.
> We have persistence xml where all the entities are declared.
> Sometimes(These days its very frequent) that we are facing the below issue. 
> This really affects our build and deployment model.
> java.lang.reflect.UndeclaredThrowableException
> at com.sun.proxy.$Proxy29.createQuery(Unknown Source)
> at 
> com.mysema.query.jpa.impl.AbstractJPAQuery.createQuery(AbstractJPAQuery.java:127)[744:com.mysema.querydsl.jpa:3.6.3]
>   ….
> Caused by: java.lang.reflect.UndeclaredThrowableException
> at com.sun.proxy.$Proxy103.createQuery(Unknown Source)
> ... 67 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedMethodAccessor112.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_79]
> at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_79]
> at 
> org.apache.aries.jpa.container.impl.EntityManagerProxyFactory$EMHandler.invoke(EntityManagerProxyFactory.java:31)
> ... 68 more
> Caused by:  
> org.apache.openjpa.persistence.ArgumentException: An error occurred while 
> parsing the query filter "select cETAssignmentException_
> from CETAssignmentException cETAssignmentException_
> where lower(cETAssignmentException_.id.officeCode) = ?1 and 
> lower(cETAssignmentException_.id.businessSegmentCode) = ?2". Error message: 
> The name "CETAssignmentException" is not a recognized entity or identifier. 
> Perhaps you meant CDDCaseLocation, which is a close match. Known entity 
> names: [IndividualReferencesAsu, RmaExcllist, WbSanctionsQuestionnaire, 
> IndividualAsu, SpeDocumentsChecklist, Property, MvSciClientGrpInfo, 
> CddAdditionalDetails, CopiFatcaSubownersInfo, DocumentTemplate, 
> ClientBranchOffices, RoccoDecisionReportPK, CddCaseActorsPK, DdCase, 
> ClientIndividualReln, EcddSciCasePublish, ForeignExchange, CaseDetail, 
> CopiFatcaTaxInfo, NorkomTxn, CopiFatcaDocument, ClientGroup, 
> ClientIndividual, OpaScreen, Country, MvSciClientInfo, IndividualNorkom, 
> CddRisks, IndividualNorkomAsu, ClientOfficeProducts, CddCaseAuditPK, 
> AtfFieldsMaster, CddDdrBasedRouting, PreScreening, CddCasePK, 
> MvSciClientOrgInfo, WorkflowState, RoccoRequest, RoccoRequestAuditPK, Risks, 
> MvSciClientAddrInfo, GaProductTransaction, ClientDocuments, Bba, 
> ClientBranch, ClientNorkomPK, ClientISIC, MvSciArmCodeInfo, DDCCountries, 
> Client, GroupNorkom, IndividualClientReln, Ga, OPARuleBases, 
> CDDWorkflowState, SubReasonCodes, MoneyMarket, OpaError, DefenceGoodsStatus, 
> CDDStateMachine, LegalConstitution, ReasonCodes, RoccoRequestAudit, Address, 
> BranchNorkom, FiDocumentsChecklist, CopiFatcaStatusInfo, CddCase, Cdd, 
> CddPepSummary, Bonds, BbaDocumentsChecklist, ClientOffice, CDDCaseLocation, 
> MvSciEmpInfo, GaDocumentChecklist, TaskBasedReasonCodes, BranchAddress, 
> DgStatusDecisionMatrix, CashEquities, IndividualReferences, EcddSciPublish, 
> CddCaseAudit, BusinessSubSegment, OPACases, ClientIndividualRelnPK, 
> IndividualReferencesAsuPK, Product, ClientNorkom, BusinessSegment, 
> DocumentCategory, CoDocumentsChecklist, CddCaseAuditSubReasons, 
> IndividualClientRelnAsu, TaskBasedReasonCodesPK, CustomerCountryRisk, 
> PreScreeningComment, BbaRiskQuestion, IndividualDocuments, FormButton, 
> Documents, BbaAdditionalDetails, ClientType, GuarantorNorkomPK, 
> AnticipatedTransaction, CddPepDetail, GoodsType, CddCaseActors, 
> GuarantorNorkom, CddCountryRisk, CountryOff

[jira] [Comment Edited] (OPENJPA-2919) Connection pool can be exhausted when connections are killed on the DB side

2024-02-23 Thread Romain Manni-Bucau (Jira)



[ 
https://issues.apache.org/jira/browse/OPENJPA-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820042#comment-17820042
 ] 

Romain Manni-Bucau edited comment on OPENJPA-2919 at 2/23/24 12:26 PM:
---

I don't use dbcp since some years but using tomcat-dbcp the keys are 1. enforce 
a global default query/statement timeout (validation + runtime) + 2. ensure 
validation runs often enough for your infra (can be from every 10mn to every 
30s depending the perfs and instability).


was (Author: romain.manni-bucau):
I don't use dbcp since some years but using tomcat-dbcp the keys are 1. enforce 
a global default query/statement timeout (validation + runtime) + ensure 
validation runs often enough for your infra (can be from every 10mn to every 
30s depending the perfs and instability).

> Connection pool can be exhausted when connections are killed on the DB side
> ---
>
> Key: OPENJPA-2919
> URL: https://issues.apache.org/jira/browse/OPENJPA-2919
> Project: OpenJPA
>  Issue Type: Bug
>  Components: jdbc
>Affects Versions: 3.2.2
>Reporter: Dénes Bodó
>Priority: Critical
>  Labels: deadlock, robustness
> Attachments: Screenshot 2024-02-23 at 09.02.19.png
>
>
> Apache Oozie 5.2.1 uses OpenJPA 2.4.2 and commons-dbcp 1.4 and commons-pool 
> 1.5.4. These are ancient versions, I know.
> h1. Description
> The issue is that when due to some network issues or "maintenance work" on 
> the DB side (especially PostgreSQL) which causes the DB connection to be 
> closed, it results exhausted Pool on the client side. Many threads are 
> waiting at this point:
> {noformat}
> "pool-2-thread-4" #20 prio=5 os_prio=31 tid=0x7faf7903b800 nid=0x8603 
> waiting on condition [0x00030f3e7000]
>java.lang.Thread.State: WAITING (parking)
>   at sun.misc.Unsafe.park(Native Method)
>   - parking to wait for  <0x00066aca8e70> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>   at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>   at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>   at 
> org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:1324)
>  {noformat}
> According to my observation this is because the JDBC driver does not get 
> closed on the client side, nor the abstract DBCP connection 
> _org.apache.commons.dbcp2.PoolableConnection_ .
> h1. Repro
> (Un)Fortunately I can reproduce the issue using the latest and greatest 
> commons-dbcp 2.11.0 and commons-pool 2.12.0 along with OpenJPA 3.2.2.
> I've just created a Java application to reproduce the issue: 
> [https://github.com/dionusos/pool_exhausted_repro] . See README.md for 
> detailed repro steps.
> h1. What we tried so far
> I got in touch with DBCP team who confirmed that in case of an error in the 
> connection the client (in this case OpenJPA is the client of DBCP) should 
> handle the exception like closing the connection: DBCP-595. I agree with them 
> as based on the investigation I did I can also confirm that DBCP is really 
> robust when the client releases the broken connection object after catching 
> SQLException. Please check the 4 comments on DBCP-595 for extra details.
> h1. Ask
> OpenJPA team!
>  * Could you please confirm that my findings are valid?
>  * Did I do anything wrong in my repro program?
>  * Oozie has retry logic implemented: 
> [https://github.com/apache/oozie/blob/318fac5/core/src/main/java/org/apache/oozie/service/JPAService.java#L397L427]
>  but this cannot avoid the reported dead lock.
>  * Do you have any questions I can answer to help in the investigation?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (OPENJPA-2919) Connection pool can be exhausted when connections are killed on the DB side

2024-02-23 Thread Romain Manni-Bucau (Jira)



[ 
https://issues.apache.org/jira/browse/OPENJPA-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820042#comment-17820042
 ] 

Romain Manni-Bucau commented on OPENJPA-2919:
-

I don't use dbcp since some years but using tomcat-dbcp the keys are 1. enforce 
a global default query/statement timeout (validation + runtime) + ensure 
validation runs often enough for your infra (can be from every 10mn to every 
30s depending the perfs and instability).

> Connection pool can be exhausted when connections are killed on the DB side
> ---
>
> Key: OPENJPA-2919
> URL: https://issues.apache.org/jira/browse/OPENJPA-2919
> Project: OpenJPA
>  Issue Type: Bug
>  Components: jdbc
>Affects Versions: 3.2.2
>Reporter: Dénes Bodó
>Priority: Critical
>  Labels: deadlock, robustness
> Attachments: Screenshot 2024-02-23 at 09.02.19.png
>
>
> Apache Oozie 5.2.1 uses OpenJPA 2.4.2 and commons-dbcp 1.4 and commons-pool 
> 1.5.4. These are ancient versions, I know.
> h1. Description
> The issue is that when due to some network issues or "maintenance work" on 
> the DB side (especially PostgreSQL) which causes the DB connection to be 
> closed, it results exhausted Pool on the client side. Many threads are 
> waiting at this point:
> {noformat}
> "pool-2-thread-4" #20 prio=5 os_prio=31 tid=0x7faf7903b800 nid=0x8603 
> waiting on condition [0x00030f3e7000]
>java.lang.Thread.State: WAITING (parking)
>   at sun.misc.Unsafe.park(Native Method)
>   - parking to wait for  <0x00066aca8e70> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>   at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>   at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>   at 
> org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:1324)
>  {noformat}
> According to my observation this is because the JDBC driver does not get 
> closed on the client side, nor the abstract DBCP connection 
> _org.apache.commons.dbcp2.PoolableConnection_ .
> h1. Repro
> (Un)Fortunately I can reproduce the issue using the latest and greatest 
> commons-dbcp 2.11.0 and commons-pool 2.12.0 along with OpenJPA 3.2.2.
> I've just created a Java application to reproduce the issue: 
> [https://github.com/dionusos/pool_exhausted_repro] . See README.md for 
> detailed repro steps.
> h1. What we tried so far
> I got in touch with DBCP team who confirmed that in case of an error in the 
> connection the client (in this case OpenJPA is the client of DBCP) should 
> handle the exception like closing the connection: DBCP-595. I agree with them 
> as based on the investigation I did I can also confirm that DBCP is really 
> robust when the client releases the broken connection object after catching 
> SQLException. Please check the 4 comments on DBCP-595 for extra details.
> h1. Ask
> OpenJPA team!
>  * Could you please confirm that my findings are valid?
>  * Did I do anything wrong in my repro program?
>  * Oozie has retry logic implemented: 
> [https://github.com/apache/oozie/blob/318fac5/core/src/main/java/org/apache/oozie/service/JPAService.java#L397L427]
>  but this cannot avoid the reported dead lock.
>  * Do you have any questions I can answer to help in the investigation?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (OPENJPA-2919) Connection pool can be exhausted when connections are killed on the DB side

2024-02-23 Thread Jira



[ 
https://issues.apache.org/jira/browse/OPENJPA-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820029#comment-17820029
 ] 

Dénes Bodó commented on OPENJPA-2919:
-

Thank you [~romain.manni-bucau] .
{quote}ensure to have some test and a proper validation query 
{quote}
If you are talking about this configuration
{code:java}
connectionProperties.append("MaxActive=").append(1).append(",");
connectionProperties.append("MaxTotal=").append(2).append(",");
connectionProperties.append("MinIdle=").append(1).append(",");
connectionProperties.append("TestOnBorrow=").append("true").append(",");
connectionProperties.append("TestOnReturn=").append("true").append(",");
connectionProperties.append("TestWhileIdle=").append("true").append(",");
connectionProperties.append("fastFailValidation=").append("true").append(",");
connectionProperties.append("ValidationQuery=").append("SELECT 1").append(","); 
{code}
it does not matter regarding the repro if we turn validation on or off. With 
validation enabled the repro time is around 1 minute, without validation it is 
a couple of seconds.

 

Unfortunately the connections which are closed on the DB side are not closed on 
the client side so they stick in the pool but nobody can use them.

!Screenshot 2024-02-23 at 09.02.19.png|width=778,height=279!

 

The only hack-y way I found to close the stuck connection is
{code:java}
((EntityManagerImpl) em).getBroker().getStoreManager().close(); {code}
which will free() the connection which was in use when the Exception happened. 
Otherwise this connection stays in the pool occupying a slot for ever.

 

As you say this is a configuration issue, could you please suggest a 
description how it should be configured? Going through the official OpenJPA 
docs wasn't helping me in this case.

Thank you

> Connection pool can be exhausted when connections are killed on the DB side
> ---
>
> Key: OPENJPA-2919
> URL: https://issues.apache.org/jira/browse/OPENJPA-2919
> Project: OpenJPA
>  Issue Type: Bug
>  Components: jdbc
>Affects Versions: 3.2.2
>Reporter: Dénes Bodó
>Priority: Critical
>  Labels: deadlock, robustness
> Attachments: Screenshot 2024-02-23 at 09.02.19.png
>
>
> Apache Oozie 5.2.1 uses OpenJPA 2.4.2 and commons-dbcp 1.4 and commons-pool 
> 1.5.4. These are ancient versions, I know.
> h1. Description
> The issue is that when due to some network issues or "maintenance work" on 
> the DB side (especially PostgreSQL) which causes the DB connection to be 
> closed, it results exhausted Pool on the client side. Many threads are 
> waiting at this point:
> {noformat}
> "pool-2-thread-4" #20 prio=5 os_prio=31 tid=0x7faf7903b800 nid=0x8603 
> waiting on condition [0x00030f3e7000]
>java.lang.Thread.State: WAITING (parking)
>   at sun.misc.Unsafe.park(Native Method)
>   - parking to wait for  <0x00066aca8e70> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>   at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>   at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>   at 
> org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:1324)
>  {noformat}
> According to my observation this is because the JDBC driver does not get 
> closed on the client side, nor the abstract DBCP connection 
> _org.apache.commons.dbcp2.PoolableConnection_ .
> h1. Repro
> (Un)Fortunately I can reproduce the issue using the latest and greatest 
> commons-dbcp 2.11.0 and commons-pool 2.12.0 along with OpenJPA 3.2.2.
> I've just created a Java application to reproduce the issue: 
> [https://github.com/dionusos/pool_exhausted_repro] . See README.md for 
> detailed repro steps.
> h1. What we tried so far
> I got in touch with DBCP team who confirmed that in case of an error in the 
> connection the client (in this case OpenJPA is the client of DBCP) should 
> handle the exception like closing the connection: DBCP-595. I agree with them 
> as based on the investigation I did I can also confirm that DBCP is really 
> robust when the client releases the broken connection object after catching 
> SQLException. Please check the 4 comments on DBCP-595 for extra details.
> h1. Ask
> OpenJPA team!
>  * Could you please confirm that my findings are valid?
>  * Did I do anything wrong in my repro program?
>  * Oozie has retry logic implemented: 
> [https://github.com/apache/oozie/blob/318fac5/core/src/main/java/org/apache/oozie/service/JPAService.java#L397L427]
>  but this cannot avoid the reported dead lock.
>  * Do you have any questions I can answer to help in the investigation?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (OPENJPA-2919) Connection pool can be exhausted when connections are killed on the DB side

2024-02-23 Thread Jira



 [ 
https://issues.apache.org/jira/browse/OPENJPA-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dénes Bodó updated OPENJPA-2919:

Attachment: Screenshot 2024-02-23 at 09.02.19.png

> Connection pool can be exhausted when connections are killed on the DB side
> ---
>
> Key: OPENJPA-2919
> URL: https://issues.apache.org/jira/browse/OPENJPA-2919
> Project: OpenJPA
>  Issue Type: Bug
>  Components: jdbc
>Affects Versions: 3.2.2
>Reporter: Dénes Bodó
>Priority: Critical
>  Labels: deadlock, robustness
> Attachments: Screenshot 2024-02-23 at 09.02.19.png
>
>
> Apache Oozie 5.2.1 uses OpenJPA 2.4.2 and commons-dbcp 1.4 and commons-pool 
> 1.5.4. These are ancient versions, I know.
> h1. Description
> The issue is that when due to some network issues or "maintenance work" on 
> the DB side (especially PostgreSQL) which causes the DB connection to be 
> closed, it results exhausted Pool on the client side. Many threads are 
> waiting at this point:
> {noformat}
> "pool-2-thread-4" #20 prio=5 os_prio=31 tid=0x7faf7903b800 nid=0x8603 
> waiting on condition [0x00030f3e7000]
>java.lang.Thread.State: WAITING (parking)
>   at sun.misc.Unsafe.park(Native Method)
>   - parking to wait for  <0x00066aca8e70> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>   at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>   at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>   at 
> org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:1324)
>  {noformat}
> According to my observation this is because the JDBC driver does not get 
> closed on the client side, nor the abstract DBCP connection 
> _org.apache.commons.dbcp2.PoolableConnection_ .
> h1. Repro
> (Un)Fortunately I can reproduce the issue using the latest and greatest 
> commons-dbcp 2.11.0 and commons-pool 2.12.0 along with OpenJPA 3.2.2.
> I've just created a Java application to reproduce the issue: 
> [https://github.com/dionusos/pool_exhausted_repro] . See README.md for 
> detailed repro steps.
> h1. What we tried so far
> I got in touch with DBCP team who confirmed that in case of an error in the 
> connection the client (in this case OpenJPA is the client of DBCP) should 
> handle the exception like closing the connection: DBCP-595. I agree with them 
> as based on the investigation I did I can also confirm that DBCP is really 
> robust when the client releases the broken connection object after catching 
> SQLException. Please check the 4 comments on DBCP-595 for extra details.
> h1. Ask
> OpenJPA team!
>  * Could you please confirm that my findings are valid?
>  * Did I do anything wrong in my repro program?
>  * Oozie has retry logic implemented: 
> [https://github.com/apache/oozie/blob/318fac5/core/src/main/java/org/apache/oozie/service/JPAService.java#L397L427]
>  but this cannot avoid the reported dead lock.
>  * Do you have any questions I can answer to help in the investigation?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (OPENJPA-2919) Connection pool can be exhausted when connections are killed on the DB side

2024-02-22 Thread Romain Manni-Bucau (Jira)



[ 
https://issues.apache.org/jira/browse/OPENJPA-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819534#comment-17819534
 ] 

Romain Manni-Bucau commented on OPENJPA-2919:
-

Hi [~dionusos] , seems the pool is not configured, ensure to have some test 
and a proper validation query - or use tomcat-jdbc instead of dbcp which 
supports JDBC isValid() - plus a query timeout for the validation test. This 
will guarantee the connections are dropped properly if your driver is not able 
to detect it.

But long story short this is unrelated to JPA, just a misconfiguration of the 
connections.

> Connection pool can be exhausted when connections are killed on the DB side
> ---
>
> Key: OPENJPA-2919
> URL: https://issues.apache.org/jira/browse/OPENJPA-2919
> Project: OpenJPA
>  Issue Type: Bug
>  Components: jdbc
>Affects Versions: 3.2.2
>Reporter: Dénes Bodó
>Priority: Critical
>  Labels: deadlock, robustness
>
> Apache Oozie 5.2.1 uses OpenJPA 2.4.2 and commons-dbcp 1.4 and commons-pool 
> 1.5.4. These are ancient versions, I know.
> h1. Description
> The issue is that when due to some network issues or "maintenance work" on 
> the DB side (especially PostgreSQL) which causes the DB connection to be 
> closed, it results exhausted Pool on the client side. Many threads are 
> waiting at this point:
> {noformat}
> "pool-2-thread-4" #20 prio=5 os_prio=31 tid=0x7faf7903b800 nid=0x8603 
> waiting on condition [0x00030f3e7000]
>java.lang.Thread.State: WAITING (parking)
>   at sun.misc.Unsafe.park(Native Method)
>   - parking to wait for  <0x00066aca8e70> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>   at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>   at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>   at 
> org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:1324)
>  {noformat}
> According to my observation this is because the JDBC driver does not get 
> closed on the client side, nor the abstract DBCP connection 
> _org.apache.commons.dbcp2.PoolableConnection_ .
> h1. Repro
> (Un)Fortunately I can reproduce the issue using the latest and greatest 
> commons-dbcp 2.11.0 and commons-pool 2.12.0 along with OpenJPA 3.2.2.
> I've just created a Java application to reproduce the issue: 
> [https://github.com/dionusos/pool_exhausted_repro] . See README.md for 
> detailed repro steps.
> h1. What we tried so far
> I got in touch with DBCP team who confirmed that in case of an error in the 
> connection the client (in this case OpenJPA is the client of DBCP) should 
> handle the exception like closing the connection: DBCP-595. I agree with them 
> as based on the investigation I did I can also confirm that DBCP is really 
> robust when the client releases the broken connection object after catching 
> SQLException. Please check the 4 comments on DBCP-595 for extra details.
> h1. Ask
> OpenJPA team!
>  * Could you please confirm that my findings are valid?
>  * Did I do anything wrong in my repro program?
>  * Oozie has retry logic implemented: 
> [https://github.com/apache/oozie/blob/318fac5/core/src/main/java/org/apache/oozie/service/JPAService.java#L397L427]
>  but this cannot avoid the reported dead lock.
>  * Do you have any questions I can answer to help in the investigation?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (OPENJPA-2919) Connection pool can be exhausted when connections are killed on the DB side

2024-02-22 Thread Jira

Dénes Bodó created OPENJPA-2919:
---

Summary: Connection pool can be exhausted when connections are
killed on the DB side
Key: OPENJPA-2919
URL: https://issues.apache.org/jira/browse/OPENJPA-2919
Project: OpenJPA
Issue Type: Bug
Components: jdbc
Affects Versions: 3.2.2
Reporter: Dénes Bodó

Apache Oozie 5.2.1 uses OpenJPA 2.4.2 and commons-dbcp 1.4 and commons-pool
1.5.4. These are ancient versions, I know.
h1. Description

The issue is that when due to some network issues or "maintenance work" on the
DB side (especially PostgreSQL) which causes the DB connection to be closed, it
results exhausted Pool on the client side. Many threads are waiting at this
point:
{noformat}
"pool-2-thread-4" #20 prio=5 os_prio=31 tid=0x7faf7903b800 nid=0x8603
waiting on condition [0x00030f3e7000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00066aca8e70> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:1324)
{noformat}
According to my observation this is because the JDBC driver does not get closed
on the client side, nor the abstract DBCP connection
_org.apache.commons.dbcp2.PoolableConnection_ .
h1. Repro

(Un)Fortunately I can reproduce the issue using the latest and greatest
commons-dbcp 2.11.0 and commons-pool 2.12.0 along with OpenJPA 3.2.2.

I've just created a Java application to reproduce the issue:
[https://github.com/dionusos/pool_exhausted_repro] . See README.md for detailed
repro steps.
h1. What we tried so far

I got in touch with DBCP team who confirmed that in case of an error in the
connection the client (in this case OpenJPA is the client of DBCP) should
handle the exception like closing the connection: DBCP-595. I agree with them
as based on the investigation I did I can also confirm that DBCP is really
robust when the client releases the broken connection object after catching
SQLException. Please check the 4 comments on DBCP-595 for extra details.
h1. Ask

OpenJPA team!
* Could you please confirm that my findings are valid?
* Did I do anything wrong in my repro program?
* Oozie has retry logic implemented:
[https://github.com/apache/oozie/blob/318fac5/core/src/main/java/org/apache/oozie/service/JPAService.java#L397L427]
but this cannot avoid the reported dead lock.
* Do you have any questions I can answer to help in the investigation?

1 2 3 4 5 6 7 8 9 10 >

1 - 100 of 14189 matches

Mail list logo