Federico Mariani created CAMEL-26987:
----------------------------------------
Summary: camel-jpa - the producer reuses the EntityManager of an
exchange after its connection broke, so every redelivery fails with "JDBC begin
transaction failed"
Key: CAMEL-26987
URL: https://issues.apache.org/jira/browse/CAMEL-26987
Project: Camel
Issue Type: Bug
Components: camel-jpa
Affects Versions: 4.14.9, 4.18.5, 4.23.0
Reporter: Federico Mariani
The JPA producer keeps one {{EntityManager}} per exchange until the exchange
completes, so every redelivery of an exchange uses the same {{EntityManager}}:
* {{JpaProducer.process}} gets it from {{JpaHelper.getTargetEntityManager}}. It
has no error handling of its own.
* {{JpaHelper.getTargetEntityManager}} reuses the one stored in the
{{CamelEntityManager}} exchange property. It creates a new one only when that
one is closed ({{!em.isOpen()}}, CAMEL-9177).
* {{JpaHelper.createEntityManager}} stores the new one on the exchange. It is
closed only when the exchange completes, by
{{JpaCloseEntityManagerOnCompletion.onDone}}.
When the JDBC connection held by that {{EntityManager}} breaks during a send
(database restart, failover, network drop), this happens with Hibernate:
# The insert fails. Spring then rolls back, and the rollback of the
{{EntityManager}}'s local transaction fails too. Spring logs
{{TransactionSynchronization.afterCompletion threw exception ... Unable to
rollback against JDBC Connection}} at ERROR.
# Hibernate keeps the broken connection. When the rollback fails, it skips its
after-completion step, which is what releases the connection
({{AbstractLogicalConnectionImplementor.rollback}},
{{JdbcResourceLocalTransactionCoordinatorImpl.TransactionDriverControlImpl.rollback}}).
This holds for both connection handling modes:
{{DELAYED_ACQUISITION_AND_HOLD}} (set by Spring's
{{HibernateJpaVendorAdapter}}, as in Spring Boot) and Hibernate's default,
which releases the connection after the transaction.
# Every redelivery calls {{joinTransaction()}} -> {{EntityTransaction.begin()}}
on that same broken connection and fails with
{{org.hibernate.TransactionException: JDBC begin transaction failed}}. This
happens even though the database and the pool are fine again. Once redeliveries
are exhausted, the exchange fails (or goes to the dead letter channel).
The consumer had the same problem (CAMEL-11197). It was fixed by closing the
consumer's {{EntityManager}} when a poll fails ({{JpaConsumer.poll}}, which
cites the JPA rule that the persistence context is undefined after a rollback).
{{JpaMessageIdRepository}} closes its {{EntityManager}} after each call. Only
the producer keeps it after a failure.
h3. Steps to reproduce
* Hibernate on H2.
* A {{DataSource}} that closes every open connection while the first insert is
sent, like a database restart. New connections work right away.
* The route {{direct:x -> jpa:SomeEntity}}, with
{{maximumRedeliveries(3).redeliveryDelay(0)}}.
*Expected:* the first redelivery uses a new {{EntityManager}}, and the entity
is persisted.
*Actual:* all 3 redeliveries fail:
{noformat}
the exchange failed after 3 redeliveries with: TransactionException: JDBC begin
transaction failed:
<- JdbcSQLNonTransientException: The object is already closed [90007-252]
{noformat}
The result is the same with {{HibernateJpaVendorAdapter}} and with Hibernate's
defaults. {{sharedEntityManager=true}} recovers, because Spring's shared
{{EntityManager}} uses a new {{EntityManager}} per transaction. So does closing
the exchange's {{EntityManager}}s in an {{onRedelivery}} processor. Both are
workarounds.
h3. Suggested fix
In {{JpaProducer.process}}, when the find, query or entity processing throws,
close the {{EntityManager}} if camel-jpa created it for this exchange (it is a
value of the exchange's {{CamelEntityManager}} property map), and ignore
exceptions from {{close()}}. On the redelivery, the existing {{!em.isOpen()}}
check in {{getTargetEntityManager}} then creates a new one. This is the same
idea as the {{JpaConsumer}} fix.
* An {{EntityManager}} passed in the {{CamelEntityManager}} header
({{usePassedInEntityManager}}) is never stored in that map, so the user's
{{EntityManager}} is not touched.
* Spring's shared {{EntityManager}} ({{sharedEntityManager}}) is not stored in
the map either, and closing it is a no-op anyway.
* If the {{EntityManager}} is joined to an outer transaction (transacted
route), {{close()}} is deferred until that transaction completes. The JPA spec
requires this, and so does Spring's application-managed {{EntityManager}}
proxy. The transaction is not cut short.
Side effect: entities managed by that {{EntityManager}} in an earlier JPA step
of the same exchange are detached after a failure in the producer. The JPA spec
already leaves them undefined after a rollback ("Transaction Rollback" in the
persistence context lifetime chapter).
Minor, while there: in {{JpaHelper.getTargetEntityManager}}, {{if
(allowRecreate && em == null || !em.isOpen())}} reads as {{(allowRecreate && em
== null) || !em.isOpen()}}. It is harmless, because {{em}} cannot be null
there, but it is misleading.
h3. Related
* CAMEL-11197: camel-jpa consumer fails to poll after database connection is
lost (the consumer counterpart)
* CAMEL-9177: combination of JPA-Component, loop and wiretap throws
entitymanger cloesd exception (added the {{!em.isOpen()}} recreate path the fix
relies on)
* CAMEL-16470 and CAMEL-18377: reuse of the exchange's {{EntityManager}} across
endpoints and within a transaction (the fix must keep both)
_Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)