allthingssecurity opened a new pull request, #29405: URL: https://github.com/apache/camel/pull/29405
# Description [CAMEL-26708](https://issues.apache.org/jira/browse/CAMEL-26708) Two `JdbcKeyValueRepository` instances on one database (two nodes) could both win `putIfAbsent` for the same key, and reading an expired entry could delete the entry another node had just stored for that key. - `putIfAbsent`: when the INSERT failed with a `DuplicateKeyException`, the key was read again inside the same, rollback-only transaction and that value was returned. Under REPEATABLE READ (MySQL/MariaDB default, H2) the snapshot of the first SELECT does not show the other row, and under READ COMMITTED the other caller may have deleted it already (e.g. the Idempotent Consumer's `removeOnFailure`). Either way `null` ("the put succeeded") was returned although the INSERT was rolled back, so `KeyValueIdempotentRepository.add` let the same message through on both nodes. On PostgreSQL the re-read fails, as the transaction is aborted after the error. - Lazy expiry in `doGet`: an expired row was deleted with `DELETE ... WHERE ITEM_KEY = ?`. If another node stored the key between the SELECT and that DELETE, its live entry was deleted. Change: - `putIfAbsent` lets the `DuplicateKeyException` roll back the transaction and reads the key again in a new transaction, which returns the value stored by the other caller, or inserts again if that entry was deleted meanwhile. The primary key decides; no schema change. After 3 attempts the `DuplicateKeyException` is thrown. When `putIfAbsent` joined an existing transaction of the caller (`PROPAGATION_REQUIRED`), it is thrown at once, as that transaction cannot read the key again (before, the caller's transaction was marked rollback-only too and failed at commit). - `doGet` deletes an expired row with a new statement `deleteIfExpiredString`: `DELETE FROM CAMEL_KEYVALUE WHERE ITEM_KEY = ? AND EXPIRES_AT > 0 AND EXPIRES_AT <= ?`, the same expiry condition as `deleteExpiredString` and the existing check in `doGet`, with the same `now`. It is configurable and follows `tableName` like the other statements; generated configurer and bean metadata (and the catalog copy) updated. - The notes on `putIfAbsent` of the JDBC and JPA backends in `keyValueRepository.adoc` (and its catalog copy) are updated, and the 4.24 upgrade guide says that users with their own column names must set `deleteIfExpiredString`, and when `putIfAbsent` now throws. Not in this PR: - `replace` / `delete(key, expected)` still match an expired row; making them expiry-aware changes the parameters of the customizable `updateIfValueString` / `deleteIfValueString`, and no Camel component calls them. Can be a follow-up. - `JpaKeyValueRepository` has the same two patterns (re-read in the failed transaction, unconditional lazy delete); it is a different module and can be a follow-up. - Older errors on the `keyValueRepository.adoc` page are left as they are (the backend table says "✗ (non-atomic)" for JDBC while the CAS table says ✓; the YAML example uses `org.apache.camel.component.sql.JdbcKeyValueRepository` instead of the `org.apache.camel.processor.keyvalue.jdbc` package). Can be a follow-up. - Under snapshot/serializable isolation a concurrent insert can surface as a serialization failure (`ConcurrencyFailureException`) rather than a duplicate key; that is not retried, as before. For `main` only (4.24): it adds an option and `putIfAbsent` can now throw. Tests (`JdbcKeyValueRepositoryConcurrencyTest`, embedded H2, two repositories on one database; node A's `JdbcTemplate` pauses its n-th statement on a latch, so both nodes interleave in a fixed order, no sleeps): - `putIfAbsentHasOneWinnerUnderRepeatableRead`: A reads the key, B stores it, A's INSERT fails; A must get B's value. - `putIfAbsentDoesNotSucceedWithoutStoringTheValue` (READ COMMITTED): as above, then B deletes the key before A reads it again; if A is told it succeeded, its value must be stored. - `readingAnExpiredEntryDoesNotDeleteTheEntryStoredMeanwhile`: A reads an expired row, B stores the key, A's delete must keep B's entry. - `putIfAbsentGivesUpAfterThreeDuplicateKeys` and `putIfAbsentInTransactionOfCallerDoesNotTryAgain`: the bound and the joined-transaction case. On main all 5 fail (the first three return `null` without storing / lose B's entry); with the fix 5/5, and 100/100 over 20 runs (javac against the 4.24.0-SNAPSHOT jars; existing `JdbcKeyValueRepositoryTest` 29/29). camel-sql module tests: 336 run (offline), 0 failures. `SqlFunctionDataSourceTest` was left out: it installs an embedded MariaDB and cannot run offline here (unrelated to this change). # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-26708](https://issues.apache.org/jira/browse/CAMEL-26708). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built the core modules and camel-sql with the formatter and import-sort plugins and ran the camel-sql tests; the build regenerated the configurer and bean metadata of `JdbcKeyValueRepository` with the committed content. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
