[
https://issues.apache.org/jira/browse/CAMEL-25611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-25611:
--------------------------------
Fix Version/s: 4.24.0
> camel-kudu - the producer ignores the row error of an insert, update, delete
> or upsert (duplicate key, row not found, failed request): the exchange
> succeeds although the row was not written; and it never closes its Kudu
> sessions
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25611
> URL: https://issues.apache.org/jira/browse/CAMEL-25611
> Project: Camel
> Issue Type: Bug
> Components: camel-kudu
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> The four row operations of {{KuduProducer}} end with (lines 106, 127, 147,
> 167 at main a7169d7c6ab6)
> {code:java}
> connection.newSession().apply(insert); // delete, update, upsert
> {code}
> and drop the result. Two defects follow from this line.
> h3. 1. A row error is reported as a success
> A session of kudu-client has the flush mode {{AUTO_FLUSH_SYNC}} by default
> (the constructor of {{AsyncKuduSession}}, 1.18.1). In that mode
> {{KuduSession.apply}} sends the operation and returns its
> {{OperationResponse}}. The client does not throw in two cases; it returns a
> response with a row error ({{hasRowError()}}, {{getRowError()}}):
> * a row that the tablet server rejected. Examples: an insert of a key that is
> already present ("Already present"), an update or a delete of a row that does
> not exist ("Not found").
> * a request that failed: read from kudu-client 1.18.1, not run against a Kudu
> server. {{AsyncKuduSession.doAutoFlushSync}} adds
> {{AsyncKuduSession$SingleOperationErrCallback}} as errback of the write. It
> turns every {{KuduException}} of the request (a timeout, a tablet server that
> cannot be reached, a row outside the range partitions as "Not found") into an
> {{OperationResponse}} whose {{RowError}} has the status of the exception, so
> {{KuduSession.apply}} returns normally. After such a failure it is not known
> whether the row was written.
> The producer does not look at the response, so in both cases the exchange
> completes without an exception and the route continues as if the row had been
> written. With
> {{from("jms:orders").to("kudu:host:7051/orders?operation=update")}} an update
> for a row that is not there, or an update that timed out, is acknowledged and
> gone.
> h3. 2. One session per exchange stays registered with the client
> {{AsyncKuduClient.newSession()}} adds every session to its {{sessions}} set;
> only {{AsyncKuduSession.close()}} removes it ({{removeSession}}). The
> producer never closes the session, so the client keeps one more
> {{AsyncKuduSession}} for every exchange until the client itself is closed
> (when the endpoint stops, or never for a client that the application
> provides).
> h3. Reproduction
> New {{KuduProducerRowErrorTest}} (10 tests). It runs the producer against a
> mocked Kudu client (the Kudu harness of the module only runs on Linux
> x86_64); the mocked session answers as {{AUTO_FLUSH_SYNC}} does, with a
> response that carries a row error. With the producer of main all ten fail:
> * {{rowErrorFailsTheExchange}} for insert, delete, update and upsert:
> {{exception of an exchange whose insert was answered with a row error ==>
> Unexpected null value, expected:
> <org.apache.camel.component.kudu.KuduRowErrorException> but was: <null>}};
> * {{failedRequestFailsTheExchangeWithItsStatus}} (a delete answered with a
> row error whose status is "Timed out", the shape the client gives a failed
> request): the same message;
> * {{sessionIsClosedAfterAnAppliedRow}} for the four operations: {{Wanted but
> not invoked: kuduSession.close()}};
> * {{sessionIsClosedWhenTheOperationFails}}: {{array lengths differ, expected:
> <1> but was: <0>}} (the failure of the close is not attached, because the
> session is not closed after an operation that throws).
> h3. Proposed fix
> One private method for the four operations: apply the operation, throw the
> new {{KuduRowErrorException}} (a {{RuntimeCamelException}} that carries the
> {{RowError}} of the Kudu client; message "Kudu reported a row error for the
> operation: ...") when the response has a row error, and close the session
> afterwards. The status of the row error tells a rejected row from a failed
> request. When the operation failed, a failure of the close is added to its
> exception as suppressed.
> Behaviour change, with a 4.24 upgrade-guide entry and a section in the
> component documentation: an insert of an existing key and an update or delete
> of a missing row now fail the exchange, and so does a write whose request
> failed (timeout, tablet server not reachable). Routes that relied on rejected
> rows being skipped silently can use {{upsert}} or handle the exception
> ({{onException(KuduRowErrorException.class).handled(true)}}).
> camel-kudu tests with the fix on macOS arm64: 24 run, 0 failures, 12 skipped
> (the 12 tests that need the Kudu harness). The existing harness tests do not
> provoke a row error (read, not run). They skip by assumption when the harness
> cannot start ({{Assumptions.assumeTrue(ikc.hasKuduHarness(), ...)}} in
> {{AbstractKuduTest}}), and the open ticket CAMEL-23493 reports that this is
> what happens to most of them, so the mocked test may be the only test that
> exercises these producer operations.
> How it was found: a sweep of the producers for results of the remote API that
> are never inspected, and a small TLA+ model (a table, exchanges with one row
> operation each, the sessions of the client). For the code of main TLC reports
> a successful exchange whose row was not applied and a session left after the
> first exchange; with the fix both properties hold, and no applied row fails
> an exchange. The model is only a sketch.
> Not verified against a Kudu server: that the tablet server reports a
> duplicate key and a missing row as row errors is taken from the code of
> kudu-client 1.18.1 ({{OperationResponse}}, {{RowError}}, the default flush
> mode) and from its documentation, not from a run; that a failed request
> becomes a row error is taken from the code of kudu-client 1.18.1 only
> ({{AsyncKuduSession$SingleOperationErrCallback}}). The kudu-client versions
> of the older release branches were not read.
> Affected: main (a7169d7c6ab6, tested with the mocked client). camel-4.23.x,
> camel-4.22.x, camel-4.18.x and camel-4.14.x have the same four lines (106,
> 127, 147, 167): read, not run.
> Duplicate check (2026-10-13): JIRA text "kudu row error", "kudu insert
> duplicate": none; GitHub pull requests with "kudu" in the title: dependency
> updates, CAMEL-20401 (null client on endpoint stop) and CAMEL-22241, none
> about the result of an operation or the sessions.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
