shashank created CAMEL-25611:
--------------------------------
Summary: camel-kudu - the producer ignores the row error of an
insert, update, delete or upsert (duplicate key, row not found, failed
request): the exchange succeeds although the row was not written; and it never
closes its Kudu sessions
Key: CAMEL-25611
URL: https://issues.apache.org/jira/browse/CAMEL-25611
Project: Camel
Issue Type: Bug
Components: camel-kudu
Reporter: shashank
Assignee: shashank
The four row operations of {{KuduProducer}} end with (lines 106, 127, 147, 167
at main a7169d7c6ab6)
{code:java}
connection.newSession().apply(insert); // delete, update, upsert
{code}
and drop the result. Two defects follow from this line.
h3. 1. A row error is reported as a success
A session of kudu-client has the flush mode {{AUTO_FLUSH_SYNC}} by default (the
constructor of {{AsyncKuduSession}}, 1.18.1). In that mode
{{KuduSession.apply}} sends the operation and returns its
{{OperationResponse}}. The client does not throw in two cases; it returns a
response with a row error ({{hasRowError()}}, {{getRowError()}}):
* a row that the tablet server rejected. Examples: an insert of a key that is
already present ("Already present"), an update or a delete of a row that does
not exist ("Not found").
* a request that failed: read from kudu-client 1.18.1, not run against a Kudu
server. {{AsyncKuduSession.doAutoFlushSync}} adds
{{AsyncKuduSession$SingleOperationErrCallback}} as errback of the write. It
turns every {{KuduException}} of the request (a timeout, a tablet server that
cannot be reached, a row outside the range partitions as "Not found") into an
{{OperationResponse}} whose {{RowError}} has the status of the exception, so
{{KuduSession.apply}} returns normally. After such a failure it is not known
whether the row was written.
The producer does not look at the response, so in both cases the exchange
completes without an exception and the route continues as if the row had been
written. With
{{from("jms:orders").to("kudu:host:7051/orders?operation=update")}} an update
for a row that is not there, or an update that timed out, is acknowledged and
gone.
h3. 2. One session per exchange stays registered with the client
{{AsyncKuduClient.newSession()}} adds every session to its {{sessions}} set;
only {{AsyncKuduSession.close()}} removes it ({{removeSession}}). The producer
never closes the session, so the client keeps one more {{AsyncKuduSession}} for
every exchange until the client itself is closed (when the endpoint stops, or
never for a client that the application provides).
h3. Reproduction
New {{KuduProducerRowErrorTest}} (10 tests). It runs the producer against a
mocked Kudu client (the Kudu harness of the module only runs on Linux x86_64);
the mocked session answers as {{AUTO_FLUSH_SYNC}} does, with a response that
carries a row error. With the producer of main all ten fail:
* {{rowErrorFailsTheExchange}} for insert, delete, update and upsert:
{{exception of an exchange whose insert was answered with a row error ==>
Unexpected null value, expected:
<org.apache.camel.component.kudu.KuduRowErrorException> but was: <null>}};
* {{failedRequestFailsTheExchangeWithItsStatus}} (a delete answered with a row
error whose status is "Timed out", the shape the client gives a failed
request): the same message;
* {{sessionIsClosedAfterAnAppliedRow}} for the four operations: {{Wanted but
not invoked: kuduSession.close()}};
* {{sessionIsClosedWhenTheOperationFails}}: {{array lengths differ, expected:
<1> but was: <0>}} (the failure of the close is not attached, because the
session is not closed after an operation that throws).
h3. Proposed fix
One private method for the four operations: apply the operation, throw the new
{{KuduRowErrorException}} (a {{RuntimeCamelException}} that carries the
{{RowError}} of the Kudu client; message "Kudu reported a row error for the
operation: ...") when the response has a row error, and close the session
afterwards. The status of the row error tells a rejected row from a failed
request. When the operation failed, a failure of the close is added to its
exception as suppressed.
Behaviour change, with a 4.24 upgrade-guide entry and a section in the
component documentation: an insert of an existing key and an update or delete
of a missing row now fail the exchange, and so does a write whose request
failed (timeout, tablet server not reachable). Routes that relied on rejected
rows being skipped silently can use {{upsert}} or handle the exception
({{onException(KuduRowErrorException.class).handled(true)}}).
camel-kudu tests with the fix on macOS arm64: 24 run, 0 failures, 12 skipped
(the 12 tests that need the Kudu harness). The existing harness tests do not
provoke a row error (read, not run). They skip by assumption when the harness
cannot start ({{Assumptions.assumeTrue(ikc.hasKuduHarness(), ...)}} in
{{AbstractKuduTest}}), and the open ticket CAMEL-23493 reports that this is
what happens to most of them, so the mocked test may be the only test that
exercises these producer operations.
How it was found: a sweep of the producers for results of the remote API that
are never inspected, and a small TLA+ model (a table, exchanges with one row
operation each, the sessions of the client). For the code of main TLC reports a
successful exchange whose row was not applied and a session left after the
first exchange; with the fix both properties hold, and no applied row fails an
exchange. The model is only a sketch.
Not verified against a Kudu server: that the tablet server reports a duplicate
key and a missing row as row errors is taken from the code of kudu-client
1.18.1 ({{OperationResponse}}, {{RowError}}, the default flush mode) and from
its documentation, not from a run; that a failed request becomes a row error is
taken from the code of kudu-client 1.18.1 only
({{AsyncKuduSession$SingleOperationErrCallback}}). The kudu-client versions of
the older release branches were not read.
Affected: main (a7169d7c6ab6, tested with the mocked client). camel-4.23.x,
camel-4.22.x, camel-4.18.x and camel-4.14.x have the same four lines (106, 127,
147, 167): read, not run.
Duplicate check (2026-10-13): JIRA text "kudu row error", "kudu insert
duplicate": none; GitHub pull requests with "kudu" in the title: dependency
updates, CAMEL-20401 (null client on endpoint stop) and CAMEL-22241, none about
the result of an operation or the sessions.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
