Croway opened a new pull request, #30082: URL: https://github.com/apache/camel/pull/30082
[CAMEL-26996](https://issues.apache.org/jira/browse/CAMEL-26996) ### What was wrong With `outputType=StreamList`, when reading a row fails, `ResultSetIterator.next()` closes the iterator and throws, and `hasNext()` is `!closed` (camel-jdbc and camel-sql have the same code). After the failure the iterator therefore says "no more rows", like an iterator that has read the last row. The failure comes from the splitter's own iteration (not from a sub-exchange), so `MulticastTask.doFailed` → `doDone(..., forceExhaust=false)` leaves it not exhausted and the route's error handler redelivers to the splitter, with the same body: the same iterator, which now has no more rows. With a redelivery policy the split completes and the exchange **succeeds with only the rows read before the failure**. Found with fault-injection tests (Toxiproxy in front of PostgreSQL: connection reset, backend restart). ### The fix `ResultSetIterator` (camel-jdbc and camel-sql) remembers the `SQLException`: from then on `hasNext()` (and so `next()`) throws a `RuntimeCamelException` with that `SQLException` as cause, also after `close()` (the splitter closes a `Closeable` iterator body when the split fails). The first failure is unchanged (`Cannot process result`). A redelivered split now fails each time, and the exchange fails once the redeliveries are exhausted. camel-spring-jdbc uses the camel-jdbc iterator, so it gets the fix too. ### Design choices for reviewers - **Fix in the iterator, not in the splitter / error handler.** Not redelivering a split over an iterator would change every iterator body, including ones that can go on after a failure (e.g. a paging iterator that retries the failed page). Only the iterator knows that it cannot be read again. `IOHelper.closeIterator` already does the same for a `Scanner` (rethrows the `IOException` the scanner swallowed, so a failed read does not look like the end of the input). - **`hasNext()` throws** rather than returning `true` and letting `next()` throw: after a failure whether more rows exist is unknown, and a caller that catches the exception of `next()` and loops on `hasNext()` would otherwise spin forever. The splitter treats an exception from `hasNext()` like one from `next()` (both end in `doFailed`). - A redelivery of the split can only fail: it cannot read the stream again. Running the query again needs the redelivery to happen at the producer (e.g. the query and the split in a route called with `to("direct:...")`), which is up to the route. - Not changed: a row whose `outputClass` mapping fails in camel-jdbc (the mapping is in `camel-support`'s `StreamListIterator`, outside the result set iterator); the result set itself is fine there. ### Relation to CAMEL-26991 CAMEL-26991 (camel-sql: `StreamList` takes part in the route transaction, branch not opened as a PR yet) also touches camel-sql's `ResultSetIterator`, but only its imports, fields, constructors and `safeCloseConnection()` (the connection handling). This PR only changes `hasNext()`/`next()` and adds one field after `rowNum`, so the two merge without conflict (checked with `git merge-tree`). CAMEL-26997 (camel-jdbc commits before a `StreamList` is read, which on PostgreSQL with a fetch size fails the stream after the first batch) is a separate PR: with it and this one, that scenario no longer truncates silently. ### Relation to #30060 (docs) #30060 adds a NOTE to `jdbc-component.adoc` and `sql-component.adoc` that describes the current behaviour (a redelivered split over a broken `StreamList` completes with only the rows read before the failure, so do not configure redeliveries for that route). Once this PR is merged that NOTE is no longer true: it must be dropped or reworded (a redelivery of the split now fails too; to read the rows again, run the query again). Whichever of the two PRs merges second adjusts it. ### Tests - `JdbcProducerStreamListReadFailureTest` (camel-jdbc) and `SqlProducerStreamListReadFailureTest` (camel-sql): H2 behind a `DelegatingDataSource` whose result sets fail `next()` (SQLState `08006`, as pgjdbc on a lost connection) once the route has handled the first row. A split with `maximumRedeliveries(2)` must fail with the `SQLException` as cause after 2 redeliveries; the iterator must keep throwing on `hasNext()`/`next()`, also after `close()`. camel-jdbc also keeps the case without an error handler (already failing correctly). - **Fail before the fix**: camel-jdbc 2 of 3 fail (the redelivered exchange succeeds: `expected: <RuntimeCamelException> but was: <null>`; `hasNext()` does not throw), camel-sql 2 of 2 fail. - After the fix: the new tests and the full camel-jdbc (68 tests) and camel-sql (393 tests) suites pass. ### Upgrade guide Yes, 4.24: a `StreamList` that failed to read now keeps failing (a redelivered split fails instead of succeeding with part of the rows; `hasNext()` throws after a read failure). _Claude Code on behalf of Croway_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
