[
https://issues.apache.org/jira/browse/CAMEL-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Mariani reassigned CAMEL-26996:
----------------------------------------
Assignee: Federico Mariani
> camel-jdbc / camel-sql - outputType=StreamList: once reading a row has
> failed, the iterator looks finished, so a redelivered split succeeds with
> part of the rows
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26996
> URL: https://issues.apache.org/jira/browse/CAMEL-26996
> Project: Camel
> Issue Type: Bug
> Components: camel-jdbc, camel-sql
> Affects Versions: 4.14.9, 4.18.5, 4.23.0
> Reporter: Federico Mariani
> Assignee: Federico Mariani
> Priority: Major
> Fix For: 4.24.0
>
>
> With {{outputType=StreamList}} the body is a {{ResultSetIterator}} over the
> open {{ResultSet}}. When reading a row fails, {{next()}} closes the iterator
> and throws, and {{hasNext()}} is {{!closed}}, so after the failure the
> iterator says that there are no more rows, as when it has read the last one.
> camel-jdbc
> ({{components/camel-jdbc/src/main/java/org/apache/camel/component/jdbc/ResultSetIterator.java}},
> lines 72-97 on main):
> {code:java}
> public boolean hasNext() {
> return !closed.get();
> }
> ...
> } catch (SQLException e) {
> close();
> throw new RuntimeCamelException("Cannot process result", e);
> }
> {code}
> camel-sql
> ({{components/camel-sql/src/main/java/org/apache/camel/component/sql/ResultSetIterator.java}},
> lines 54-72) has the same code.
> The first pass is right: the split fails with {{RuntimeCamelException: Cannot
> process result}}. But the failure comes from the iteration of the splitter
> itself, not from a sub-exchange, so it is not marked as exhausted
> ({{MulticastProcessor.MulticastTask.doFailed}} calls {{doDone}} with
> {{forceExhaust=false}}), and the error handler of the route redelivers to the
> splitter with the same body: the same iterator, which now has "no more rows".
> The split completes and the exchange succeeds. The rows after the failure are
> lost without an error; the only trace is the DEBUG log of the redelivery.
> A typical cause is a connection lost in the middle of the stream (with a
> fetch size the rows come from a server-side cursor, a batch at a time), but
> any {{SQLException}} while reading a row does it. Found with fault-injection
> tests (Toxiproxy in front of a real PostgreSQL: connection reset, backend
> restart, blackhole).
> h3. Steps to reproduce
> {code:java}
> errorHandler(defaultErrorHandler().maximumRedeliveries(3).redeliveryDelay(0));
> from("direct:stream")
> .setBody(constant("select id from items order by id"))
> .to("jdbc:ds?outputType=StreamList") // or sql:select id from items
> order by id?outputType=StreamList
> .split(body()).streaming()
> .to("mock:row")
> .end();
> {code}
> 30 rows, and a data source whose result set fails to fetch a row in the
> middle of the stream (the database session ends while the route handles row
> 12).
> h3. Actual (4.23.0 and 4.24.0-SNAPSHOT)
> * Without error handler: the exchange fails (correct).
> * With the redelivery policy, camel-jdbc and camel-sql: the exchange
> succeeds, with 12 of the 30 rows routed.
> h3. Expected
> The exchange fails once the redeliveries are exhausted: a redelivery cannot
> read the rest of a result set that has failed, and must not turn the failure
> into a success.
> h3. Suggested fix
> Keep the failure in {{ResultSetIterator}}: once reading a row has failed,
> {{hasNext()}} and {{next()}} throw again (a {{RuntimeCamelException}} with
> the first {{SQLException}} as cause), also after {{close()}}, so whatever
> reads the iterator again gets an error, not an end of data.
> {{IOHelper.closeIterator}} does the same for a {{Scanner}}: it rethrows the
> {{IOException}} the scanner swallowed, so a failed read does not look like
> the end of the input.
> Not redelivering a split over an iterator (in the splitter or the error
> handler) would also change iterators that can go on after a failure; only the
> iterator knows that it cannot.
> h3. Related
> * CAMEL-7447 ({{StreamList}} in camel-jdbc), CAMEL-9849 ({{StreamList}} in
> camel-sql).
> * CAMEL-26997: {{StreamList}} with a fetch size on PostgreSQL (the producer
> commits before the stream is read), which with this issue and a redelivery
> policy truncates the result with no fault at all.
> _Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)