Federico Mariani created CAMEL-26996:
----------------------------------------

             Summary: camel-jdbc / camel-sql - outputType=StreamList: once 
reading a row has failed, the iterator looks finished, so a redelivered split 
succeeds with part of the rows
                 Key: CAMEL-26996
                 URL: https://issues.apache.org/jira/browse/CAMEL-26996
             Project: Camel
          Issue Type: Bug
          Components: camel-jdbc, camel-sql
    Affects Versions: 4.14.9, 4.18.5, 4.23.0
            Reporter: Federico Mariani


With {{outputType=StreamList}} the body is a {{ResultSetIterator}} over the 
open {{ResultSet}}. When reading a row fails, {{next()}} closes the iterator 
and throws, and {{hasNext()}} is {{!closed}}, so after the failure the iterator 
says that there are no more rows, as when it has read the last one.

camel-jdbc 
({{components/camel-jdbc/src/main/java/org/apache/camel/component/jdbc/ResultSetIterator.java}},
 lines 72-97 on main):
{code:java}
public boolean hasNext() {
    return !closed.get();
}
...
} catch (SQLException e) {
    close();
    throw new RuntimeCamelException("Cannot process result", e);
}
{code}
camel-sql 
({{components/camel-sql/src/main/java/org/apache/camel/component/sql/ResultSetIterator.java}},
 lines 54-72) has the same code.

The first pass is right: the split fails with {{RuntimeCamelException: Cannot 
process result}}. But the failure comes from the iteration of the splitter 
itself, not from a sub-exchange, so it is not marked as exhausted 
({{MulticastProcessor.MulticastTask.doFailed}} calls {{doDone}} with 
{{forceExhaust=false}}), and the error handler of the route redelivers to the 
splitter with the same body: the same iterator, which now has "no more rows". 
The split completes and the exchange succeeds. The rows after the failure are 
lost without an error; the only trace is the DEBUG log of the redelivery.

A typical cause is a connection lost in the middle of the stream (with a fetch 
size the rows come from a server-side cursor, a batch at a time), but any 
{{SQLException}} while reading a row does it. Found with fault-injection tests 
(Toxiproxy in front of a real PostgreSQL: connection reset, backend restart, 
blackhole).

h3. Steps to reproduce

{code:java}
errorHandler(defaultErrorHandler().maximumRedeliveries(3).redeliveryDelay(0));

from("direct:stream")
    .setBody(constant("select id from items order by id"))
    .to("jdbc:ds?outputType=StreamList")   // or sql:select id from items order 
by id?outputType=StreamList
    .split(body()).streaming()
        .to("mock:row")
    .end();
{code}

30 rows, and a data source whose result set fails to fetch a row in the middle 
of the stream (the database session ends while the route handles row 12).

h3. Actual (4.23.0 and 4.24.0-SNAPSHOT)

* Without error handler: the exchange fails (correct).
* With the redelivery policy, camel-jdbc and camel-sql: the exchange succeeds, 
with 12 of the 30 rows routed.

h3. Expected

The exchange fails once the redeliveries are exhausted: a redelivery cannot 
read the rest of a result set that has failed, and must not turn the failure 
into a success.

h3. Suggested fix

Keep the failure in {{ResultSetIterator}}: once reading a row has failed, 
{{hasNext()}} and {{next()}} throw again (a {{RuntimeCamelException}} with the 
first {{SQLException}} as cause), also after {{close()}}, so whatever reads the 
iterator again gets an error, not an end of data. {{IOHelper.closeIterator}} 
does the same for a {{Scanner}}: it rethrows the {{IOException}} the scanner 
swallowed, so a failed read does not look like the end of the input.

Not redelivering a split over an iterator (in the splitter or the error 
handler) would also change iterators that can go on after a failure; only the 
iterator knows that it cannot.

h3. Related

* CAMEL-7447 ({{StreamList}} in camel-jdbc), CAMEL-9849 ({{StreamList}} in 
camel-sql).
* The camel-jdbc issue about {{StreamList}} with a fetch size on PostgreSQL 
(the producer commits before the stream is read), which with this issue and a 
redelivery policy truncates the result with no fault at all.

_Claude Code on behalf of Croway_




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to