Croway commented on PR #30087:
URL: https://github.com/apache/camel/pull/30087#issuecomment-6899401349

   Thanks for the review. Both points are addressed, in two new commits.
   
   **1. `streamDownload=true` (57a00569a916)**: fixed here, in camel-ftp only.
   
   I reproduced it on the fake server: 10 files, `streamDownload=true`, the 
server stops answering in the middle of the transfer of the 3rd file, while the 
route reads the stream. On the previous head it was one file, not each 
remaining one: the route's read timed out (`timeout`), the rollback waited 
`soTimeout` in `completePendingCommand()` and reported only to the 
on-completion exception handler, as you said. Then the 4th file sent PASV on 
the dead connection, waited `soTimeout` and failed. That timeout reached 
`FtpConsumer.handleException()` and ended the batch. So one more file failed, 
after one more `soTimeout` and one more command on the dead connection.
   
   The fix:
   * `FtpOperations.releaseRetrievedFileResources()`: when the reply that ends 
a streamed transfer times out, it now closes the connection without QUIT 
(`forceDisconnect()`), as `FtpConsumer.handleException()` does after a 
timed-out retrieval. A late reply would otherwise be read as the reply to the 
next command.
   * `FtpConsumer.isBatchAllowed()` ends the batch when the connection is 
closed, not only when `loggedIn` is false. The reset of `pendingExchanges` 
moved there from `handleException()`.
   
   No change in camel-file: the timeout still goes only to the on-completion 
exception handler, and the consumer sees that the connection is closed. A side 
effect: the batch also ends when a reconnect inside a commit or rollback 
(delete, rename) fails and leaves the client disconnected. Before, each 
remaining file failed at once with "Connection is not open". Now they are left 
to the next poll.
   
   New test 
`FtpConsumerBatchAfterReadTimeoutTest.streamedReadTimeoutFailsOnlyTheFileBeingRead`,
 for both stall modes:
   * Before the fix, `MID_TRANSFER` fails with `the files that failed ==> 
expected: <[f-03.txt]> but was: <[f-03.txt, f-04.txt]>`. `BEFORE_REPLY` already 
passed, because the timeout happens while the file is retrieved.
   * After the fix, both pass: only f-03 fails, nothing is sent on the dead 
connection, it is closed, and all 10 files are delivered by the next poll.
   
   The section "A server that stops answering" and the upgrade guide now cover 
`streamDownload=true`: the consumer waits up to `soTimeout` for the reply that 
ends the transfer, then closes the connection in the same way.
   
   **2. Upgrade guide (222255bfda51)**: added. I checked it in the code and on 
the fake server (QUIT never answered, `soTimeout=15000`). `doDisconnect()` 
throws `GenericFileOperationFailedException` (`Read timed out`) after 2 s and 
closes the socket.
   * A producer with `disconnect=true` or `disconnectOnBatchComplete=true` logs 
it as a WARN in `RemoteFileProducer.postWriteCheck()`: `Exception occurred 
during disconnecting from: ... This exception is ignored.`
   * The consumer (`RemoteFileConsumer.disconnect()`) and the producer's 
`doStop()` log it at DEBUG.
   
   The entry says this.
   
   All camel-ftp unit tests (198 run, 9 skipped) and ITs (500, 17 skipped) 
pass. The PR description is updated.
   
   _Claude Code on behalf of Croway_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to