[ 
https://issues.apache.org/jira/browse/CAMEL-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claus Ibsen resolved CAMEL-27000.
---------------------------------
    Resolution: Fixed

Fixed via https://github.com/apache/camel/pull/30087

> camel-ftp - after a read timeout the consumer keeps using the dead connection 
> for the rest of the batch: every remaining file waits soTimeout and fails
> -------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-27000
>                 URL: https://issues.apache.org/jira/browse/CAMEL-27000
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-ftp
>    Affects Versions: 4.14.9, 4.18.5, 4.23.0
>            Reporter: Federico Mariani
>            Assignee: Federico Mariani
>            Priority: Major
>             Fix For: 4.24.0
>
>
> When the FTP server, or the network path to it, stops answering in the middle 
> of a poll (a hung server, a connection dropped by a firewall without a 
> reset), the command or download in progress fails after {{soTimeout}} 
> (control connection) or {{timeout}} (data connection). The consumer then goes 
> on with the next file of the batch on the same connection:
> * {{GenericFileConsumer.processBatch()}} (camel-file, line 237 at 
> {{camel-4.23.0}}) only ends the batch when the consumer is stopping 
> ({{isBatchAllowed()}}). A failed download does not leave the loop: 
> {{retrieveFileOrAbort()}} (lines 533-566) aborts the file, reports it to the 
> exception handler (line 559), and the loop goes on.
> * commons-net (3.13.0) does not close the connection after a 
> {{SocketTimeoutException}}, and after a failed transfer the reply to RETR is 
> still unread, so the control connection cannot be used any more (a late reply 
> would be read as the reply to the next command). Each next file starts with a 
> command on it: with the default {{stepwise=true}}, the PWD of 
> {{FtpOperations.retrieveFileToStreamInBody()}} (camel-ftp, line 483), which 
> waits {{soTimeout}} for a reply that never comes. The file fails and is 
> reported (a WARN with a stack trace with the default exception handler), and 
> so on for every remaining file.
> * The poll itself succeeds. The next poll's 
> {{RemoteFileConsumer.connectIfNecessary()}} (camel-ftp-common, lines 256-277) 
> sends a NOOP on the same connection (line 260) and waits {{soTimeout}} once 
> more, then connects again (line 272) without closing the old connection: 
> commons-net {{SocketClient._connect()}} replaces the socket without closing 
> it, so the dead control connection stays open until the socket is garbage 
> collected.
> So the consumer recovers after (remaining files + 2) x {{soTimeout}} instead 
> of about one {{soTimeout}}. With the defaults ({{soTimeout}} 300 s), a server 
> that stops answering at the 4th file of a batch of 20 costs 18 x 5 minutes = 
> 1.5 hours and 17 WARN stack traces; the consumer health check stays UP 
> meanwhile, as the poll does not fail.
> On main, CAMEL-25915 sends a NOOP before the next poll after an I/O failure 
> when {{sendNoop=false}}; it does not end the batch, and the result is the 
> same.
> Found with fault-injection tests (a proxy that stops forwarding on an open 
> connection); reproduced with a unit test and an in-process fake FTP server 
> that stops answering on one connection while it serves new ones.
> h3. Steps to reproduce
> 10 files, 
> {{ftp://...?passiveMode=true&noop=true&sortBy=file:name&delay=100&soTimeout=1000&timeout=1000}};
>  the server stops answering at {{RETR f-03.txt}}, either before the reply 
> (the client times out on the control connection) or after it sent half of the 
> file (the client times out on the data connection).
> h3. Expected vs actual (4.23.0 and 4.24.0-SNAPSHOT, soTimeout 1 s)
> || ||Expected||Actual||
> |Files that fail|f-03 only|f-03 to f-10 (8 files, 8 WARN)|
> |Commands sent on the dead connection after the timeout|none|PWD x 7, NOOP|
> |Time from the stall until all 10 files are delivered|about 1 x soTimeout|9.2 
> s = (7 + 2) x soTimeout|
> |Dead control connection|closed|never closed|
> h3. Suggested fix
> After a file failed because the server did not answer in time (a 
> {{SocketTimeoutException}}: {{soTimeout}} on the control connection, 
> {{timeout}} on the data connection), close the connection without waiting on 
> it (without the QUIT of CAMEL-26999) and end the batch: {{processBatch()}} 
> releases the remaining files, as when the consumer stops, and the next poll 
> connects again and picks them up. Close a connection that is still open 
> before connecting again.
> h3. Related
> * CAMEL-26999: disconnecting waits {{soTimeout}} for the reply to QUIT.
> * CAMEL-25915, CAMEL-25896: the NOOP before the next poll.
> * CAMEL-10440 (Won't Fix): errors during a download do not reach the poll 
> strategy.
> * CAMEL-26657: the PWD/CWD per file with {{stepwise=true}}.
> * CAMEL-8718: an earlier connection leak on reconnect (invalid credentials).
> _Claude Code on behalf of Croway_



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to