This is an automated email from the ASF dual-hosted git repository.

davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git


The following commit(s) were added to refs/heads/main by this push:
     new ff82c3dd1bde CAMEL-26999, CAMEL-27000: camel-ftp - a server that stops 
answering no longer holds the stop, the failed poll and the rest of the batch 
for soTimeout (#30087)
ff82c3dd1bde is described below

commit ff82c3dd1bde078905ebbcd5e0a76c4372dda633
Author: Federico Mariani <[email protected]>
AuthorDate: Sun Nov 15 23:03:42 2026 +0100

    CAMEL-26999, CAMEL-27000: camel-ftp - a server that stops answering no 
longer holds the stop, the failed poll and the rest of the batch for soTimeout 
(#30087)
    
    * CAMEL-26999: camel-ftp - a disconnect no longer waits up to soTimeout…
    
    … for the reply to QUIT
    
    FtpOperations.doDisconnect() called FTPClient.logout(), which waits for the
    reply to QUIT up to soTimeout (5 minutes by default), and forceDisconnect()
    called the same method. When the server stopped answering on an open
    connection, the stop of an ftp consumer or producer held CamelContext.stop()
    for soTimeout, and a failed poll waited soTimeout once more before the next
    poll could connect.
    
    The forced disconnect now closes the connection without QUIT, and a 
disconnect
    waits for the reply to QUIT at most 2 seconds (soTimeout when shorter).
    
    * CAMEL-27000: camel-ftp - after a read timeout the consumer leaves the…
    
    … rest of the batch to the next poll
    
    When the server stopped answering while the ftp consumer retrieved a file,
    the file failed after soTimeout (or timeout during the transfer), and the
    consumer went on with the next file of the batch on the same connection:
    each remaining file sent its first command on the dead connection and waited
    soTimeout, and the next poll waited once more on its NOOP. commons-net also
    replaced the socket of the dead connection without closing it.
    
    After a file failed with a SocketTimeoutException, the consumer now closes 
the
    connection without waiting on it and ends the batch; the next poll connects
    again and picks up the remaining files. FtpOperations closes a connection 
that
    is still open before connecting again.
    
    * CAMEL-27000: camel-ftp - with streamDownload, a timeout at the end of…
    
    … the transfer also leaves the rest of the batch to the next poll
    
    With streamDownload=true the route reads the file, and the reply that ends 
the transfer is read by
    releaseRetrievedFileResources() in the commit or rollback of the file, 
whose failure goes to the
    on-completion exception handler, not to the consumer. When that reply timed 
out, the next file of
    the batch sent PASV on the connection that stopped answering, waited 
soTimeout and failed.
    
    FtpOperations.releaseRetrievedFileResources() now closes the connection 
without QUIT when that
    reply times out, and FtpConsumer.isBatchAllowed() ends the batch when the 
connection is closed.
    
    * CAMEL-26999: camel-ftp - upgrade guide: a server slower than 2 s to a…
    
    …nswer QUIT makes the disconnect fail
    
    A producer with disconnect=true or disconnectOnBatchComplete=true logs it 
as a WARN where it used
    to wait for the reply; the consumer and the stop log it at DEBUG.
    
    
    Co-Authored-By: Claude <[email protected]>
---
 .../apache/camel/catalog/docs/ftp-component.adoc   |  15 +
 .../camel-ftp/src/main/docs/ftp-component.adoc     |  15 +
 .../camel/component/file/remote/FtpConsumer.java   |  34 ++
 .../camel/component/file/remote/FtpOperations.java |  57 ++-
 .../camel/component/file/remote/FakeFtpServer.java | 395 +++++++++++++++++++++
 .../FtpConsumerBatchAfterReadTimeoutTest.java      | 182 ++++++++++
 .../remote/FtpDisconnectFromSilentServerTest.java  | 137 +++++++
 .../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc    |  21 ++
 8 files changed, 853 insertions(+), 3 deletions(-)

diff --git 
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ftp-component.adoc
 
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ftp-component.adoc
index 5e51f1f747a1..23a2866e4912 100644
--- 
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ftp-component.adoc
+++ 
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ftp-component.adoc
@@ -253,6 +253,21 @@ It is only a single endpoint that does not support 
concurrent consumers.
 
 The FTP producer does *not* have this issue, it supports concurrency.
 
+=== A server that stops answering
+
+When the FTP/FTPS server stops answering on an open connection (it hangs, or a 
firewall drops the connection without
+closing it), the client notices it only when `soTimeout` expires (or `timeout` 
during a transfer). Camel then does not
+wait on that connection again:
+
+* When the consumer retrieves a file and the server does not answer in time, 
only that file fails: the consumer closes
+  the connection, which it cannot use any more, and leaves the other files of 
the poll to the next poll, which connects
+  again. With `streamDownload=true`, the route reads the file from the server: 
when that read times out, the file fails
+  in the route, and the consumer waits up to `soTimeout` for the reply that 
ends the transfer, then closes the
+  connection in the same way.
+* After a failed poll, the consumer closes the connection without sending 
`QUIT`.
+* When disconnecting, for example when the consumer or the producer stops, the 
client sends `QUIT` and waits at most
+  2 seconds (`soTimeout` when it is shorter) for its reply.
+
 [[FTP-DefaultWhenConsumingFiles]]
 === Default when consuming files
 
diff --git a/components/camel-ftp/src/main/docs/ftp-component.adoc 
b/components/camel-ftp/src/main/docs/ftp-component.adoc
index 5e51f1f747a1..23a2866e4912 100644
--- a/components/camel-ftp/src/main/docs/ftp-component.adoc
+++ b/components/camel-ftp/src/main/docs/ftp-component.adoc
@@ -253,6 +253,21 @@ It is only a single endpoint that does not support 
concurrent consumers.
 
 The FTP producer does *not* have this issue, it supports concurrency.
 
+=== A server that stops answering
+
+When the FTP/FTPS server stops answering on an open connection (it hangs, or a 
firewall drops the connection without
+closing it), the client notices it only when `soTimeout` expires (or `timeout` 
during a transfer). Camel then does not
+wait on that connection again:
+
+* When the consumer retrieves a file and the server does not answer in time, 
only that file fails: the consumer closes
+  the connection, which it cannot use any more, and leaves the other files of 
the poll to the next poll, which connects
+  again. With `streamDownload=true`, the route reads the file from the server: 
when that read times out, the file fails
+  in the route, and the consumer waits up to `soTimeout` for the reply that 
ends the transfer, then closes the
+  connection in the same way.
+* After a failed poll, the consumer closes the connection without sending 
`QUIT`.
+* When disconnecting, for example when the consumer or the producer stops, the 
client sends `QUIT` and waits at most
+  2 seconds (`soTimeout` when it is shorter) for its reply.
+
 [[FTP-DefaultWhenConsumingFiles]]
 === Default when consuming files
 
diff --git 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpConsumer.java
 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpConsumer.java
index 79f8ea5b087c..56ebf5d92ddb 100644
--- 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpConsumer.java
+++ 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpConsumer.java
@@ -16,6 +16,7 @@
  */
 package org.apache.camel.component.file.remote;
 
+import java.net.SocketTimeoutException;
 import java.util.Arrays;
 import java.util.Comparator;
 import java.util.List;
@@ -290,6 +291,39 @@ public class FtpConsumer extends 
RemoteFileConsumer<FTPFile> {
         return false;
     }
 
+    /**
+     * A file failed because the server did not answer in time (soTimeout on 
the control connection, or timeout on the
+     * data connection): the server may no longer answer, and a reply that 
comes late would be read as the reply to the
+     * next command, so the connection cannot be used for the rest of the 
batch. It is closed without waiting on it, and
+     * the rest of the batch is left to the next poll, which connects again. 
Otherwise each remaining file would wait
+     * soTimeout on the connection and fail.
+     */
+    @Override
+    protected void handleException(String message, Exchange exchange, 
Throwable t) {
+        super.handleException(message, exchange, t);
+        if (loggedIn && 
ObjectHelper.getException(SocketTimeoutException.class, t) != null) {
+            LOG.debug("Closing the connection to {} after a timeout, the rest 
of the batch is left to the next poll",
+                    remoteServer());
+            forceDisconnect();
+        }
+    }
+
+    /**
+     * The batch goes on only while the connection is open. It is closed after 
the server did not answer in time: by
+     * {@link #handleException(String, Exchange, Throwable)} when the consumer 
retrieves the file, or by
+     * {@link FtpOperations#releaseRetrievedFileResources(Exchange)} at the 
end of a file streamed to the route
+     * (streamDownload=true). The rest of the batch is then left to the next 
poll, which connects again.
+     */
+    @Override
+    public boolean isBatchAllowed() {
+        if (!loggedIn || !getOperations().isConnected()) {
+            // the rest of the batch is not processed
+            pendingExchanges = 0;
+            return false;
+        }
+        return super.isBatchAllowed();
+    }
+
     @Override
     protected boolean ignoreCannotRetrieveFile(String name, Exchange exchange, 
Exception cause) {
         // the client may still have the reply code 550 of an earlier command 
when the failure is a connection that the
diff --git 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpOperations.java
 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpOperations.java
index 572f426f3a75..fe5f3d9b0e3e 100644
--- 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpOperations.java
+++ 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpOperations.java
@@ -23,8 +23,10 @@ import java.io.FileOutputStream;
 import java.io.IOException;
 import java.io.InputStream;
 import java.io.OutputStream;
+import java.net.SocketTimeoutException;
 import java.time.Duration;
 import java.util.Iterator;
+import java.util.Objects;
 import java.util.concurrent.ScheduledExecutorService;
 
 import org.apache.camel.Exchange;
@@ -56,6 +58,11 @@ import org.slf4j.LoggerFactory;
  */
 public class FtpOperations implements RemoteFileOperations<FTPFile> {
 
+    /**
+     * How long a disconnect waits at most for the reply to QUIT (soTimeout 
when it is shorter).
+     */
+    private static final int QUIT_REPLY_TIMEOUT = 2000;
+
     protected final Logger log = LoggerFactory.getLogger(getClass());
     protected final FTPClient client;
     protected final FTPClientConfig clientConfig;
@@ -118,6 +125,17 @@ public class FtpOperations implements 
RemoteFileOperations<FTPFile> {
         String username = configuration.getUsername();
         String account = ((FtpConfiguration) configuration).getAccount();
 
+        // connecting replaces the socket of the client without closing it: 
close a connection that is still open, such
+        // as one whose server no longer answers, so it is not left open
+        if (client.isConnected()) {
+            log.trace("Client disconnect before connecting again");
+            try {
+                client.disconnect();
+            } catch (IOException e) {
+                log.trace("Ignore exception during disconnect: {}", 
e.getMessage());
+            }
+        }
+
         if (clientConfig != null) {
             log.trace("Configuring FTPClient with config: {}", clientConfig);
             client.configure(clientConfig);
@@ -157,9 +175,10 @@ public class FtpOperations implements 
RemoteFileOperations<FTPFile> {
                     if (payload.exception instanceof 
GenericFileOperationFailedException genericFileOperationFailedException) {
                         throw genericFileOperationFailedException;
                     } else {
+                        // no reply string when the client has no reply, such 
as after it closed the previous connection
                         throw new GenericFileOperationFailedException(
-                                client.getReplyCode(), 
client.getReplyString(), payload.exception.getMessage(),
-                                payload.exception);
+                                client.getReplyCode(), 
Objects.toString(client.getReplyString(), ""),
+                                payload.exception.getMessage(), 
payload.exception);
                     }
                 } else {
                     throw new GenericFileOperationFailedException(
@@ -303,9 +322,20 @@ public class FtpOperations implements 
RemoteFileOperations<FTPFile> {
         }
     }
 
+    /**
+     * Closes the connection without sending QUIT: a forced disconnect 
recovers from a failure, after which the server
+     * may no longer answer, so it must not wait up to soTimeout for the reply.
+     */
     @Override
     public void forceDisconnect() throws GenericFileOperationFailedException {
-        doDisconnect();
+        
clientActivityListener.onDisconnecting(endpoint.getConfiguration().remoteServerInformation());
+        try {
+            log.trace("Client disconnect");
+            client.disconnect();
+        } catch (IOException e) {
+            throw new 
GenericFileOperationFailedException(client.getReplyCode(), 
client.getReplyString(), e.getMessage(), e);
+        }
+        
clientActivityListener.onDisconnected(endpoint.getConfiguration().remoteServerInformation());
     }
 
     protected void doDisconnect() throws GenericFileOperationFailedException {
@@ -313,6 +343,15 @@ public class FtpOperations implements 
RemoteFileOperations<FTPFile> {
         
clientActivityListener.onDisconnecting(endpoint.getConfiguration().remoteServerInformation());
         try {
             log.trace("Client logout");
+            // the connection is closed anyway, so a server that no longer 
answers (it hangs, or the connection was lost
+            // without being closed) must not hold the disconnect, such as the 
stop of the consumer or producer, for
+            // soTimeout while waiting for the reply to QUIT
+            if (client.isConnected()) {
+                int soTimeout = client.getSoTimeout();
+                if (soTimeout <= 0 || soTimeout > QUIT_REPLY_TIMEOUT) {
+                    client.setSoTimeout(QUIT_REPLY_TIMEOUT);
+                }
+            }
             client.logout();
             client.disconnect();
         } catch (IOException e) {
@@ -458,6 +497,18 @@ public class FtpOperations implements 
RemoteFileOperations<FTPFile> {
                     throw new GenericFileOperationFailedException(
                             client.getReplyCode(), client.getReplyString());
                 }
+            } catch (SocketTimeoutException e) {
+                // the server did not answer in time at the end of the 
streamed transfer: a reply that comes late would be
+                // read as the reply to the next command, so the connection 
cannot be used any more. It is closed
+                // without waiting on it, and the consumer leaves the rest of 
the batch to the next poll, which connects
+                // again
+                GenericFileOperationFailedException failure = new 
GenericFileOperationFailedException(e.getMessage(), e);
+                try {
+                    forceDisconnect();
+                } catch (GenericFileOperationFailedException fe) {
+                    failure.addSuppressed(fe);
+                }
+                throw failure;
             } catch (IOException e) {
                 throw new GenericFileOperationFailedException(e.getMessage(), 
e);
             }
diff --git 
a/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FakeFtpServer.java
 
b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FakeFtpServer.java
new file mode 100644
index 000000000000..c84710caac1a
--- /dev/null
+++ 
b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FakeFtpServer.java
@@ -0,0 +1,395 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.file.remote;
+
+import java.io.BufferedReader;
+import java.io.ByteArrayOutputStream;
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.OutputStream;
+import java.net.InetAddress;
+import java.net.ServerSocket;
+import java.net.Socket;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.function.Predicate;
+
+/**
+ * A minimal in-memory FTP server (passive mode only) that can stop answering 
on a connection that stays open, as a
+ * server that hangs, or a network path that is lost without closing the 
connection, does.
+ * <p>
+ * It serves the commands the ftp client sends with the default options and 
{@code passiveMode=true}: login, SYST, TYPE,
+ * PWD, CWD, CDUP, MKD, NOOP, PASV, LIST, NLST, RETR, STOR, DELE, RNFR/RNTO, 
SIZE, MDTM and QUIT. A silent connection
+ * keeps reading and records every command it receives, but never answers 
again. New connections are served normally, so
+ * a client that opens a new connection recovers.
+ */
+final class FakeFtpServer implements Closeable {
+
+    /** How the server stops answering when the trigger command arrives. */
+    enum Stall {
+        /** The trigger command gets no reply: the client waits for the reply 
on the control connection. */
+        BEFORE_REPLY,
+        /**
+         * The trigger RETR gets its 150 reply and half of the file on the 
data connection, then nothing more on either
+         * connection: the client waits on the data connection.
+         */
+        MID_TRANSFER
+    }
+
+    private final ServerSocket serverSocket;
+    private final ExecutorService executor = Executors.newCachedThreadPool(r 
-> {
+        Thread t = new Thread(r, "fake-ftp-server");
+        t.setDaemon(true);
+        return t;
+    });
+    private final Map<String, byte[]> files = new ConcurrentHashMap<>();
+    private final Set<String> directories = ConcurrentHashMap.newKeySet();
+    private final List<Connection> connections = new CopyOnWriteArrayList<>();
+    private final AtomicInteger connectionIds = new AtomicInteger();
+    private volatile Predicate<String> trigger;
+    private volatile Stall stall = Stall.BEFORE_REPLY;
+    private volatile boolean closed;
+
+    FakeFtpServer() throws IOException {
+        serverSocket = new ServerSocket(0, 50, 
InetAddress.getLoopbackAddress());
+        directories.add("/");
+        executor.execute(this::acceptLoop);
+    }
+
+    int port() {
+        return serverSocket.getLocalPort();
+    }
+
+    void addFile(String path, byte[] content) {
+        String file = normalize(path);
+        String parent = parentOf(file);
+        while (!parent.equals("/")) {
+            directories.add(parent);
+            parent = parentOf(parent);
+        }
+        files.put(file, content);
+    }
+
+    Map<String, byte[]> files() {
+        return files;
+    }
+
+    /**
+     * The first connection that receives a command matching {@code command} 
stops answering, from that command on.
+     */
+    void stallOn(Predicate<String> command, Stall how) {
+        this.stall = how;
+        this.trigger = command;
+    }
+
+    /** Every open connection stops answering from now on. New connections are 
served. */
+    void silenceOpenConnections() {
+        connections.forEach(c -> c.silent = true);
+    }
+
+    /** The commands the client sent on a connection after that connection 
stopped answering. */
+    List<String> commandsOnSilentConnections() {
+        List<String> answer = new ArrayList<>();
+        connections.forEach(c -> answer.addAll(c.unanswered));
+        return answer;
+    }
+
+    /** How many control connections the client has opened so far. */
+    int connectionsOpened() {
+        return connectionIds.get();
+    }
+
+    /** How many connections that stopped answering the client has not closed. 
*/
+    long silentConnectionsOpen() {
+        return connections.stream().filter(c -> c.silent && !c.ended).count();
+    }
+
+    @Override
+    public void close() throws IOException {
+        closed = true;
+        serverSocket.close();
+        for (Connection c : connections) {
+            c.close();
+        }
+        executor.shutdownNow();
+    }
+
+    private void acceptLoop() {
+        while (!closed) {
+            try {
+                Socket socket = serverSocket.accept();
+                Connection c = new Connection(socket, 
connectionIds.incrementAndGet());
+                connections.add(c);
+                executor.execute(c);
+            } catch (IOException e) {
+                // closed
+                return;
+            }
+        }
+    }
+
+    private final class Connection implements Runnable {
+
+        private final Socket socket;
+        private final int id;
+        private final List<String> unanswered = new CopyOnWriteArrayList<>();
+        private volatile boolean silent;
+        private volatile boolean ended;
+        private ServerSocket passive;
+        // a data connection left open by a stalled transfer
+        private Socket stalledData;
+        private OutputStream out;
+        private String cwd = "/";
+        private String renameFrom;
+
+        Connection(Socket socket, int id) {
+            this.socket = socket;
+            this.id = id;
+        }
+
+        @Override
+        public void run() {
+            try {
+                BufferedReader in = new BufferedReader(
+                        new InputStreamReader(socket.getInputStream(), 
StandardCharsets.UTF_8));
+                out = socket.getOutputStream();
+                reply("220 fake FTP server ready (connection " + id + ")");
+                String line;
+                while ((line = in.readLine()) != null) {
+                    Predicate<String> t = trigger;
+                    if (!silent && t != null && t.test(line)) {
+                        // one shot: only the first matching connection stalls
+                        trigger = null;
+                        if (stall == Stall.MID_TRANSFER && 
line.startsWith("RETR ")) {
+                            startTransferThenStall(line.substring(5).trim());
+                        }
+                        silent = true;
+                        continue;
+                    }
+                    if (silent) {
+                        unanswered.add(line);
+                        continue;
+                    }
+                    if (!handle(line)) {
+                        break;
+                    }
+                }
+            } catch (IOException e) {
+                // the client or the test closed the connection
+            } finally {
+                ended = true;
+                close();
+            }
+        }
+
+        private boolean handle(String line) throws IOException {
+            int space = line.indexOf(' ');
+            String cmd = (space < 0 ? line : line.substring(0, 
space)).toUpperCase();
+            String arg = space < 0 ? "" : line.substring(space + 1).trim();
+            switch (cmd) {
+                case "USER" -> reply("331 Password required");
+                case "PASS" -> reply("230 Logged in");
+                case "SYST" -> reply("215 UNIX Type: L8");
+                case "TYPE", "MODE", "STRU", "OPTS", "SITE", "NOOP" -> 
reply("200 OK");
+                case "PWD", "XPWD" -> reply("257 \"" + cwd + "\" is the 
current directory");
+                case "CWD" -> {
+                    String dir = resolve(arg);
+                    if (directories.contains(dir)) {
+                        cwd = dir;
+                        reply("250 OK");
+                    } else {
+                        reply("550 No such directory");
+                    }
+                }
+                case "CDUP" -> {
+                    cwd = parentOf(cwd);
+                    reply("250 OK");
+                }
+                case "MKD" -> {
+                    directories.add(resolve(arg));
+                    reply("257 \"" + resolve(arg) + "\" created");
+                }
+                case "PASV" -> {
+                    closePassive();
+                    passive = new ServerSocket(0, 1, 
InetAddress.getLoopbackAddress());
+                    passive.setSoTimeout(10000);
+                    int p = passive.getLocalPort();
+                    reply("227 Entering Passive Mode (127,0,0,1," + (p >> 8) + 
"," + (p & 0xff) + ")");
+                }
+                case "LIST", "NLST" -> list(cmd, arg);
+                case "RETR" -> retrieve(arg);
+                case "STOR" -> store(arg);
+                case "DELE" -> reply(files.remove(resolve(arg)) != null ? "250 
Deleted" : "550 No such file");
+                case "RNFR" -> {
+                    renameFrom = resolve(arg);
+                    reply(files.containsKey(renameFrom) ? "350 Ready for RNTO" 
: "550 No such file");
+                }
+                case "RNTO" -> {
+                    byte[] data = renameFrom != null ? 
files.remove(renameFrom) : null;
+                    if (data != null) {
+                        addFile(resolve(arg), data);
+                        reply("250 Renamed");
+                    } else {
+                        reply("550 No such file");
+                    }
+                }
+                case "SIZE" -> {
+                    byte[] data = files.get(resolve(arg));
+                    reply(data != null ? "213 " + data.length : "550 No such 
file");
+                }
+                case "MDTM" -> reply(files.containsKey(resolve(arg)) ? "213 
20250101000000" : "550 No such file");
+                case "QUIT" -> {
+                    reply("221 Bye");
+                    return false;
+                }
+                default -> reply("502 Command not implemented");
+            }
+            return true;
+        }
+
+        private void list(String cmd, String arg) throws IOException {
+            String dir = cwd;
+            for (String token : arg.split(" ")) {
+                if (!token.isEmpty() && !token.startsWith("-")) {
+                    dir = resolve(token);
+                }
+            }
+            StringBuilder listing = new StringBuilder();
+            for (Map.Entry<String, byte[]> e : new 
TreeMap<>(files).entrySet()) {
+                if (parentOf(e.getKey()).equals(dir)) {
+                    String name = 
e.getKey().substring(e.getKey().lastIndexOf('/') + 1);
+                    if (cmd.equals("NLST")) {
+                        listing.append(name).append("\r\n");
+                    } else {
+                        listing.append("-rw-r--r-- 1 camel camel 
").append(e.getValue().length)
+                                .append(" Jan 01  2025 
").append(name).append("\r\n");
+                    }
+                }
+            }
+            transfer(listing.toString().getBytes(StandardCharsets.UTF_8));
+        }
+
+        private void retrieve(String arg) throws IOException {
+            byte[] data = files.get(resolve(arg));
+            if (data == null) {
+                reply("550 No such file");
+                return;
+            }
+            transfer(data);
+        }
+
+        private void transfer(byte[] data) throws IOException {
+            reply("150 Opening data connection");
+            try (Socket d = passive.accept(); OutputStream o = 
d.getOutputStream()) {
+                o.write(data);
+            } finally {
+                closePassive();
+            }
+            reply("226 Transfer complete");
+        }
+
+        private void startTransferThenStall(String arg) throws IOException {
+            byte[] data = files.get(resolve(arg));
+            reply("150 Opening data connection");
+            stalledData = passive.accept();
+            OutputStream o = stalledData.getOutputStream();
+            o.write(data, 0, data.length / 2);
+            o.flush();
+            // the rest of the file and the 226 reply never come, and the data 
connection stays open
+        }
+
+        private void store(String arg) throws IOException {
+            reply("150 Opening data connection");
+            ByteArrayOutputStream bos = new ByteArrayOutputStream();
+            try (Socket d = passive.accept(); InputStream i = 
d.getInputStream()) {
+                i.transferTo(bos);
+            } finally {
+                closePassive();
+            }
+            addFile(resolve(arg), bos.toByteArray());
+            reply("226 Transfer complete");
+        }
+
+        private String resolve(String path) {
+            if (path.isEmpty()) {
+                return cwd;
+            }
+            return normalize(path.startsWith("/") ? path : cwd + "/" + path);
+        }
+
+        private void reply(String line) throws IOException {
+            out.write((line + "\r\n").getBytes(StandardCharsets.UTF_8));
+            out.flush();
+        }
+
+        private void closePassive() {
+            if (passive != null) {
+                try {
+                    passive.close();
+                } catch (IOException e) {
+                    // ignore
+                }
+                passive = null;
+            }
+        }
+
+        void close() {
+            closePassive();
+            try {
+                if (stalledData != null) {
+                    stalledData.close();
+                }
+                socket.close();
+            } catch (IOException e) {
+                // ignore
+            }
+        }
+    }
+
+    static String normalize(String path) {
+        List<String> parts = new ArrayList<>();
+        for (String p : path.split("/")) {
+            if (p.isEmpty() || p.equals(".")) {
+                continue;
+            }
+            if (p.equals("..")) {
+                if (!parts.isEmpty()) {
+                    parts.remove(parts.size() - 1);
+                }
+            } else {
+                parts.add(p);
+            }
+        }
+        return "/" + String.join("/", parts);
+    }
+
+    static String parentOf(String path) {
+        int i = path.lastIndexOf('/');
+        return i <= 0 ? "/" : path.substring(0, i);
+    }
+}
diff --git 
a/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FtpConsumerBatchAfterReadTimeoutTest.java
 
b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FtpConsumerBatchAfterReadTimeoutTest.java
new file mode 100644
index 000000000000..3b1f975ccbfe
--- /dev/null
+++ 
b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FtpConsumerBatchAfterReadTimeoutTest.java
@@ -0,0 +1,182 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.file.remote;
+
+import java.nio.charset.StandardCharsets;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.camel.Exchange;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.spi.ExceptionHandler;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.EnumSource;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * When the FTP server stops answering on the connection of the consumer in 
the middle of a batch, only the file being
+ * transferred fails: the consumer closes the connection, which cannot be used 
any more, and leaves the rest of the
+ * batch to the next poll, which connects again. Before, every remaining file 
of the batch sent a command on the dead
+ * connection and waited soTimeout for its reply. The same holds when the file 
is streamed to the route
+ * (streamDownload=true).
+ */
+public class FtpConsumerBatchAfterReadTimeoutTest extends CamelTestSupport {
+
+    private static final int FILES = 10;
+
+    private final RecordingExceptionHandler failures = new 
RecordingExceptionHandler();
+    private FakeFtpServer server;
+
+    @BeforeEach
+    void startServer() throws Exception {
+        server = new FakeFtpServer();
+        context.getRegistry().bind("failures", failures);
+    }
+
+    @AfterEach
+    void stopServer() throws Exception {
+        server.close();
+    }
+
+    @ParameterizedTest
+    @EnumSource(FakeFtpServer.Stall.class)
+    void readTimeoutFailsOnlyTheFileBeingTransferred(FakeFtpServer.Stall 
stall) throws Exception {
+        for (int i = 1; i <= FILES; i++) {
+            server.addFile(String.format("/inbox/f-%02d.txt", i), ("content of 
file " + i).getBytes(StandardCharsets.UTF_8));
+        }
+        // the connection stops answering at the 3rd file: before the reply to 
RETR (the client times out on the
+        // control connection), or in the middle of the transfer (the client 
times out on the data connection)
+        server.stallOn(command -> command.equals("RETR f-03.txt"), stall);
+
+        Map<String, String> delivered = new ConcurrentHashMap<>();
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                from("ftp://[email protected]:"; + server.port() + 
"/inbox?password=secret&passiveMode=true"
+                     + 
"&noop=true&sortBy=file:name&initialDelay=0&delay=100&soTimeout=1000&timeout=1000"
+                     + "&exceptionHandler=#failures")
+                        .process(e -> 
delivered.put(e.getMessage().getHeader(Exchange.FILE_NAME, String.class),
+                                e.getMessage().getBody(String.class)));
+            }
+        });
+
+        await().atMost(30, TimeUnit.SECONDS).until(() -> delivered.size() == 
FILES);
+
+        for (int i = 1; i <= FILES; i++) {
+            assertEquals("content of file " + i, 
delivered.get(String.format("f-%02d.txt", i)));
+        }
+        assertEquals(List.of("f-03.txt"), failures.files, "the files that 
failed");
+        // no file of the batch waited soTimeout for a reply on the connection 
that stopped answering
+        assertEquals(List.of(), server.commandsOnSilentConnections(), "the 
commands sent after the stall");
+        await().atMost(5, TimeUnit.SECONDS).until(() -> 
server.silentConnectionsOpen() == 0);
+    }
+
+    @ParameterizedTest
+    @EnumSource(FakeFtpServer.Stall.class)
+    void streamedReadTimeoutFailsOnlyTheFileBeingRead(FakeFtpServer.Stall 
stall) throws Exception {
+        for (int i = 1; i <= FILES; i++) {
+            server.addFile(String.format("/inbox/f-%02d.txt", i), ("content of 
file " + i).getBytes(StandardCharsets.UTF_8));
+        }
+        // with streamDownload the route reads the file from the data 
connection: the connection stops answering at the
+        // 3rd file, before the reply to RETR (the consumer fails to retrieve 
the file), or in the middle of the transfer
+        // (the route fails to read the file, then the consumer waits for the 
reply that ends the transfer)
+        server.stallOn(command -> command.equals("RETR inbox/f-03.txt"), 
stall);
+
+        Map<String, String> delivered = new ConcurrentHashMap<>();
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                onException(Exception.class)
+                        .process(e -> 
failures.files.add(e.getMessage().getHeader(Exchange.FILE_NAME_ONLY, 
String.class)));
+
+                from("ftp://[email protected]:"; + server.port() + 
"/inbox?password=secret&passiveMode=true"
+                     + 
"&noop=true&sortBy=file:name&initialDelay=0&delay=100&soTimeout=1000&timeout=1000"
+                     + 
"&streamDownload=true&stepwise=false&exceptionHandler=#failures")
+                        .process(e -> 
delivered.put(e.getMessage().getHeader(Exchange.FILE_NAME, String.class),
+                                e.getMessage().getBody(String.class)));
+            }
+        });
+
+        await().atMost(30, TimeUnit.SECONDS).until(() -> delivered.size() == 
FILES);
+
+        for (int i = 1; i <= FILES; i++) {
+            assertEquals("content of file " + i, 
delivered.get(String.format("f-%02d.txt", i)));
+        }
+        // in the route or in the consumer
+        assertEquals(List.of("f-03.txt"), failures.files, "the files that 
failed");
+        // the next file of the batch did not wait soTimeout for a reply on 
the connection that stopped answering
+        assertEquals(List.of(), server.commandsOnSilentConnections(), "the 
commands sent after the stall");
+        await().atMost(5, TimeUnit.SECONDS).until(() -> 
server.silentConnectionsOpen() == 0);
+    }
+
+    @Test
+    void connectionThatStoppedAnsweringIsClosedBeforeConnectingAgain() throws 
Exception {
+        server.addFile("/inbox/a.txt", 
"hello".getBytes(StandardCharsets.UTF_8));
+        List<String> received = new CopyOnWriteArrayList<>();
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                from("ftp://[email protected]:"; + server.port() + 
"/inbox?password=secret&passiveMode=true"
+                     + "&noop=true&initialDelay=0&delay=1000&soTimeout=1000")
+                        .process(e -> 
received.add(e.getMessage().getHeader(Exchange.FILE_NAME, String.class)));
+            }
+        });
+        await().atMost(10, TimeUnit.SECONDS).until(() -> received.size() == 1);
+
+        // the connection stops answering between two polls: the NOOP of the 
next poll times out, and the consumer
+        // connects again
+        server.silenceOpenConnections();
+        await().atMost(10, TimeUnit.SECONDS).until(() -> 
server.connectionsOpened() == 2);
+
+        assertTrue(server.commandsOnSilentConnections().contains("NOOP"));
+        // the connection that stopped answering is closed, not left open 
behind the new one
+        await().atMost(5, TimeUnit.SECONDS).until(() -> 
server.silentConnectionsOpen() == 0);
+    }
+
+    /** Records the name of the files whose processing failed. */
+    static final class RecordingExceptionHandler implements ExceptionHandler {
+
+        final List<String> files = new CopyOnWriteArrayList<>();
+
+        @Override
+        public void handleException(Throwable exception) {
+            handleException(null, null, exception);
+        }
+
+        @Override
+        public void handleException(String message, Throwable exception) {
+            handleException(message, null, exception);
+        }
+
+        @Override
+        public void handleException(String message, Exchange exchange, 
Throwable exception) {
+            if (exchange != null) {
+                
files.add(exchange.getMessage().getHeader(Exchange.FILE_NAME_ONLY, 
String.class));
+            }
+        }
+    }
+}
diff --git 
a/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FtpDisconnectFromSilentServerTest.java
 
b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FtpDisconnectFromSilentServerTest.java
new file mode 100644
index 000000000000..8087f1a142e6
--- /dev/null
+++ 
b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FtpDisconnectFromSilentServerTest.java
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.file.remote;
+
+import java.nio.charset.StandardCharsets;
+import java.time.Duration;
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.camel.Exchange;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * A disconnect from an FTP server that stopped answering on an open 
connection (a hung server, or a connection lost
+ * without being closed) does not wait up to soTimeout for the reply to QUIT: 
a stop with nothing in flight returns
+ * within a few seconds, and after a failed poll the forced disconnect does 
not send QUIT.
+ */
+public class FtpDisconnectFromSilentServerTest extends CamelTestSupport {
+
+    private static final int SO_TIMEOUT = 15000;
+    // nothing is in flight: the stop should take the shutdown timeout at 
most, with a margin, not soTimeout
+    private static final Duration STOP_LIMIT = Duration.ofSeconds(10);
+
+    private FakeFtpServer server;
+
+    @BeforeEach
+    void startServer() throws Exception {
+        server = new FakeFtpServer();
+        context.getShutdownStrategy().setTimeout(2);
+        context.getShutdownStrategy().setTimeUnit(TimeUnit.SECONDS);
+    }
+
+    @AfterEach
+    void stopServer() throws Exception {
+        server.close();
+    }
+
+    private String ftpUri(String directory, int soTimeout, String options) {
+        return "ftp://[email protected]:"; + server.port() + "/" + directory + 
"?password=secret&passiveMode=true&soTimeout="
+               + soTimeout + "&" + options;
+    }
+
+    @Test
+    void stopIsNotHeldByAnIdleConsumerWhoseServerStoppedAnswering() throws 
Exception {
+        server.addFile("/inbox/a.txt", 
"hello".getBytes(StandardCharsets.UTF_8));
+        List<String> received = new CopyOnWriteArrayList<>();
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                // one poll at start, then idle: nothing is in flight when the 
server stops answering
+                from(ftpUri("inbox", SO_TIMEOUT, 
"noop=true&initialDelay=0&delay=600000"))
+                        .process(e -> 
received.add(e.getMessage().getHeader(Exchange.FILE_NAME, String.class)));
+            }
+        });
+        await().atMost(10, TimeUnit.SECONDS).until(() -> received.size() == 1);
+
+        server.silenceOpenConnections();
+        Duration took = timedStop();
+
+        assertTrue(took.compareTo(STOP_LIMIT) < 0,
+                "CamelContext.stop() took " + took.toMillis() + " ms with 
soTimeout " + SO_TIMEOUT + " ms");
+        // the consumer still says goodbye, it does not wait long for the reply
+        assertEquals(List.of("QUIT"), server.commandsOnSilentConnections());
+    }
+
+    @Test
+    void stopIsNotHeldByAnIdleProducerWhoseServerStoppedAnswering() throws 
Exception {
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                from("direct:upload").to(ftpUri("outbox", SO_TIMEOUT, 
"disconnect=false"));
+            }
+        });
+        template.sendBodyAndHeader("direct:upload", "hello", 
Exchange.FILE_NAME, "a.txt");
+        assertTrue(server.files().containsKey("/outbox/a.txt"));
+
+        // the producer keeps its connection (disconnect=false)
+        server.silenceOpenConnections();
+        Duration took = timedStop();
+
+        assertTrue(took.compareTo(STOP_LIMIT) < 0,
+                "CamelContext.stop() took " + took.toMillis() + " ms with 
soTimeout " + SO_TIMEOUT + " ms");
+        assertEquals(List.of("QUIT"), server.commandsOnSilentConnections());
+    }
+
+    @Test
+    void failedPollDoesNotSendQuitOnTheConnectionThatStoppedAnswering() throws 
Exception {
+        server.addFile("/inbox/a.txt", 
"hello".getBytes(StandardCharsets.UTF_8));
+        // the LIST of the first poll gets no reply: the poll fails after 
soTimeout, and the poll strategy forces a
+        // disconnect so the next poll connects again
+        server.stallOn(command -> command.startsWith("LIST"), 
FakeFtpServer.Stall.BEFORE_REPLY);
+        List<String> received = new CopyOnWriteArrayList<>();
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                from(ftpUri("inbox", 1000, 
"noop=true&initialDelay=0&delay=100"))
+                        .process(e -> 
received.add(e.getMessage().getHeader(Exchange.FILE_NAME, String.class)));
+            }
+        });
+
+        await().atMost(20, TimeUnit.SECONDS).until(() -> received.size() == 1);
+
+        assertEquals(2, server.connectionsOpened());
+        // the forced disconnect closed the connection without sending QUIT 
and waiting soTimeout for the reply
+        assertEquals(List.of(), server.commandsOnSilentConnections());
+        await().atMost(5, TimeUnit.SECONDS).until(() -> 
server.silentConnectionsOpen() == 0);
+    }
+
+    private Duration timedStop() {
+        long start = System.nanoTime();
+        context.stop();
+        return Duration.ofNanos(System.nanoTime() - start);
+    }
+}
diff --git 
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc 
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index d4c1c02bd737..eb619a3b9c95 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -336,6 +336,27 @@ these messages now reach a started route). The hand-over 
on a graceful stop can
 the delegated consumer takes to stop (a stop that blocks holds back the 
failover until it returns). Exchanges in flight
 that the delegated consumer does not wait for in its stop may still be in 
flight when the leadership moves, as before.
 
+=== camel-ftp - a server that stops answering no longer holds the stop and the 
rest of the batch
+
+The `ftp` and `ftps` components no longer wait up to `soTimeout` (5 minutes by 
default) again and again on a connection
+whose server stopped answering:
+
+* A disconnect, such as when the consumer or the producer stops, still sends 
`QUIT`, but waits at most 2 seconds for
+  its reply (`soTimeout` when it is shorter), then closes the connection. The 
disconnect that the consumer forces after
+  a failed poll closes the connection without sending `QUIT`.
+* A server that takes more than 2 seconds to answer `QUIT` now makes the 
disconnect fail after 2 seconds (the
+  connection is closed all the same). A producer with `disconnect=true` or 
`disconnectOnBatchComplete=true` logs it as
+  a WARN (`Exception occurred during disconnecting from: ... This exception is 
ignored.`) where it used to wait for the
+  reply; the consumer, and the stop of the consumer or the producer, log it at 
DEBUG.
+* When the server does not answer in time while the consumer retrieves a file 
(`soTimeout`, or `timeout` during the
+  transfer), only that file fails: the consumer closes the connection and 
leaves the other files of the poll to the
+  next poll, which connects again. Before, the consumer went on with the next 
file on the same connection, and each
+  remaining file waited `soTimeout` and failed. With `streamDownload=true`, 
when the server does not answer the end of
+  a transfer within `soTimeout` (for example after the route timed out reading 
the file), the consumer closes the
+  connection and leaves the other files of the poll to the next poll in the 
same way.
+
+See the xref:components::ftp-component.adoc[FTP component].
+
 === camel-ftp - sftp checks whether a file exists with one lstat request
 
 The `sftp` component now checks whether a remote file exists 
(`fileExist=Ignore|Fail|Move`, `tempFileName`,

Reply via email to