This is an automated email from the ASF dual-hosted git repository. gnodet pushed a commit to branch backport/28214-to-camel-4.22.x in repository https://gitbox.apache.org/repos/asf/camel.git
commit 31b39827930f8027f6672bfa5c79051d296fc2af Author: Guillaume Nodet <[email protected]> AuthorDate: Thu Oct 29 14:17:23 2026 +0000 [backport camel-4.22.x] CAMEL-25886: camel-ftp/mina-sftp - resolve listed symbolic links for jailStartingDirectory --- .../camel/catalog/docs/mina-sftp-component.adoc | 12 +- .../apache/camel/catalog/docs/sftp-component.adoc | 15 ++ .../camel/component/file/GenericFileConsumer.java | 34 ++- .../file/remote/AbstractSftpConsumer.java | 138 ++++++++++ .../component/file/remote/SftpRemoteFile.java | 7 + .../file/remote/SftpSymbolicLinkOperations.java | 53 ++++ .../camel-ftp/src/main/docs/sftp-component.adoc | 15 ++ .../component/file/remote/SftpOperations.java | 82 +++++- .../file/remote/SftpRemoteFileJCraft.java | 5 + ...emoteFileConsumerStartingDirectoryJailTest.java | 255 +++++++++++++++++- ...SftpConsumerJailStartingDirectorySymlinkIT.java | 291 +++++++++++++++++++++ .../src/main/docs/mina-sftp-component.adoc | 12 +- .../file/remote/mina/MinaSftpOperations.java | 55 +++- .../file/remote/mina/MinaSftpRemoteFile.java | 5 + .../mina/MinaSftpOperationsDisconnectTest.java | 38 +++ ...SftpConsumerJailStartingDirectorySymlinkIT.java | 291 +++++++++++++++++++++ .../ROOT/pages/camel-4x-upgrade-guide-4_22.adoc | 9 + 17 files changed, 1309 insertions(+), 8 deletions(-) diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/mina-sftp-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/mina-sftp-component.adoc index c574abf5eb14..461b2b0494f0 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/mina-sftp-component.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/mina-sftp-component.adoc @@ -569,7 +569,17 @@ IMPORTANT: The `chmod` and `chmodDirectory` options only work on POSIX-compatibl == Symbolic Links -The consumer follows symbolic links to their target files. The producer writes to symlink targets. +The consumer follows symbolic links to their target files, within the starting directory unless `jailStartingDirectory=false` (see below). The producer writes to symlink targets. + +With `jailStartingDirectory` enabled (the default), the consumer resolves a listed file that the server reports as a +symbolic link before it accepts the file, one path segment at a time with the `lstat` and `readlink` requests. A link +whose target is outside the starting directory, or cannot be resolved (such as a dangling link), is skipped and logged +at WARN the first time. A link whose target contains a backslash (`\`) is skipped too, as a server may take it as a +path separator or as part of a name. Links that resolve inside the starting directory, and a starting directory that is +itself a link, work as usual. Set `jailStartingDirectory=false` to follow links to other directories, such as the links +in `data` below when `data` is the starting directory. Only the files that the listing reports as links are resolved (a +server that lists the attributes of the target instead does not show them as links), and the check and the download +that follows are not atomic, so the permissions and link settings of the SFTP server still matter. NOTE: **Absolute symlinks in chroot environments**: If the SFTP server uses a chroot jail, absolute symlinks may not resolve correctly. Use **relative symlinks** for maximum compatibility. diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/sftp-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/sftp-component.adoc index 7522a5d1d490..f893b297d3c3 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/sftp-component.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/sftp-component.adoc @@ -311,6 +311,21 @@ ssh -vvv <server_address> As of Camel 3.18.1, these values can also be set on SFTP endpoints by setting the corresponding URI parameters. +=== Symbolic links and the starting directory + +With `jailStartingDirectory` enabled (the default), the consumer resolves a listed file that the server reports as a +symbolic link before it accepts the file. A link whose target is outside the starting directory, or cannot be resolved +(such as a dangling link), is skipped and logged at WARN the first time. A link whose target contains a backslash (`\`) +is skipped too, as a server may take it as a path separator or as part of a name. Links that resolve inside the +starting directory, and a starting directory that is itself a link, work as usual. Set `jailStartingDirectory=false` to +follow links to other directories. + +The link is resolved on the server one path segment at a time, with the `lstat` and `readlink` requests, so each listed +link takes a few more requests per poll; other files take none. Only the files that the listing reports as links are +resolved: OpenSSH and Apache MINA SSHD list a link as a link, but a server that lists the attributes of the target +instead does not show it as a link, and with `useList=false` there is no listing. The check and the download that +follows are not atomic, so the permissions and link settings of the SFTP server still matter. + == More Information For more information, you can look at the xref:ftp-component.adoc[FTP component]. diff --git a/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileConsumer.java b/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileConsumer.java index a06532ecd713..56fc7ed6cf64 100644 --- a/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileConsumer.java +++ b/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileConsumer.java @@ -639,12 +639,15 @@ public abstract class GenericFileConsumer<T> extends ScheduledBatchPollingConsum // the resolved path must stay within the starting directory, as the name it was built from is // reported by the remote server and is not necessarily a single path segment, and a local file // can be a symbolic link - if (endpoint.isJailStartingDirectory() && !isWithinStartingDirectory(absoluteFilePath)) { + if (endpoint.isJailStartingDirectory() && !isWithinStartingDirectory(file, absoluteFilePath)) { + String message = isUnresolvableLink(absoluteFilePath) + ? "Skipping file as symbolic link cannot be resolved: {}" + : "Skipping file as it resolves outside the starting directory: {}"; // such a file usually stays where it is and is listed again on every poll, so only warn the first time if (skippedOutsideStartingDirectory.put(absoluteFilePath, Boolean.TRUE) == null) { - LOG.warn("Skipping file as it resolves outside the starting directory: {}", absoluteFilePath); + LOG.warn(message, absoluteFilePath); } else { - LOG.debug("Skipping file as it resolves outside the starting directory: {}", absoluteFilePath); + LOG.debug(message, absoluteFilePath); } return false; } @@ -714,6 +717,31 @@ public abstract class GenericFileConsumer<T> extends ScheduledBatchPollingConsum return true; } + /** + * Strategy to determine whether a listed file stays within the configured starting directory. + * <p/> + * Checks the resolved path with {@link #isWithinStartingDirectory(String)} by default. Consumers whose listing + * tells more about a file than its path, such as whether it is a symbolic link, override this. The check is only + * consulted when {@link GenericFileEndpoint#isJailStartingDirectory()} is enabled. + * + * @param file the listed file + * @param absoluteFilePath the resolved absolute path of the listed file + * @return {@code true} if the file stays within the starting directory + */ + protected boolean isWithinStartingDirectory(Supplier<GenericFile<T>> file, String absoluteFilePath) { + return isWithinStartingDirectory(absoluteFilePath); + } + + /** + * Strategy to determine whether a skipped file could not be resolved (such as a dangling symbolic link). + * + * @param absoluteFilePath the resolved absolute path of the listed file + * @return {@code true} if the file could not be resolved rather than resolving outside + */ + protected boolean isUnresolvableLink(String absoluteFilePath) { + return false; + } + /** * Strategy to perform hidden file matching based on endpoint configuration. * <p/> diff --git a/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/AbstractSftpConsumer.java b/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/AbstractSftpConsumer.java index f2ee39f1071d..21593e23a3d7 100644 --- a/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/AbstractSftpConsumer.java +++ b/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/AbstractSftpConsumer.java @@ -16,9 +16,13 @@ */ package org.apache.camel.component.file.remote; +import java.util.ArrayDeque; import java.util.Arrays; import java.util.Comparator; +import java.util.Deque; import java.util.List; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; import java.util.function.Supplier; import org.apache.camel.Exchange; @@ -41,8 +45,18 @@ public abstract class AbstractSftpConsumer extends RemoteFileConsumer<SftpRemote private static final Logger LOG = LoggerFactory.getLogger(AbstractSftpConsumer.class); + /** + * The most symbolic links followed to resolve a path, as a longer chain is most likely a loop + */ + private static final int MAX_SYMBOLIC_LINKS = 32; + protected String endpointPath; + // the starting directory with its symbolic links resolved on the server, computed once per poll + private volatile String resolvedStartingDirectory; + + private final Set<String> unresolvableLinks = ConcurrentHashMap.newKeySet(); + protected AbstractSftpConsumer(RemoteFileEndpoint<SftpRemoteFile> endpoint, Processor processor, RemoteFileOperations<SftpRemoteFile> operations, GenericFileProcessStrategy<SftpRemoteFile> processStrategy) { @@ -78,8 +92,18 @@ public abstract class AbstractSftpConsumer extends RemoteFileConsumer<SftpRemote } } + @Override + protected void doStop() throws Exception { + unresolvableLinks.clear(); + super.doStop(); + } + @Override protected boolean pollDirectory(Exchange dynamic, String fileName, List<GenericFile<SftpRemoteFile>> fileList, int depth) { + // resolved lazily during this poll on the first symbolic link encountered, as the starting directory can be replaced between polls + resolvedStartingDirectory = null; + unresolvableLinks.clear(); + String currentDir = null; if (isStepwise()) { // must remember current dir so we stay in that directory after the poll @@ -247,6 +271,120 @@ public abstract class AbstractSftpConsumer extends RemoteFileConsumer<SftpRemote return false; } + @Override + protected boolean isWithinStartingDirectory(Supplier<GenericFile<SftpRemoteFile>> file, String absoluteFilePath) { + if (!super.isWithinStartingDirectory(file, absoluteFilePath)) { + return false; + } + GenericFile<SftpRemoteFile> remoteFile = file.get(); + if (remoteFile.getFile() == null || !remoteFile.getFile().isSymbolicLink()) { + return true; + } + // the server follows a symbolic link when the file is retrieved, so the target of a listed link must be within + // the starting directory too + if (!(operations instanceof SftpSymbolicLinkOperations links)) { + LOG.debug("Cannot resolve symbolic link: {} as the operations cannot read symbolic links", absoluteFilePath); + return false; + } + try { + String startingDirectory = resolvedStartingDirectory; + if (startingDirectory == null) { + startingDirectory = resolveSymbolicLinks(links, null, endpointPath); + resolvedStartingDirectory = startingDirectory; + } + // the path of the file below the starting directory is resolved from the resolved starting directory + String relativeFilePath = remoteFile.getRelativeFilePath(); + String target = relativeFilePath != null + ? resolveSymbolicLinks(links, startingDirectory, relativeFilePath) + : resolveSymbolicLinks(links, null, remoteFile.getAbsoluteFilePath()); + return GenericFileHelper.isWithinDirectory(target, startingDirectory, '/'); + } catch (GenericFileOperationFailedException e) { + if (links.isTransportError(e)) { + throw e; + } + // such as a dangling symbolic link + unresolvableLinks.add(absoluteFilePath); + LOG.debug("Cannot resolve symbolic link: {} against the starting directory due to: {}", absoluteFilePath, + e.getMessage()); + return false; + } + } + + @Override + protected boolean isUnresolvableLink(String absoluteFilePath) { + return unresolvableLinks.contains(absoluteFilePath); + } + + /** + * Resolves a path on the server the way the server does when it opens the path: one segment at a time, replacing + * each symbolic link with its target, which is relative to the directory of the link unless it is absolute. The + * realpath request is not used for this, as some servers (such as Apache MINA SSHD) do not resolve links in it. A + * link whose target contains a backslash is not resolved, as a server may take a backslash as a path separator (as + * on Windows) or as part of a name, so the path that the server follows is not known. + * + * @param operations the operations to look up the segments with + * @param directory the absolute path, without symbolic links, that a relative path is resolved against, or + * {@code null} for the home directory + * @param path the path to resolve + * @return the absolute path, without symbolic links and without {@code .} and {@code ..} segments + */ + private static String resolveSymbolicLinks(SftpSymbolicLinkOperations operations, String directory, String path) { + String name = path != null ? path : ""; + Deque<String> resolved = new ArrayDeque<>(); + Deque<String> pending = new ArrayDeque<>(); + if (!name.startsWith("/")) { + if (directory != null) { + // the directory has no links to resolve + for (String segment : directory.split("/")) { + if (!segment.isEmpty()) { + resolved.addLast(segment); + } + } + } else { + pending.addAll(Arrays.asList(operations.getHomeDirectory().split("/"))); + } + } + pending.addAll(Arrays.asList(name.split("/"))); + + int links = 0; + while (!pending.isEmpty()) { + String segment = pending.removeFirst(); + if (segment.isEmpty() || ".".equals(segment)) { + continue; + } + if ("..".equals(segment)) { + resolved.pollLast(); + continue; + } + String candidate = "/" + (resolved.isEmpty() ? "" : String.join("/", resolved) + "/") + segment; + if (operations.isSymbolicLink(candidate)) { + if (++links > MAX_SYMBOLIC_LINKS) { + throw new GenericFileOperationFailedException("Too many levels of symbolic links: " + name); + } + String target = operations.readSymbolicLink(candidate); + if (ObjectHelper.isEmpty(target)) { + throw new GenericFileOperationFailedException("Symbolic link without a target: " + candidate); + } + if (target.indexOf('\\') != -1) { + // splitting on both separators would be wrong on a server that takes a backslash as part of a name + throw new GenericFileOperationFailedException( + "Symbolic link with a backslash in its target: " + candidate); + } + if (target.startsWith("/")) { + resolved.clear(); + } + // the segments of the target take the place of the link + String[] segments = target.split("/"); + for (int i = segments.length - 1; i >= 0; i--) { + pending.addFirst(segments[i]); + } + } else { + resolved.addLast(segment); + } + } + return "/" + String.join("/", resolved); + } + @Override protected Supplier<String> getRelativeFilePath(String endpointPath, String path, String absolutePath, SftpRemoteFile file) { return () -> { diff --git a/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/SftpRemoteFile.java b/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/SftpRemoteFile.java index f135525ba6c8..8a9efff88da3 100644 --- a/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/SftpRemoteFile.java +++ b/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/SftpRemoteFile.java @@ -41,6 +41,13 @@ public interface SftpRemoteFile<T> { */ boolean isDirectory(); + /** + * Whether the server listed it as a symbolic link + */ + default boolean isSymbolicLink() { + return false; + } + /** * The file size */ diff --git a/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/SftpSymbolicLinkOperations.java b/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/SftpSymbolicLinkOperations.java new file mode 100644 index 000000000000..20481e124f91 --- /dev/null +++ b/components/camel-ftp-common/src/main/java/org/apache/camel/component/file/remote/SftpSymbolicLinkOperations.java @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.component.file.remote; + +import org.apache.camel.component.file.GenericFileOperationFailedException; + +/** + * The operations of an SFTP server that the consumer uses to resolve the symbolic links of a listing. + */ +public interface SftpSymbolicLinkOperations { + + /** + * The absolute path of the directory that the server resolves relative paths against, such as the home directory of + * the user. + */ + String getHomeDirectory() throws GenericFileOperationFailedException; + + /** + * Whether the path is a symbolic link, without following the link (the {@code lstat} request). + * + * @throws GenericFileOperationFailedException if the path does not exist or cannot be read + */ + boolean isSymbolicLink(String path) throws GenericFileOperationFailedException; + + /** + * The target of the symbolic link, as stored in the link (the {@code readlink} request). + * + * @throws GenericFileOperationFailedException if the path is not a symbolic link or cannot be read + */ + String readSymbolicLink(String path) throws GenericFileOperationFailedException; + + /** + * Whether the failure is due to a transport or connection error rather than an unresolvable symbolic link. + */ + default boolean isTransportError(GenericFileOperationFailedException e) { + return false; + } + +} diff --git a/components/camel-ftp/src/main/docs/sftp-component.adoc b/components/camel-ftp/src/main/docs/sftp-component.adoc index 7522a5d1d490..f893b297d3c3 100644 --- a/components/camel-ftp/src/main/docs/sftp-component.adoc +++ b/components/camel-ftp/src/main/docs/sftp-component.adoc @@ -311,6 +311,21 @@ ssh -vvv <server_address> As of Camel 3.18.1, these values can also be set on SFTP endpoints by setting the corresponding URI parameters. +=== Symbolic links and the starting directory + +With `jailStartingDirectory` enabled (the default), the consumer resolves a listed file that the server reports as a +symbolic link before it accepts the file. A link whose target is outside the starting directory, or cannot be resolved +(such as a dangling link), is skipped and logged at WARN the first time. A link whose target contains a backslash (`\`) +is skipped too, as a server may take it as a path separator or as part of a name. Links that resolve inside the +starting directory, and a starting directory that is itself a link, work as usual. Set `jailStartingDirectory=false` to +follow links to other directories. + +The link is resolved on the server one path segment at a time, with the `lstat` and `readlink` requests, so each listed +link takes a few more requests per poll; other files take none. Only the files that the listing reports as links are +resolved: OpenSSH and Apache MINA SSHD list a link as a link, but a server that lists the attributes of the target +instead does not show it as a link, and with `useList=false` there is no listing. The check and the download that +follows are not atomic, so the permissions and link settings of the SFTP server still matter. + == More Information For more information, you can look at the xref:ftp-component.adoc[FTP component]. diff --git a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java index e921423868fa..735cc9b947f8 100644 --- a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java +++ b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java @@ -80,7 +80,7 @@ import static org.apache.camel.util.ObjectHelper.isNotEmpty; * <p/> * The JSCH session and channel are not thread-safe so we need to synchronize access to using this operation. */ -public class SftpOperations implements RemoteFileOperations<SftpRemoteFile> { +public class SftpOperations implements RemoteFileOperations<SftpRemoteFile>, SftpSymbolicLinkOperations { private static final Logger LOG = LoggerFactory.getLogger(SftpOperations.class); private static final Pattern UP_DIR_PATTERN = Pattern.compile("/[^/]+"); private static final int OK_STATUS = 0; @@ -940,6 +940,86 @@ public class SftpOperations implements RemoteFileOperations<SftpRemoteFile> { } } + @Override + public String getHomeDirectory() throws GenericFileOperationFailedException { + lock.lock(); + try { + return channel.getHome(); + } catch (SftpException e) { + throw new GenericFileOperationFailedException("Cannot get the home directory", e); + } finally { + lock.unlock(); + } + } + + @Override + public boolean isSymbolicLink(String path) throws GenericFileOperationFailedException { + lock.lock(); + try { + return channel.lstat(quotePathForJsch(path)).isLink(); + } catch (SftpException e) { + throw new GenericFileOperationFailedException("Cannot get the attributes of: " + path, e); + } finally { + lock.unlock(); + } + } + + @Override + public String readSymbolicLink(String path) throws GenericFileOperationFailedException { + lock.lock(); + try { + return channel.readlink(quotePathForJsch(path)); + } catch (SftpException e) { + throw new GenericFileOperationFailedException("Cannot read the symbolic link: " + path, e); + } finally { + lock.unlock(); + } + } + + @Override + public boolean isTransportError(GenericFileOperationFailedException e) { + if (!isConnected()) { + return true; + } + // JSch reports a status sent by the server as an SftpException without a cause, and wraps an I/O or session + // failure as an SftpException (SSH_FX_FAILURE) with that failure as its cause + return e.getCause() instanceof SftpException sftp && sftp.getCause() != null; + } + + /** + * Escapes the characters that JSch interprets in a path argument, so that the server acts on the literal path. + * <p/> + * {@link ChannelSftp#lstat(String)} and {@link ChannelSftp#readlink(String)} pass the path through JSch's + * {@code isUnique}/{@code glob_remote}, which treats {@code *} and {@code ?} as wildcards and {@code \} as an + * escape character and removes it. Without this, a listed link whose target contains one of these characters (for + * example a target that steps through a sibling symbolic link named with a backslash) would check a different path + * from the one the server opens when retrieving the file, which would let the jail check be bypassed. This mirrors + * JSch's own (package-private) {@code Util.quote}. + * + * @param path the literal path to look up + * @return the path with {@code \}, {@code *} and {@code ?} escaped for JSch + */ + private static String quotePathForJsch(String path) { + if (path == null) { + return null; + } + StringBuilder quoted = null; + for (int i = 0; i < path.length(); i++) { + char c = path.charAt(i); + if (c == '\\' || c == '*' || c == '?') { + if (quoted == null) { + quoted = new StringBuilder(path.length() + 8); + quoted.append(path, 0, i); + } + quoted.append('\\'); + } + if (quoted != null) { + quoted.append(c); + } + } + return quoted != null ? quoted.toString() : path; + } + @Override public boolean retrieveFile(String name, Exchange exchange, long size) throws GenericFileOperationFailedException { diff --git a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpRemoteFileJCraft.java b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpRemoteFileJCraft.java index 189650a0cbc9..8ab5476fff2d 100644 --- a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpRemoteFileJCraft.java +++ b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpRemoteFileJCraft.java @@ -46,6 +46,11 @@ public class SftpRemoteFileJCraft implements SftpRemoteFile<ChannelSftp.LsEntry> return entry.getAttrs().isDir(); } + @Override + public boolean isSymbolicLink() { + return entry.getAttrs().isLink(); + } + @Override public long getFileLength() { return entry.getAttrs().getSize(); diff --git a/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/RemoteFileConsumerStartingDirectoryJailTest.java b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/RemoteFileConsumerStartingDirectoryJailTest.java index c4a8008a597c..c46ac16334f7 100644 --- a/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/RemoteFileConsumerStartingDirectoryJailTest.java +++ b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/RemoteFileConsumerStartingDirectoryJailTest.java @@ -16,16 +16,31 @@ */ package org.apache.camel.component.file.remote; +import java.io.IOException; +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.function.Supplier; + +import com.jcraft.jsch.ChannelSftp; +import com.jcraft.jsch.SftpException; +import org.apache.camel.component.file.GenericFile; +import org.apache.camel.component.file.GenericFileOperationFailedException; import org.apache.camel.test.junit6.CamelTestSupport; +import org.apache.camel.util.FileUtil; import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertTrue; /** * The file name used to build the path of a polled file comes from the directory listing returned by the remote server, * so it is not guaranteed to be a single path segment. Verifies the resolved path is kept inside the directory being - * polled before it is used as the operand for retrieving, deleting or renaming. + * polled before it is used as the operand for retrieving, deleting or renaming. A listed file that the SFTP server + * reports as a symbolic link is followed by the server when it is retrieved, so its target must be inside the directory + * too. */ class RemoteFileConsumerStartingDirectoryJailTest extends CamelTestSupport { @@ -35,6 +50,12 @@ class RemoteFileConsumerStartingDirectoryJailTest extends CamelTestSupport { }); } + private AbstractSftpConsumer sftpConsumer(String uri, SftpOperations operations) { + SftpEndpoint endpoint = context.getEndpoint(uri, SftpEndpoint.class); + return new SftpConsumer(endpoint, exchange -> { + }, operations, null); + } + @Test void shouldAcceptPathsWithinTheStartingDirectory() throws Exception { RemoteFileConsumer<?> consumer = consumer("ftp://hostname/poll";); @@ -74,4 +95,236 @@ class RemoteFileConsumerStartingDirectoryJailTest extends CamelTestSupport { // no directory is configured, but navigating above the session root still escapes assertFalse(consumer.isWithinStartingDirectory("../secret.txt")); } + + @Test + void shouldNotResolveListedFilesThatAreNotSymbolicLinks() { + StubSftpOperations server = new StubSftpOperations().file("/home/camel/poll/hello.txt"); + AbstractSftpConsumer consumer = sftpConsumer("sftp://hostname/poll";, server); + + assertTrue(consumer.isWithinStartingDirectory(listed("poll", "hello.txt", false), "poll/hello.txt")); + assertTrue(consumer.isWithinStartingDirectory(listed("poll", "sub/hello.txt", false), "poll/sub/hello.txt")); + // the path built from the listed name is still checked + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "../hello.txt", false), "poll/../hello.txt")); + // without asking the server + assertEquals(0, server.lookups); + } + + @Test + void shouldAcceptSymbolicLinkResolvingInsideTheStartingDirectory() { + StubSftpOperations server = new StubSftpOperations() + .file("/home/camel/poll/hello.txt") + .file("/home/camel/poll/data/report.txt") + .link("/home/camel/poll/latest.txt", "hello.txt") + .link("/home/camel/poll/report.txt", "/home/camel/poll/data/report.txt") + .link("/home/camel/poll/current", "data") + .link("/home/camel/poll/current.txt", "current/report.txt") + .link("/home/camel/poll/again.txt", "../poll/latest.txt") + .link("/home/camel/poll/data/hello.txt", "../hello.txt"); + AbstractSftpConsumer consumer = sftpConsumer("sftp://hostname/poll";, server); + + // relative to the directory of the link, absolute, through a link to a directory, and to another link + assertTrue(consumer.isWithinStartingDirectory(listed("poll", "latest.txt", true), "poll/latest.txt")); + assertTrue(consumer.isWithinStartingDirectory(listed("poll", "report.txt", true), "poll/report.txt")); + assertTrue(consumer.isWithinStartingDirectory(listed("poll", "current.txt", true), "poll/current.txt")); + assertTrue(consumer.isWithinStartingDirectory(listed("poll", "again.txt", true), "poll/again.txt")); + // listed in a sub directory (recursive) + assertTrue(consumer.isWithinStartingDirectory(listed("poll", "data/hello.txt", true), "poll/data/hello.txt")); + + // a starting directory that is itself a link + StubSftpOperations linkedServer = new StubSftpOperations() + .file("/srv/inbox/hello.txt") + .link("/home/camel/inbox", "/srv/inbox") + .link("/srv/inbox/latest.txt", "hello.txt"); + AbstractSftpConsumer linkedConsumer = sftpConsumer("sftp://hostname/inbox";, linkedServer); + + assertTrue(linkedConsumer.isWithinStartingDirectory(listed("inbox", "latest.txt", true), "inbox/latest.txt")); + } + + @Test + void shouldSkipSymbolicLinkResolvingOutsideTheStartingDirectory() { + StubSftpOperations server = new StubSftpOperations() + .file("/home/camel/poll/hello.txt") + .file("/home/camel/archive/report.txt") + .file("/home/camel/polling/report.txt") + .link("/home/camel/poll/report.txt", "../archive/report.txt") + .link("/home/camel/poll/absolute.txt", "/home/camel/archive/report.txt") + .link("/home/camel/poll/archive", "../archive") + .link("/home/camel/poll/through.txt", "archive/report.txt") + .link("/home/camel/poll/polling.txt", "../polling/report.txt") + .link("/home/camel/poll/sub/report.txt", "../../archive/report.txt"); + AbstractSftpConsumer consumer = sftpConsumer("sftp://hostname/poll";, server); + + // relative upwards, absolute, and through a link to a directory outside + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "report.txt", true), "poll/report.txt")); + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "absolute.txt", true), "poll/absolute.txt")); + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "through.txt", true), "poll/through.txt")); + // a sibling directory whose name merely extends the polled directory name is not contained + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "polling.txt", true), "poll/polling.txt")); + // listed in a sub directory (recursive) + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "sub/report.txt", true), "poll/sub/report.txt")); + } + + @Test + void shouldSkipSymbolicLinkThatCannotBeResolved() { + StubSftpOperations server = new StubSftpOperations() + .file("/home/camel/poll/hello.txt") + .link("/home/camel/poll/missing.txt", "gone.txt") + .link("/home/camel/poll/ping.txt", "pong.txt") + .link("/home/camel/poll/pong.txt", "ping.txt") + .link("/home/camel/poll/empty.txt", ""); + AbstractSftpConsumer consumer = sftpConsumer("sftp://hostname/poll";, server); + + // a dangling link, a loop, and a link without a target + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "missing.txt", true), "poll/missing.txt")); + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "ping.txt", true), "poll/ping.txt")); + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "empty.txt", true), "poll/empty.txt")); + } + + @Test + void shouldSkipSymbolicLinkWhoseTargetContainsBackslash() { + // a server that takes a backslash as a path separator too, as on Windows + StubSftpOperations server = new StubSftpOperations().backslashSeparator() + .file("/home/camel/poll/hello.txt") + .file("/home/camel/poll/sub/file.txt") + .file("/home/outside.txt") + .link("/home/camel/poll/report.txt", "..\\..\\outside.txt") + .link("/home/camel/poll/inside.txt", "sub\\file.txt"); + AbstractSftpConsumer consumer = sftpConsumer("sftp://hostname/poll";, server); + + // the server follows the first link out of the starting directory, and the second one inside, but which path + // a server follows for a backslash depends on the server, so both are skipped as links that cannot be resolved + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "report.txt", true), "poll/report.txt")); + assertTrue(consumer.isUnresolvableLink("poll/report.txt")); + assertFalse(consumer.isWithinStartingDirectory(listed("poll", "inside.txt", true), "poll/inside.txt")); + assertTrue(consumer.isUnresolvableLink("poll/inside.txt")); + + // a server that takes a backslash as part of a name + StubSftpOperations literalServer = new StubSftpOperations() + .file("/home/camel/poll/a\\b.txt") + .link("/home/camel/poll/literal.txt", "a\\b.txt"); + AbstractSftpConsumer literalConsumer = sftpConsumer("sftp://hostname/poll";, literalServer); + + assertFalse(literalConsumer.isWithinStartingDirectory(listed("poll", "literal.txt", true), "poll/literal.txt")); + assertTrue(literalConsumer.isUnresolvableLink("poll/literal.txt")); + } + + @Test + void isTransportErrorClassification() { + SftpOperations disconnected = new SftpOperations(); + assertTrue(disconnected.isTransportError(new GenericFileOperationFailedException("disconnected")), + "Disconnected operations should classify any failure as transport error"); + + SftpOperations connected = new SftpOperations() { + @Override + public boolean isConnected() { + return true; + } + }; + + // JSch wraps an I/O or session failure as SSH_FX_FAILURE with that failure as the cause + assertTrue(connected.isTransportError(new GenericFileOperationFailedException( + "broken pipe", new SftpException( + ChannelSftp.SSH_FX_FAILURE, "java.io.IOException: broken pipe", new IOException("broken pipe"))))); + + // A status sent by the server is not a transport error, whatever its code. JSch never raises the + // SSH_FX_CONNECTION_LOST and SSH_FX_NO_CONNECTION pseudo-errors itself, so they can only be such a status too + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "connection lost", new SftpException(ChannelSftp.SSH_FX_CONNECTION_LOST, "connection lost")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "no connection", new SftpException(ChannelSftp.SSH_FX_NO_CONNECTION, "no connection")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "no such file", new SftpException(ChannelSftp.SSH_FX_NO_SUCH_FILE, "no such file")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "permission denied", new SftpException(ChannelSftp.SSH_FX_PERMISSION_DENIED, "permission denied")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "failure", new SftpException(ChannelSftp.SSH_FX_FAILURE, "failure")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "bad message", new SftpException(ChannelSftp.SSH_FX_BAD_MESSAGE, "bad message")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "op unsupported", new SftpException(ChannelSftp.SSH_FX_OP_UNSUPPORTED, "op unsupported")))); + } + + private static Supplier<GenericFile<SftpRemoteFile>> listed( + String directory, String relativeFilePath, boolean symbolicLink) { + RemoteFile<SftpRemoteFile> file = new RemoteFile<>(); + file.setFile(new SftpRemoteFileSingle(FileUtil.stripPath(relativeFilePath)) { + @Override + public boolean isSymbolicLink() { + return symbolicLink; + } + }); + file.setEndpointPath(directory); + file.setRelativeFilePath(relativeFilePath); + file.setAbsoluteFilePath(directory + "/" + relativeFilePath); + return () -> file; + } + + /** + * The files of an SFTP server whose home directory is /home/camel: the paths that exist, and the symbolic links + * among them. + */ + private static final class StubSftpOperations extends SftpOperations { + + private final Set<String> files = new HashSet<>(); + private final Map<String, String> links = new HashMap<>(); + private int lookups; + private boolean backslashSeparator; + + /** + * Takes a backslash as a path separator too when looking up a path, as a server on Windows does. + */ + StubSftpOperations backslashSeparator() { + backslashSeparator = true; + return this; + } + + StubSftpOperations file(String path) { + // the file and the directories it is in + for (String name = path; name != null && !"/".equals(name); name = FileUtil.onlyPath(name)) { + files.add(name); + } + return this; + } + + StubSftpOperations link(String path, String target) { + file(FileUtil.onlyPath(path)); + links.put(path, target); + return this; + } + + @Override + public boolean isConnected() { + return true; + } + + @Override + public String getHomeDirectory() { + return "/home/camel"; + } + + @Override + public boolean isSymbolicLink(String path) { + lookups++; + String name = serverPath(path); + if (links.containsKey(name)) { + return true; + } + if (files.contains(name)) { + return false; + } + throw new GenericFileOperationFailedException("No such file: " + path); + } + + @Override + public String readSymbolicLink(String path) { + return links.get(serverPath(path)); + } + + /** + * The path that the server looks up for the path of a request. + */ + private String serverPath(String path) { + return backslashSeparator ? FileUtil.compactPath(path.replace('\\', '/'), '/') : path; + } + } } diff --git a/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/sftp/integration/SftpConsumerJailStartingDirectorySymlinkIT.java b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/sftp/integration/SftpConsumerJailStartingDirectorySymlinkIT.java new file mode 100644 index 000000000000..7c49615b6ab5 --- /dev/null +++ b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/sftp/integration/SftpConsumerJailStartingDirectorySymlinkIT.java @@ -0,0 +1,291 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.component.file.remote.sftp.integration; + +import java.io.IOException; +import java.nio.file.FileAlreadyExistsException; +import java.nio.file.FileSystemException; +import java.nio.file.Files; +import java.nio.file.Path; + +import org.apache.camel.Exchange; +import org.apache.camel.builder.RouteBuilder; +import org.apache.camel.component.mock.MockEndpoint; +import org.junit.jupiter.api.Assumptions; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.DisabledOnOs; +import org.junit.jupiter.api.condition.EnabledIf; +import org.junit.jupiter.api.condition.OS; + +/** + * With jailStartingDirectory (the default) the SFTP consumer resolves a listed symbolic link on the server, and only + * consumes it when its target is inside the starting directory. + */ +@EnabledIf(value = "org.apache.camel.test.infra.ftp.services.embedded.SftpUtil#hasRequiredAlgorithms('src/test/resources/hostkey.pem')") +class SftpConsumerJailStartingDirectorySymlinkIT extends SftpServerTestSupport { + + private Path base; + + @BeforeEach + void createTestDirectory() throws IOException { + // the root directory of the server is shared by the tests of this class and kept between runs + base = Files.createTempDirectory(Files.createDirectories(service.getFtpRootDir()), "test"); + } + + @Test + void consumesLinkResolvingInsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("latest.txt"), Path.of("hello.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World", "Hello World"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "latest.txt"); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + void skipsLinkResolvingOutsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(inbox.resolve("report.txt"), Path.of("..", "archive", "report.txt")); + createSymbolicLink(inbox.resolve("absolute.txt"), archive.resolve("report.txt").toAbsolutePath()); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + // the links are not part of the batch of the poll that picked up hello.txt + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + void followsLinkOutsideStartingDirectoryWhenJailDisabled() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(inbox.resolve("report.txt"), Path.of("..", "archive", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Archived"); + + consume("inbox", "&jailStartingDirectory=false"); + + mock.assertIsSatisfied(); + } + + @Test + void skipsDanglingLink() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("missing.txt"), Path.of("gone.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + // the link is not part of the batch of the poll that picked up hello.txt + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + void consumesLinkInStartingDirectoryReachedThroughLink() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("latest.txt"), Path.of("hello.txt")); + createSymbolicLink(base.resolve("inbox-link"), Path.of("inbox")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World", "Hello World"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "latest.txt"); + + consume("inbox-link", ""); + + mock.assertIsSatisfied(); + } + + @Test + void skipsLinkWhoseTargetStepsThroughBackslashNamedLinkOutsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path outside = Files.createDirectories(base.resolve("outside")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(outside.resolve("report.txt"), "Outside"); + // a real directory inside the starting directory, named as the link below with its backslash removed, with a + // file at the path that a check which drops the backslash would resolve and accept as inside the jail + Files.writeString(Files.createDirectories(inbox.resolve("pivotx")).resolve("report.txt"), "Inside"); + // a symbolic link whose name contains a backslash, pointing outside the starting directory + createSymbolicLink(inbox.resolve("pivot\\x"), Path.of("..", "outside")); + // a link whose stored target steps through the backslash-named link; the server follows it to the outside file + createSymbolicLink(inbox.resolve("latest.txt"), Path.of("pivot\\x", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + // the links are not part of the batch of the poll that picked up hello.txt (latest.txt is skipped, not consumed) + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + @DisabledOnOs(value = OS.WINDOWS, disabledReason = "Windows does not allow * and ? in file names") + void consumesLinksNamedWithWildcardsResolvingInsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("star*.txt"), Path.of("hello.txt")); + createSymbolicLink(inbox.resolve("what?.txt"), Path.of("hello.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World", "Hello World", "Hello World"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "star*.txt", "what?.txt"); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + @DisabledOnOs(value = OS.WINDOWS, disabledReason = "Windows does not allow * and ? in file names") + void consumesLinksResolvingInsideThroughWildcardNamedDirectories() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + // read as a pattern, each of the two directory names matches both directories + Files.writeString(Files.createDirectories(inbox.resolve("sub*dir")).resolve("report.txt"), "Star"); + Files.writeString(Files.createDirectories(inbox.resolve("sub?dir")).resolve("report.txt"), "Question"); + createSymbolicLink(inbox.resolve("star.txt"), Path.of("sub*dir", "report.txt")); + createSymbolicLink(inbox.resolve("question.txt"), Path.of("sub?dir", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceivedInAnyOrder("Hello World", "Star", "Question"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "star.txt", "question.txt"); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + @DisabledOnOs(value = OS.WINDOWS, disabledReason = "Windows does not allow * and ? in file names") + void skipsLinksNamedWithWildcardsResolvingOutsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(inbox.resolve("star*.txt"), Path.of("..", "archive", "report.txt")); + createSymbolicLink(inbox.resolve("what?.txt"), Path.of("..", "archive", "report.txt")); + // a link to the directory outside, and a link whose target steps through it + createSymbolicLink(inbox.resolve("arch*ve"), Path.of("..", "archive")); + createSymbolicLink(inbox.resolve("latest.txt"), Path.of("arch*ve", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + // the links are not part of the batch of the poll that picked up hello.txt + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + void skipsLinkResolvingOutsideStartingDirectoryStepwiseFalse() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(inbox.resolve("report.txt"), Path.of("..", "archive", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", "&stepwise=false"); + + mock.assertIsSatisfied(); + } + + @Test + void skipsDanglingLinkStepwiseFalse() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("missing.txt"), Path.of("gone.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", "&stepwise=false"); + + mock.assertIsSatisfied(); + } + + @Test + void skipsLinkResolvingOutsideInSubDirectoryRecursive() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path sub = Files.createDirectories(inbox.resolve("sub")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(sub.resolve("report.txt"), Path.of("..", "..", "archive", "report.txt")); + createSymbolicLink(sub.resolve("inner.txt"), Path.of("..", "hello.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceivedInAnyOrder("Hello World", "Hello World"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "sub/inner.txt"); + + consume("inbox", "&recursive=true"); + + mock.assertIsSatisfied(); + } + + private void consume(String directory, String query) throws Exception { + context.addRoutes(new RouteBuilder() { + @Override + public void configure() { + from("sftp://localhost:"; + service.getPort() + "/" + base.resolve(directory) + + "?username=admin&password=admin&knownHostsFile=" + service.getKnownHostsFile() + + "&initialDelay=0&delay=100&noop=true" + query) + .convertBodyTo(String.class) + .to("mock:result"); + } + }); + } + + private static void createSymbolicLink(Path link, Path target) { + try { + Files.createSymbolicLink(link, target); + } catch (FileAlreadyExistsException e) { + throw new IllegalStateException(e); + } catch (UnsupportedOperationException | FileSystemException e) { + // such as Windows when the user may not create symbolic links + Assumptions.abort("Symbolic links cannot be created on this platform: " + e.getMessage()); + } catch (IOException e) { + throw new IllegalStateException(e); + } + } +} diff --git a/components/camel-mina-sftp/src/main/docs/mina-sftp-component.adoc b/components/camel-mina-sftp/src/main/docs/mina-sftp-component.adoc index c574abf5eb14..461b2b0494f0 100644 --- a/components/camel-mina-sftp/src/main/docs/mina-sftp-component.adoc +++ b/components/camel-mina-sftp/src/main/docs/mina-sftp-component.adoc @@ -569,7 +569,17 @@ IMPORTANT: The `chmod` and `chmodDirectory` options only work on POSIX-compatibl == Symbolic Links -The consumer follows symbolic links to their target files. The producer writes to symlink targets. +The consumer follows symbolic links to their target files, within the starting directory unless `jailStartingDirectory=false` (see below). The producer writes to symlink targets. + +With `jailStartingDirectory` enabled (the default), the consumer resolves a listed file that the server reports as a +symbolic link before it accepts the file, one path segment at a time with the `lstat` and `readlink` requests. A link +whose target is outside the starting directory, or cannot be resolved (such as a dangling link), is skipped and logged +at WARN the first time. A link whose target contains a backslash (`\`) is skipped too, as a server may take it as a +path separator or as part of a name. Links that resolve inside the starting directory, and a starting directory that is +itself a link, work as usual. Set `jailStartingDirectory=false` to follow links to other directories, such as the links +in `data` below when `data` is the starting directory. Only the files that the listing reports as links are resolved (a +server that lists the attributes of the target instead does not show them as links), and the check and the download +that follows are not atomic, so the permissions and link settings of the SFTP server still matter. NOTE: **Absolute symlinks in chroot environments**: If the SFTP server uses a chroot jail, absolute symlinks may not resolve correctly. Use **relative symlinks** for maximum compatibility. diff --git a/components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpOperations.java b/components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpOperations.java index 412b114be7ad..f286137e3b55 100644 --- a/components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpOperations.java +++ b/components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpOperations.java @@ -49,6 +49,7 @@ import org.apache.camel.component.file.remote.RemoteFile; import org.apache.camel.component.file.remote.RemoteFileConfiguration; import org.apache.camel.component.file.remote.RemoteFileOperations; import org.apache.camel.component.file.remote.SftpRemoteFile; +import org.apache.camel.component.file.remote.SftpSymbolicLinkOperations; import org.apache.camel.support.task.BlockingTask; import org.apache.camel.support.task.Tasks; import org.apache.camel.support.task.budget.Budgets; @@ -76,6 +77,8 @@ import org.apache.sshd.sftp.client.SftpClient; import org.apache.sshd.sftp.client.SftpClient.DirEntry; import org.apache.sshd.sftp.client.SftpClient.OpenMode; import org.apache.sshd.sftp.client.SftpClientFactory; +import org.apache.sshd.sftp.common.SftpConstants; +import org.apache.sshd.sftp.common.SftpException; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -84,7 +87,7 @@ import org.slf4j.LoggerFactory; * * The SSHD session and SFTP client are not thread-safe so we need to synchronize access to using this operation. */ -public class MinaSftpOperations implements RemoteFileOperations<SftpRemoteFile> { +public class MinaSftpOperations implements RemoteFileOperations<SftpRemoteFile>, SftpSymbolicLinkOperations { private static final Logger LOG = LoggerFactory.getLogger(MinaSftpOperations.class); private static final Pattern UP_DIR_PATTERN = Pattern.compile("/[^/]+"); @@ -1179,6 +1182,56 @@ public class MinaSftpOperations implements RemoteFileOperations<SftpRemoteFile> } } + @Override + public String getHomeDirectory() throws GenericFileOperationFailedException { + lock.lock(); + try { + // the server resolves relative paths against its default directory + return sftpClient.canonicalPath("."); + } catch (IOException e) { + throw new GenericFileOperationFailedException("Cannot get the home directory", e); + } finally { + lock.unlock(); + } + } + + @Override + public boolean isSymbolicLink(String path) throws GenericFileOperationFailedException { + lock.lock(); + try { + return sftpClient.lstat(path).isSymbolicLink(); + } catch (IOException e) { + throw new GenericFileOperationFailedException("Cannot get the attributes of: " + path, e); + } finally { + lock.unlock(); + } + } + + @Override + public String readSymbolicLink(String path) throws GenericFileOperationFailedException { + lock.lock(); + try { + return sftpClient.readLink(path); + } catch (IOException e) { + throw new GenericFileOperationFailedException("Cannot read the symbolic link: " + path, e); + } finally { + lock.unlock(); + } + } + + @Override + public boolean isTransportError(GenericFileOperationFailedException e) { + if (!isConnected()) { + return true; + } + Throwable cause = e.getCause(); + if (cause instanceof SftpException sftp) { + return sftp.getStatus() == SftpConstants.SSH_FX_NO_CONNECTION + || sftp.getStatus() == SftpConstants.SSH_FX_CONNECTION_LOST; + } + return cause instanceof IOException; + } + @Override public boolean retrieveFile(String name, Exchange exchange, long size) throws GenericFileOperationFailedException { diff --git a/components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpRemoteFile.java b/components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpRemoteFile.java index 7db395c8d02a..2fae31f91aa1 100644 --- a/components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpRemoteFile.java +++ b/components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpRemoteFile.java @@ -50,6 +50,11 @@ public class MinaSftpRemoteFile implements SftpRemoteFile<SftpClient.DirEntry> { return entry.getAttributes().isDirectory(); } + @Override + public boolean isSymbolicLink() { + return entry.getAttributes().isSymbolicLink(); + } + @Override public long getFileLength() { return entry.getAttributes().getSize(); diff --git a/components/camel-mina-sftp/src/test/java/org/apache/camel/component/file/remote/mina/MinaSftpOperationsDisconnectTest.java b/components/camel-mina-sftp/src/test/java/org/apache/camel/component/file/remote/mina/MinaSftpOperationsDisconnectTest.java index f89078116f95..55a5ea74e6a2 100644 --- a/components/camel-mina-sftp/src/test/java/org/apache/camel/component/file/remote/mina/MinaSftpOperationsDisconnectTest.java +++ b/components/camel-mina-sftp/src/test/java/org/apache/camel/component/file/remote/mina/MinaSftpOperationsDisconnectTest.java @@ -16,9 +16,13 @@ */ package org.apache.camel.component.file.remote.mina; +import java.io.IOException; import java.lang.reflect.Field; +import org.apache.camel.component.file.GenericFileOperationFailedException; import org.apache.sshd.client.SshClient; +import org.apache.sshd.sftp.common.SftpConstants; +import org.apache.sshd.sftp.common.SftpException; import org.junit.jupiter.api.Test; import static org.junit.jupiter.api.Assertions.assertFalse; @@ -122,6 +126,40 @@ class MinaSftpOperationsDisconnectTest { operations.disconnect(); } + @Test + void testIsTransportErrorClassification() { + MinaSftpOperations disconnected = new MinaSftpOperations(); + assertTrue(disconnected.isTransportError(new GenericFileOperationFailedException("disconnected")), + "Disconnected operations should classify any failure as transport error"); + + MinaSftpOperations connected = new MinaSftpOperations() { + @Override + public boolean isConnected() { + return true; + } + }; + + // Whitelisted transport errors + assertTrue(connected.isTransportError(new GenericFileOperationFailedException( + "connection lost", new SftpException(SftpConstants.SSH_FX_CONNECTION_LOST, "connection lost")))); + assertTrue(connected.isTransportError(new GenericFileOperationFailedException( + "no connection", new SftpException(SftpConstants.SSH_FX_NO_CONNECTION, "no connection")))); + assertTrue(connected.isTransportError(new GenericFileOperationFailedException( + "io exception", new IOException("broken pipe")))); + + // Application-level errors are not transport errors + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "no such file", new SftpException(SftpConstants.SSH_FX_NO_SUCH_FILE, "no such file")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "permission denied", new SftpException(SftpConstants.SSH_FX_PERMISSION_DENIED, "permission denied")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "failure", new SftpException(SftpConstants.SSH_FX_FAILURE, "failure")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "bad message", new SftpException(SftpConstants.SSH_FX_BAD_MESSAGE, "bad message")))); + assertFalse(connected.isTransportError(new GenericFileOperationFailedException( + "op unsupported", new SftpException(SftpConstants.SSH_FX_OP_UNSUPPORTED, "op unsupported")))); + } + private static void setField(Object target, String fieldName, Object value) throws Exception { Field field = target.getClass().getDeclaredField(fieldName); field.setAccessible(true); diff --git a/components/camel-mina-sftp/src/test/java/org/apache/camel/component/file/remote/mina/integration/MinaSftpConsumerJailStartingDirectorySymlinkIT.java b/components/camel-mina-sftp/src/test/java/org/apache/camel/component/file/remote/mina/integration/MinaSftpConsumerJailStartingDirectorySymlinkIT.java new file mode 100644 index 000000000000..7435d280b396 --- /dev/null +++ b/components/camel-mina-sftp/src/test/java/org/apache/camel/component/file/remote/mina/integration/MinaSftpConsumerJailStartingDirectorySymlinkIT.java @@ -0,0 +1,291 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.component.file.remote.mina.integration; + +import java.io.IOException; +import java.nio.file.FileAlreadyExistsException; +import java.nio.file.FileSystemException; +import java.nio.file.Files; +import java.nio.file.Path; + +import org.apache.camel.Exchange; +import org.apache.camel.builder.RouteBuilder; +import org.apache.camel.component.mock.MockEndpoint; +import org.junit.jupiter.api.Assumptions; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.DisabledOnOs; +import org.junit.jupiter.api.condition.EnabledIf; +import org.junit.jupiter.api.condition.OS; + +/** + * With jailStartingDirectory (the default) the mina-sftp consumer resolves a listed symbolic link on the server, and + * only consumes it when its target is inside the starting directory. + */ +@EnabledIf(value = "org.apache.camel.test.infra.ftp.services.embedded.SftpUtil#hasRequiredAlgorithms('src/test/resources/hostkey.pem')") +class MinaSftpConsumerJailStartingDirectorySymlinkIT extends MinaSftpServerTestSupport { + + private Path base; + + @BeforeEach + void createTestDirectory() throws IOException { + // the root directory of the server is shared by the tests of this class and kept between runs + base = Files.createTempDirectory(Files.createDirectories(service.getFtpRootDir()), "test"); + } + + @Test + void consumesLinkResolvingInsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("latest.txt"), Path.of("hello.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World", "Hello World"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "latest.txt"); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + void skipsLinkResolvingOutsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(inbox.resolve("report.txt"), Path.of("..", "archive", "report.txt")); + createSymbolicLink(inbox.resolve("absolute.txt"), archive.resolve("report.txt").toAbsolutePath()); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + // the links are not part of the batch of the poll that picked up hello.txt + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + void followsLinkOutsideStartingDirectoryWhenJailDisabled() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(inbox.resolve("report.txt"), Path.of("..", "archive", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Archived"); + + consume("inbox", "&jailStartingDirectory=false"); + + mock.assertIsSatisfied(); + } + + @Test + void skipsDanglingLink() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("missing.txt"), Path.of("gone.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + // the link is not part of the batch of the poll that picked up hello.txt + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + void consumesLinkInStartingDirectoryReachedThroughLink() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("latest.txt"), Path.of("hello.txt")); + createSymbolicLink(base.resolve("inbox-link"), Path.of("inbox")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World", "Hello World"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "latest.txt"); + + consume("inbox-link", ""); + + mock.assertIsSatisfied(); + } + + @Test + void skipsLinkWhoseTargetStepsThroughBackslashNamedLinkOutsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path outside = Files.createDirectories(base.resolve("outside")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(outside.resolve("report.txt"), "Outside"); + // a real directory inside the starting directory, named as the link below with its backslash removed, with a + // file at the path that a check which drops the backslash would resolve and accept as inside the jail + Files.writeString(Files.createDirectories(inbox.resolve("pivotx")).resolve("report.txt"), "Inside"); + // a symbolic link whose name contains a backslash, pointing outside the starting directory + createSymbolicLink(inbox.resolve("pivot\\x"), Path.of("..", "outside")); + // a link whose stored target steps through the backslash-named link; the server follows it to the outside file + createSymbolicLink(inbox.resolve("latest.txt"), Path.of("pivot\\x", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + // the links are not part of the batch of the poll that picked up hello.txt (latest.txt is skipped, not consumed) + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + @DisabledOnOs(value = OS.WINDOWS, disabledReason = "Windows does not allow * and ? in file names") + void consumesLinksNamedWithWildcardsResolvingInsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("star*.txt"), Path.of("hello.txt")); + createSymbolicLink(inbox.resolve("what?.txt"), Path.of("hello.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World", "Hello World", "Hello World"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "star*.txt", "what?.txt"); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + @DisabledOnOs(value = OS.WINDOWS, disabledReason = "Windows does not allow * and ? in file names") + void consumesLinksResolvingInsideThroughWildcardNamedDirectories() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + // read as a pattern, each of the two directory names matches both directories + Files.writeString(Files.createDirectories(inbox.resolve("sub*dir")).resolve("report.txt"), "Star"); + Files.writeString(Files.createDirectories(inbox.resolve("sub?dir")).resolve("report.txt"), "Question"); + createSymbolicLink(inbox.resolve("star.txt"), Path.of("sub*dir", "report.txt")); + createSymbolicLink(inbox.resolve("question.txt"), Path.of("sub?dir", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceivedInAnyOrder("Hello World", "Star", "Question"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "star.txt", "question.txt"); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + @DisabledOnOs(value = OS.WINDOWS, disabledReason = "Windows does not allow * and ? in file names") + void skipsLinksNamedWithWildcardsResolvingOutsideStartingDirectory() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(inbox.resolve("star*.txt"), Path.of("..", "archive", "report.txt")); + createSymbolicLink(inbox.resolve("what?.txt"), Path.of("..", "archive", "report.txt")); + // a link to the directory outside, and a link whose target steps through it + createSymbolicLink(inbox.resolve("arch*ve"), Path.of("..", "archive")); + createSymbolicLink(inbox.resolve("latest.txt"), Path.of("arch*ve", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + // the links are not part of the batch of the poll that picked up hello.txt + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", ""); + + mock.assertIsSatisfied(); + } + + @Test + void skipsLinkResolvingOutsideStartingDirectoryStepwiseFalse() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(inbox.resolve("report.txt"), Path.of("..", "archive", "report.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", "&stepwise=false"); + + mock.assertIsSatisfied(); + } + + @Test + void skipsDanglingLinkStepwiseFalse() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + createSymbolicLink(inbox.resolve("missing.txt"), Path.of("gone.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived("Hello World"); + mock.message(0).exchangeProperty(Exchange.BATCH_SIZE).isEqualTo(1); + + consume("inbox", "&stepwise=false"); + + mock.assertIsSatisfied(); + } + + @Test + void skipsLinkResolvingOutsideInSubDirectoryRecursive() throws Exception { + Path inbox = Files.createDirectories(base.resolve("inbox")); + Path sub = Files.createDirectories(inbox.resolve("sub")); + Path archive = Files.createDirectories(base.resolve("archive")); + Files.writeString(inbox.resolve("hello.txt"), "Hello World"); + Files.writeString(archive.resolve("report.txt"), "Archived"); + createSymbolicLink(sub.resolve("report.txt"), Path.of("..", "..", "archive", "report.txt")); + createSymbolicLink(sub.resolve("inner.txt"), Path.of("..", "hello.txt")); + + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceivedInAnyOrder("Hello World", "Hello World"); + mock.expectedHeaderValuesReceivedInAnyOrder(Exchange.FILE_NAME, "hello.txt", "sub/inner.txt"); + + consume("inbox", "&recursive=true"); + + mock.assertIsSatisfied(); + } + + private void consume(String directory, String query) throws Exception { + context.addRoutes(new RouteBuilder() { + @Override + public void configure() { + from("mina-sftp://localhost:"; + service.getPort() + "/" + base.resolve(directory) + + "?username=admin&password=admin&strictHostKeyChecking=no&useUserKnownHostsFile=false" + + "&initialDelay=0&delay=100&noop=true" + query) + .convertBodyTo(String.class) + .to("mock:result"); + } + }); + } + + private static void createSymbolicLink(Path link, Path target) { + try { + Files.createSymbolicLink(link, target); + } catch (FileAlreadyExistsException e) { + throw new IllegalStateException(e); + } catch (UnsupportedOperationException | FileSystemException e) { + // such as Windows when the user may not create symbolic links + Assumptions.abort("Symbolic links cannot be created on this platform: " + e.getMessage()); + } catch (IOException e) { + throw new IllegalStateException(e); + } + } +} diff --git a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc index af9a8e28fb64..5ebc25f6c55f 100644 --- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc +++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc @@ -972,6 +972,15 @@ links to other directories (this also turns off the check of `../` in producer f at WARN the first time and at DEBUG on later polls; this log-level change also applies to the remote file consumers, which keep their existing lexical check. +=== camel-ftp, camel-mina-sftp - the SFTP consumers resolve listed symbolic links for jailStartingDirectory + +With `jailStartingDirectory` enabled (the default), the `sftp` and `mina-sftp` consumers now resolve a listed file that +the server reports as a symbolic link, and skip it when its target is outside the starting directory or cannot be +resolved (such as a dangling link), as the file consumer does. A link whose target contains a backslash (`\`) is +skipped too. Before, such a link was consumed like any other file, wherever it pointed. A skipped file is logged at WARN +the first time and at DEBUG on later polls. Links resolving inside +the starting directory work as before. Set `jailStartingDirectory=false` to keep following links to other directories. + === camel-mail - MimeMultipartDataFormat inbound header filtering When unmarshalling a MIME message with `headersInline=true`, the `mime-multipart` data format now applies a
