This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new dead0782eca6 CAMEL-26925: camel-sftp - use remote file names literally
when calling JSch (#29848)
dead0782eca6 is described below
commit dead0782eca6315290e93115bb82788ca4c96c19
Author: Claus Ibsen <[email protected]>
AuthorDate: Fri Nov 13 23:44:51 2026 +0100
CAMEL-26925: camel-sftp - use remote file names literally when calling JSch
(#29848)
## Summary
[CAMEL-26925](https://issues.apache.org/jira/browse/CAMEL-26925)
The `sftp` component (`SftpOperations` in camel-ftp) passed remote paths as
they are to the JSch `ChannelSftp` methods that take a path. JSch reads `*` and
`?` in such a path as wildcards and `\` as an escape character, so a remote
file or directory name that contains `*` or `?` was matched against the names
in the directory instead of being used as is. Storing, retrieving, deleting,
renaming, changing the mode of, listing or changing to a directory could then
resolve the name to anot [...]
## Change
- `SftpOperations` passes the path through the existing `quotePathForJsch`
at each `ChannelSftp` call that interprets it: `put` (store and append, and
`storeFileDirectly`), `get` (both retrieve modes), `rm`, `rename` (both names),
`chmod` (file and directory), `cd`, and `ls` (the listings, the directory check
of `buildDirectory` and the directory listing of the existence check). Before,
only `lstat` and `readlink` were escaped.
- `mkdir` is left as it is, as JSch sends its path without interpreting it.
- The escaping happens at the `ChannelSftp` call only, so names, headers
and log messages in Camel are unchanged.
- `deleteFile` and the existence check (`getFileAttributes`) normalize the
name with the endpoint's `separator`, as `storeFile` already did. With the
default `separator=UNIX`, a `\` is a path separator in every operation, so the
delete and the check work on the same file as the store and the rename.
- A stepwise retrieve (both modes) and a stepwise store change back to the
directory they started in also when they fail, in a `finally` block, so a
failed `get` no longer leaves the channel in the directory of the file for the
next poll.
- Upgrade guide: names with `*` or `?` are now used as given. The migration
advice covers a consumer with `useList=false` and a wildcard in `fileName` (use
`include`/`antInclude`), the producer (use the real name), and a wildcard in
the endpoint directory. Entries for 4.24, 4.22.3 and 4.18.6, all on main.
## Tests
New `SftpFileNameWithSpecialCharactersIT` (embedded SFTP server of
camel-test-infra-ftp, 24 cases). `storeFileInDirectoryWithWildcardName` runs
with `existDirCheckUsingLs` on and off (the `cd` check).
- `storeFileWithWildcardInName`: a name with `?` or `*` is stored under
that name, and another file whose name matches keeps its content (stepwise on
and off).
- `storeFileInDirectoryWithWildcardName`: a `CamelFileName` whose directory
part is `.?` or `.*` is stored in a directory with that name under the endpoint
directory, with the default options.
- `tempPrefixWithBackslashInDirectoryNameWhenServerDoesNotAnswerLstat`:
with `tempPrefix`, when the existence check of the target falls back to a
directory listing, a name with backslashes is looked up and replaced where the
store puts it (`inbox/keep.txt`), and the file outside the endpoint directory
keeps its content.
- `chmodOnlyChangesStoredFile`: `chmod` sets the mode of the stored file
and leaves the other file's mode alone.
- `chmodDirectoryOnlyChangesCreatedDirectory`: `chmodDirectory` sets the
mode of the created `d?` directory only.
- `appendToFileWithWildcardInName`: `fileExist=Append` appends to the file
with the given name only (stepwise on and off).
- `checksumFileOfFileWithWildcardInName`: the checksum file
(`storeFileDirectly`) is written next to the given file only.
- `storeAgainWithBackslashInNameKeepsFileWithThatLiteralName`: with
`tempPrefix`, a second store of `sub\file.txt` replaces `sub/file.txt`, and a
file literally named `sub\file.txt` is not deleted.
- `consumeFileWithWildcardInNameToLocalWorkDirectory`: `localWorkDirectory`
retrieves the file with the given name.
- `failedStepwiseRetrieveChangesBackToTheStartingDirectory`: with
`useList=false`, `fileName=data-*.txt` and
`ignoreFileNotFoundOrPermissionError=true`, the polls whose `get` fails leave
the directory unchanged, so the file literally named `data-*.txt` is picked up
once it appears.
- `consumeAndDeleteFileWithWildcardInName`: the consumer retrieves and
deletes the file with the given name only, with `download` on and off and
stepwise on and off.
- `consumeAndMoveFileWithWildcardInName`: the consumer renames only the
file with the given name into the `move` directory (both names of `rename`),
with stepwise on and off; the other files of both directories are unchanged.
The original cases fail on main.
`storeAgainWithBackslashInNameKeepsFileWithThatLiteralName` and
`failedStepwiseRetrieveChangesBackToTheStartingDirectory` fail without the
follow-up change (normalized delete, directory restored in `finally`). All 24
pass with it. All 167 SFTP integration tests of camel-ftp (`Sftp*IT`) pass.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.../component/file/remote/SftpOperations.java | 113 +++++---
.../SftpFileNameWithSpecialCharactersIT.java | 306 +++++++++++++++++++++
.../ROOT/pages/camel-4x-upgrade-guide-4_18.adoc | 9 +
.../ROOT/pages/camel-4x-upgrade-guide-4_22.adoc | 9 +
.../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc | 17 ++
5 files changed, 413 insertions(+), 41 deletions(-)
diff --git
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java
index d341e3892170..59a6e1c1ff62 100644
---
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java
+++
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java
@@ -663,9 +663,11 @@ public class SftpOperations
public boolean deleteFile(String name) throws
GenericFileOperationFailedException {
lock.lock();
try {
+ // normalize the name as the store does, so the file that is
deleted is the one that is checked and stored
+ name = endpoint.getConfiguration().normalizePath(name);
LOG.debug("Deleting file: {}", name);
reconnectIfNecessary(null);
- channel.rm(name);
+ channel.rm(quotePathForJsch(name));
return true;
} catch (SftpException e) {
LOG.debug("Cannot delete file {}: {}", name, e.getMessage(), e);
@@ -684,7 +686,7 @@ public class SftpOperations
// make use of the '/' separator because JSch expects this
// as the file separator even on Windows
to = FileUtil.compactPath(to, '/');
- channel.rename(from, to);
+ channel.rename(quotePathForJsch(from), quotePathForJsch(to));
return true;
} catch (SftpException e) {
LOG.debug("Cannot rename file from: {} to: {}", from, to, e);
@@ -713,10 +715,10 @@ public class SftpOperations
// maybe the full directory already exists
try {
if (cdCheck) {
- channel.cd(directory);
+ channel.cd(quotePathForJsch(directory));
} else {
// just do a fast listing
- channel.ls(directory, entry ->
ChannelSftp.LsEntrySelector.BREAK);
+ channel.ls(quotePathForJsch(directory), entry ->
ChannelSftp.LsEntrySelector.BREAK);
}
success = true;
} catch (SftpException e) {
@@ -887,7 +889,7 @@ public class SftpOperations
}
LOG.trace("Changing directory: {}", path);
try {
- channel.cd(path);
+ channel.cd(quotePathForJsch(path));
} catch (SftpException e) {
throw new GenericFileOperationFailedException("Cannot change
directory to: " + path, e);
}
@@ -932,7 +934,7 @@ public class SftpOperations
path = ".";
}
- Vector<?> files = channel.ls(path);
+ Vector<?> files = channel.ls(quotePathForJsch(path));
return files.stream()
.map(f -> new SftpRemoteFileJCraft((ChannelSftp.LsEntry)
f))
@@ -951,7 +953,7 @@ public class SftpOperations
try {
LOG.trace("Listing remote files from path {} one at a time", dir);
RuntimeException[] failure = { null };
- channel.ls(dir, entry -> {
+ channel.ls(quotePathForJsch(dir), entry -> {
try {
return visitor.test(new SftpRemoteFileJCraft(entry))
? ChannelSftp.LsEntrySelector.CONTINUE :
ChannelSftp.LsEntrySelector.BREAK;
@@ -1029,14 +1031,17 @@ public class SftpOperations
/**
* Escapes the characters that JSch interprets in a path argument, so that
the server acts on the literal path.
* <p/>
- * {@link ChannelSftp#lstat(String)} and {@link
ChannelSftp#readlink(String)} pass the path through JSch's
- * {@code isUnique}/{@code glob_remote}, which treats {@code *} and {@code
?} as wildcards and {@code \} as an
- * escape character and removes it. Without this, a listed link whose
target contains one of these characters (for
- * example a target that steps through a sibling symbolic link named with
a backslash) would check a different path
- * from the one the server opens when retrieving the file, which would let
the jail check be bypassed. This mirrors
- * JSch's own (package-private) {@code Util.quote}.
+ * The {@link ChannelSftp} methods that take a remote path, such as {@code
get}, {@code put}, {@code rm},
+ * {@code rename}, {@code chmod}, {@code cd}, {@code ls}, {@code lstat}
and {@code readlink}, resolve it with JSch's
+ * own pattern matching, which treats {@code *} and {@code ?} as wildcards
and {@code \} as an escape character that
+ * is removed. Every such call in this class escapes its path, so that the
server works on the file or directory
+ * with exactly the given name, also when it contains {@code *} or {@code
?}, as the other file based components do.
+ * A {@code \} is a path separator with the default {@code separator}
option, and the name is normalized to
+ * {@code /} before it gets here; a {@code \} that is left is escaped, so
that JSch does not remove it.
+ * {@link ChannelSftp#mkdir(String)} sends its path as it is, so it is not
escaped. This mirrors JSch's own
+ * (package-private) {@code Util.quote}.
*
- * @param path the literal path to look up
+ * @param path the literal path
* @return the path with {@code \}, {@code *} and {@code ?} escaped
for JSch
*/
private static String quotePathForJsch(String path) {
@@ -1099,8 +1104,9 @@ public class SftpOperations
@SuppressWarnings("unchecked")
private boolean retrieveFileToStreamInBody(String name, Exchange exchange)
throws GenericFileOperationFailedException {
+ String currentDir = null;
+ boolean retrieved = false;
try {
- String currentDir = null;
GenericFile<ChannelSftp.LsEntry> target
= (GenericFile<ChannelSftp.LsEntry>)
exchange.getProperty(FileComponent.FILE_EXCHANGE_FILE);
ObjectHelper.notNull(target, "Exchange should have the " +
FileComponent.FILE_EXCHANGE_FILE + " set");
@@ -1123,7 +1129,7 @@ public class SftpOperations
}
// use input stream which works with Apache SSHD used for testing
- InputStream is = channel.get(remoteName);
+ InputStream is = channel.get(quotePathForJsch(remoteName));
if (endpoint.getConfiguration().isStreamDownload()) {
target.setBody(is);
@@ -1139,17 +1145,16 @@ public class SftpOperations
}
createResultHeadersFromExchange(null, exchange);
-
- // change back to current directory if we changed directory
- if (currentDir != null) {
- changeCurrentDirectory(currentDir);
- }
+ retrieved = true;
return true;
} catch (SftpException e) {
createResultHeadersFromExchange(e, exchange);
throw new GenericFileOperationFailedException("Cannot retrieve
file: " + name, e);
} catch (IOException e) {
throw new GenericFileOperationFailedException("Cannot retrieve
file: " + name, e);
+ } finally {
+ // change back to current directory if we changed directory
+ changeBackToDirectory(currentDir, retrieved);
}
}
@@ -1205,8 +1210,9 @@ public class SftpOperations
} catch (Exception e) {
throw new GenericFileOperationFailedException("Cannot create new
local work file: " + local, e);
}
+ String currentDir = null;
+ boolean retrieved = false;
try {
- String currentDir = null;
// store the java.io.File handle as the body
file.setBody(local);
@@ -1227,13 +1233,8 @@ public class SftpOperations
remoteName = FileUtil.stripPath(name);
}
- channel.get(remoteName, os);
-
- // change back to current directory if we changed directory
- if (currentDir != null) {
- changeCurrentDirectory(currentDir);
- }
-
+ channel.get(quotePathForJsch(remoteName), os);
+ retrieved = true;
} catch (SftpException e) {
createResultHeadersFromExchange(e, exchange);
LOG.trace("Error occurred during retrieving file: {} to local
directory. Deleting local work file: {}", name, temp);
@@ -1249,6 +1250,8 @@ public class SftpOperations
throw new GenericFileOperationFailedException("Cannot retrieve
file: " + name, e);
} finally {
IOHelper.close(os, "retrieve: " + name, LOG);
+ // change back to current directory if we changed directory
+ changeBackToDirectory(currentDir, retrieved);
}
createResultHeadersFromExchange(null, exchange);
@@ -1296,12 +1299,14 @@ public class SftpOperations
targetName = FileUtil.stripPath(name);
}
- // store the file
- answer = doStoreFile(name, targetName, exchange);
-
- // change back to current directory if we changed directory
- if (currentDir != null) {
- changeCurrentDirectory(currentDir);
+ boolean stored = false;
+ try {
+ // store the file
+ answer = doStoreFile(name, targetName, exchange);
+ stored = true;
+ } finally {
+ // change back to current directory if we changed directory
+ changeBackToDirectory(currentDir, stored);
}
return answer;
@@ -1310,6 +1315,30 @@ public class SftpOperations
}
}
+ /**
+ * Changes back to the directory that a stepwise operation started in,
also when the operation failed, so that the
+ * next operation does not start in the directory of the file. When the
operation failed, a failure to change back
+ * is only logged, so that it does not hide why the operation failed.
+ *
+ * @param directory the directory to change back to, or {@code null} if
the operation did not change directory
+ * @param succeeded whether the operation succeeded
+ */
+ private void changeBackToDirectory(String directory, boolean succeeded) {
+ if (directory == null) {
+ return;
+ }
+ if (succeeded) {
+ changeCurrentDirectory(directory);
+ return;
+ }
+ try {
+ changeCurrentDirectory(directory);
+ } catch (GenericFileOperationFailedException e) {
+ LOG.debug("Cannot change back to directory: {} after a failed
operation due to: {}", directory, e.getMessage(),
+ e);
+ }
+ }
+
private boolean doStoreFile(String name, String targetName, Exchange
exchange) throws GenericFileOperationFailedException {
LOG.trace("doStoreFile({})", targetName);
@@ -1357,11 +1386,11 @@ public class SftpOperations
LOG.debug("About to store file: {} using stream: {}", targetName,
is);
if (endpoint.getFileExist() == GenericFileExist.Append) {
LOG.trace("Client appendFile: {}", targetName);
- channel.put(is, targetName, ChannelSftp.APPEND);
+ channel.put(is, quotePathForJsch(targetName),
ChannelSftp.APPEND);
} else {
LOG.trace("Client storeFile: {}", targetName);
// override is default
- channel.put(is, targetName);
+ channel.put(is, quotePathForJsch(targetName));
}
if (LOG.isDebugEnabled()) {
long time = watch.taken();
@@ -1375,7 +1404,7 @@ public class SftpOperations
// parse to int using 8bit mode
int permissions = Integer.parseInt(mode, 8);
LOG.trace("Setting chmod: {} on file: {}", mode, targetName);
- channel.chmod(permissions, targetName);
+ channel.chmod(permissions, quotePathForJsch(targetName));
}
createResultHeadersFromExchange(null, exchange);
@@ -1396,7 +1425,7 @@ public class SftpOperations
lock.lock();
ByteArrayInputStream bis = new
ByteArrayInputStream(payload.getBytes());
try {
- channel.put(bis, name);
+ channel.put(bis, quotePathForJsch(name));
return true;
} catch (SftpException e) {
throw new GenericFileOperationFailedException("Cannot store file:
" + name, e);
@@ -1435,6 +1464,8 @@ public class SftpOperations
public FileAttributes getFileAttributes(String name) throws
GenericFileOperationFailedException {
lock.lock();
try {
+ // normalize the name as the store does, so the file that is
checked is the one that is stored
+ name = endpoint.getConfiguration().normalizePath(name);
SftpATTRS attrs;
try {
attrs = lstat(name);
@@ -1516,7 +1547,7 @@ public class SftpOperations
try {
@SuppressWarnings("rawtypes")
- List files = channel.ls(directory);
+ List files = channel.ls(quotePathForJsch(directory));
// can return either null or an empty list depending on FTP servers
if (files == null) {
return null;
@@ -1653,7 +1684,7 @@ public class SftpOperations
// parse to int using 8bit mode
int permissions = Integer.parseInt(chmodDirectory, 8);
try {
- channel.chmod(permissions, directory);
+ channel.chmod(permissions, quotePathForJsch(directory));
} catch (SftpException e) {
throw new GenericFileOperationFailedException("Cannot set
permission on directory: " + directory, e);
}
diff --git
a/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/sftp/integration/SftpFileNameWithSpecialCharactersIT.java
b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/sftp/integration/SftpFileNameWithSpecialCharactersIT.java
new file mode 100644
index 000000000000..bbcfc43a327c
--- /dev/null
+++
b/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/sftp/integration/SftpFileNameWithSpecialCharactersIT.java
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.file.remote.sftp.integration;
+
+import java.io.IOException;
+import java.nio.file.AccessDeniedException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.PosixFilePermissions;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.camel.Consumer;
+import org.apache.camel.Endpoint;
+import org.apache.camel.Exchange;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.model.RouteDefinition;
+import org.apache.camel.spi.PollingConsumerPollStrategy;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.DisabledOnOs;
+import org.junit.jupiter.api.condition.EnabledIf;
+import org.junit.jupiter.api.condition.OS;
+import org.junit.jupiter.api.extension.RegisterExtension;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * The sftp component uses a remote file or directory name as it is, also when
it contains {@code *} or {@code ?}, which
+ * the JSch library would otherwise read as wildcards. A {@code \} is a path
separator, in every operation.
+ */
+@EnabledIf(value =
"org.apache.camel.test.infra.ftp.services.embedded.SftpUtil#hasRequiredAlgorithms('src/test/resources/hostkey.pem')")
+@DisabledOnOs(value = OS.WINDOWS, disabledReason = "Windows does not allow *,
? and \\ in file names")
+class SftpFileNameWithSpecialCharactersIT extends CamelTestSupport {
+
+ @RegisterExtension
+ SftpExistsFileIT.CountingSftpEmbeddedService service = new
SftpExistsFileIT.CountingSftpEmbeddedService();
+
+ private Path base;
+ private Path inbox;
+
+ @BeforeEach
+ void createTestDirectory() throws IOException {
+ // the root directory of the server is shared by the tests of this
class and kept between runs
+ base =
Files.createTempDirectory(Files.createDirectories(service.getFtpRootDir()),
"test");
+ inbox = Files.createDirectories(base.resolve("inbox"));
+ }
+
+ @ParameterizedTest
+ @CsvSource({ "data-?.txt, true", "data-*.txt, true", "data-?.txt, false",
"data-*.txt, false" })
+ void storeFileWithWildcardInName(String name, boolean stepwise) throws
Exception {
+ Files.writeString(inbox.resolve("data-1.txt"), "other");
+
+ template.sendBodyAndHeader(sftpUri("inbox") + "&stepwise=" + stepwise,
"stored", Exchange.FILE_NAME, name);
+
+ assertEquals("stored", Files.readString(inbox.resolve(name)));
+ assertEquals("other", Files.readString(inbox.resolve("data-1.txt")));
+ }
+
+ @ParameterizedTest
+ @CsvSource({ ".?, true", ".*, true", ".?, false", ".*, false" })
+ void storeFileInDirectoryWithWildcardName(String directory, boolean
existDirCheckUsingLs) throws Exception {
+ template.sendBodyAndHeader(sftpUri("inbox") + "&existDirCheckUsingLs="
+ existDirCheckUsingLs, "stored",
+ Exchange.FILE_NAME, directory + "/hello.txt");
+
+ assertEquals("stored",
Files.readString(inbox.resolve(directory).resolve("hello.txt")));
+ assertFalse(Files.exists(base.resolve("hello.txt")));
+ }
+
+ @Test
+ void chmodDirectoryOnlyChangesCreatedDirectory() throws Exception {
+ Path other = Files.createDirectories(inbox.resolve("d1"));
+ Set<PosixFilePermission> mode = Files.getPosixFilePermissions(other);
+
+ template.sendBodyAndHeader(sftpUri("inbox") + "&chmodDirectory=777",
"stored", Exchange.FILE_NAME, "d?/hello.txt");
+
+ assertEquals("stored",
Files.readString(inbox.resolve("d?").resolve("hello.txt")));
+ assertEquals(PosixFilePermissions.fromString("rwxrwxrwx"),
Files.getPosixFilePermissions(inbox.resolve("d?")));
+ assertEquals(mode, Files.getPosixFilePermissions(other));
+ }
+
+ @ParameterizedTest
+ @ValueSource(booleans = { true, false })
+ void appendToFileWithWildcardInName(boolean stepwise) throws Exception {
+ Files.writeString(inbox.resolve("data-?.txt"), "first ");
+ Files.writeString(inbox.resolve("data-1.txt"), "other");
+
+ template.sendBodyAndHeader(sftpUri("inbox") +
"&fileExist=Append&stepwise=" + stepwise, "second",
+ Exchange.FILE_NAME, "data-?.txt");
+
+ assertEquals("first second",
Files.readString(inbox.resolve("data-?.txt")));
+ assertEquals("other", Files.readString(inbox.resolve("data-1.txt")));
+ }
+
+ @Test
+ void checksumFileOfFileWithWildcardInName() throws Exception {
+ Files.writeString(inbox.resolve("data-1.txt.MD5"), "other");
+
+ template.sendBodyAndHeader(sftpUri("inbox") +
"&checksumFileAlgorithm=MD5", "stored", Exchange.FILE_NAME,
+ "data-?.txt");
+
+ assertEquals("stored", Files.readString(inbox.resolve("data-?.txt")));
+ assertTrue(Files.exists(inbox.resolve("data-?.txt.MD5")));
+ assertEquals("other",
Files.readString(inbox.resolve("data-1.txt.MD5")));
+ }
+
+ @Test
+ void storeAgainWithBackslashInNameKeepsFileWithThatLiteralName() throws
Exception {
+ String uri = sftpUri("inbox") + "&tempPrefix=inprogress-";
+ template.sendBodyAndHeader(uri, "first", Exchange.FILE_NAME,
"sub\\file.txt");
+ // the \ is a path separator, so a file with this literal name is
another file
+ Files.writeString(inbox.resolve("sub\\file.txt"), "unrelated");
+
+ template.sendBodyAndHeader(uri, "second", Exchange.FILE_NAME,
"sub\\file.txt");
+
+ assertEquals("second",
Files.readString(inbox.resolve("sub").resolve("file.txt")));
+ assertEquals("unrelated",
Files.readString(inbox.resolve("sub\\file.txt")));
+ }
+
+ @Test
+ void tempPrefixWithBackslashInDirectoryNameWhenServerDoesNotAnswerLstat()
throws Exception {
+ Files.writeString(base.resolve("keep.txt"), "other");
+ // the existence check of the target then lists the directory of the
name
+ service.onLstat(() -> {
+ throw new AccessDeniedException("lstat");
+ });
+
+ template.sendBodyAndHeader(sftpUri("inbox") +
"&tempPrefix=inprogress-", "stored", Exchange.FILE_NAME,
+ "\\.\\./keep.txt");
+
+ assertEquals("stored", Files.readString(inbox.resolve("keep.txt")));
+ assertEquals("other", Files.readString(base.resolve("keep.txt")));
+ }
+
+ @ParameterizedTest
+ @ValueSource(booleans = { true, false })
+ void chmodOnlyChangesStoredFile(boolean stepwise) throws Exception {
+ Path other = inbox.resolve("data-1.txt");
+ Files.writeString(other, "other");
+ Set<PosixFilePermission> mode = Files.getPosixFilePermissions(other);
+
+ template.sendBodyAndHeader(sftpUri("inbox") + "&chmod=777&stepwise=" +
stepwise, "stored",
+ Exchange.FILE_NAME, "data-?.txt");
+
+ assertEquals("stored", Files.readString(inbox.resolve("data-?.txt")));
+ assertEquals(PosixFilePermissions.fromString("rwxrwxrwx"),
Files.getPosixFilePermissions(inbox.resolve("data-?.txt")));
+ assertEquals(mode, Files.getPosixFilePermissions(other));
+ }
+
+ @ParameterizedTest
+ @CsvSource({ "true, true", "true, false", "false, true", "false, false" })
+ void consumeAndDeleteFileWithWildcardInName(boolean download, boolean
stepwise) throws Exception {
+ Files.writeString(inbox.resolve("data-*.txt"), "wanted");
+ Files.writeString(inbox.resolve("data-1.txt"), "other");
+
+ MockEndpoint mock = getMockEndpoint("mock:result");
+ mock.expectedMessageCount(1);
+ mock.expectedHeaderReceived(Exchange.FILE_NAME, "data-*.txt");
+ if (download) {
+ mock.expectedBodiesReceived("wanted");
+ }
+
+ context.addRoutes(new RouteBuilder() {
+ @Override
+ public void configure() {
+ RouteDefinition route = from(sftpUri("inbox") +
"&fileName=data-*.txt&delete=true&initialDelay=0"
+ + "&delay=60000&download=" +
download + "&stepwise=" + stepwise);
+ if (download) {
+ route.convertBodyTo(String.class);
+ }
+ route.to("mock:result");
+ }
+ });
+
+ mock.assertIsSatisfied();
+ // the file is deleted once the exchange is done
+ await().atMost(10, TimeUnit.SECONDS).until(() ->
!Files.exists(inbox.resolve("data-*.txt")));
+ assertTrue(Files.exists(inbox.resolve("data-1.txt")));
+ }
+
+ @ParameterizedTest
+ @ValueSource(booleans = { true, false })
+ void consumeAndMoveFileWithWildcardInName(boolean stepwise) throws
Exception {
+ Files.writeString(inbox.resolve("data-?.txt"), "wanted");
+ Files.writeString(inbox.resolve("data-1.txt"), "other");
+ Path done = Files.createDirectories(inbox.resolve("done"));
+ Files.writeString(done.resolve("data-2.txt"), "done before");
+
+ MockEndpoint mock = getMockEndpoint("mock:result");
+ mock.expectedMessageCount(1);
+ mock.expectedHeaderReceived(Exchange.FILE_NAME, "data-?.txt");
+
+ context.addRoutes(new RouteBuilder() {
+ @Override
+ public void configure() {
+ from(sftpUri("inbox") +
"&fileName=data-?.txt&move=done/${file:name}&initialDelay=0"
+ + "&delay=60000&stepwise=" + stepwise)
+ .to("mock:result");
+ }
+ });
+
+ mock.assertIsSatisfied();
+ // the file is renamed into the done directory once the exchange is
done
+ await().atMost(10, TimeUnit.SECONDS).until(() ->
Files.exists(done.resolve("data-?.txt")));
+ assertEquals("wanted", Files.readString(done.resolve("data-?.txt")));
+ assertFalse(Files.exists(inbox.resolve("data-?.txt")));
+ assertEquals("other", Files.readString(inbox.resolve("data-1.txt")));
+ assertEquals("done before",
Files.readString(done.resolve("data-2.txt")));
+ }
+
+ @Test
+ void consumeFileWithWildcardInNameToLocalWorkDirectory() throws Exception {
+ Files.writeString(inbox.resolve("data-?.txt"), "wanted");
+ Files.writeString(inbox.resolve("data-1.txt"), "other");
+ Path work = Files.createTempDirectory(base, "work");
+
+ MockEndpoint mock = getMockEndpoint("mock:result");
+ mock.expectedBodiesReceived("wanted");
+ mock.expectedHeaderReceived(Exchange.FILE_NAME, "data-?.txt");
+
+ context.addRoutes(new RouteBuilder() {
+ @Override
+ public void configure() {
+ from(sftpUri("inbox") +
"&fileName=data-?.txt&noop=true&initialDelay=0&delay=60000"
+ + "&localWorkDirectory=" + work)
+ .convertBodyTo(String.class).to("mock:result");
+ }
+ });
+
+ mock.assertIsSatisfied();
+ assertEquals("other", Files.readString(inbox.resolve("data-1.txt")));
+ }
+
+ @Test
+ void failedStepwiseRetrieveChangesBackToTheStartingDirectory() throws
Exception {
+ // the endpoint directory is relative to the home directory of the
server, so the consumer changes to it
+ // from the directory it started in, at every poll
+ Files.writeString(inbox.resolve("data-1.txt"), "other");
+ AtomicInteger polls = new AtomicInteger();
+ context.getRegistry().bind("countingPollStrategy", new
PollingConsumerPollStrategy() {
+ @Override
+ public boolean begin(Consumer consumer, Endpoint endpoint) {
+ polls.incrementAndGet();
+ return true;
+ }
+
+ @Override
+ public void commit(Consumer consumer, Endpoint endpoint, int
polledMessages) {
+ // noop
+ }
+
+ @Override
+ public boolean rollback(Consumer consumer, Endpoint endpoint, int
retryCounter, Exception e) {
+ return false;
+ }
+ });
+
+ MockEndpoint mock = getMockEndpoint("mock:result");
+ mock.expectedBodiesReceived("wanted");
+
+ context.addRoutes(new RouteBuilder() {
+ @Override
+ public void configure() {
+ from(sftpUri("inbox") +
"&useList=false&fileName=data-*.txt&ignoreFileNotFoundOrPermissionError=true"
+ +
"&stepwise=true&initialDelay=0&delay=100&pollStrategy=#countingPollStrategy")
+ .convertBodyTo(String.class).to("mock:result");
+ }
+ });
+
+ // the retrieve of data-*.txt fails at these polls, as there is no
file with that name
+ await().atMost(10, TimeUnit.SECONDS).until(() -> polls.get() >= 3);
+ Files.writeString(inbox.resolve("data-*.txt"), "wanted");
+
+ mock.assertIsSatisfied();
+ assertEquals("other", Files.readString(inbox.resolve("data-1.txt")));
+ }
+
+ private String sftpUri(String directory) {
+ return "sftp://localhost:" + service.getPort() + "/" + base + "/" +
directory
+ + "?username=admin&password=admin&knownHostsFile=" +
service.getKnownHostsFile();
+ }
+}
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_18.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_18.adoc
index c81d2431cf46..e940d28f4784 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_18.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_18.adoc
@@ -77,6 +77,15 @@ at `DEBUG` level with the message only. Checkpoint failures
triggered by batch s
The webhook consumer still logs the body it validates at `DEBUG`, but no
longer the signature from the `X-Signature` header, nor the signature it
calculates.
The `InvalidMessageSignatureException` of a failed check no longer carries the
calculated signature: `getCalculatedSignature()` returns `null` and is
deprecated, as is the constructor that takes it.
+=== camel-ftp - sftp uses remote file and directory names as given
+
+The `sftp` component now uses a remote file or directory name exactly as
given, also when it contains `*` or `?`.
+Before, the JSch library read `*` and `?` in the name as wildcards, so
storing, retrieving, deleting, renaming, listing
+or changing to a directory could resolve the name to another file or directory
whose name matched. If you relied on a
+wildcard to select a file, use the real name of the file in the producer, and
the `include` or `antInclude` option of a
+consumer that lists the directory instead of a wildcard in `fileName` with
`useList=false`. See the
+xref:components::sftp-component.adoc[SFTP component].
+
=== camel-hazelcast - the hazelcast-seda consumer restores a transferred
exchange only with transferExchange=true
A `hazelcast-seda` consumer now restores the headers and exchange properties
of an exchange that a producer with
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc
index cd163d4189af..08770f5ee83e 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc
@@ -98,6 +98,15 @@ configurations are not affected.
The webhook consumer still logs the body it validates at `DEBUG`, but no
longer the signature from the `X-Signature` header, nor the signature it
calculates.
The `InvalidMessageSignatureException` of a failed check no longer carries the
calculated signature: `getCalculatedSignature()` returns `null` and is
deprecated, as is the constructor that takes it.
+=== camel-ftp - sftp uses remote file and directory names as given
+
+The `sftp` component now uses a remote file or directory name exactly as
given, also when it contains `*` or `?`.
+Before, the JSch library read `*` and `?` in the name as wildcards, so
storing, retrieving, deleting, renaming, listing
+or changing to a directory could resolve the name to another file or directory
whose name matched. If you relied on a
+wildcard to select a file, use the real name of the file in the producer, and
the `include` or `antInclude` option of a
+consumer that lists the directory instead of a wildcard in `fileName` with
`useList=false`. See the
+xref:components::sftp-component.adoc[SFTP component].
+
=== camel-hazelcast - the hazelcast-seda consumer restores a transferred
exchange only with transferExchange=true
A `hazelcast-seda` consumer now restores the headers and exchange properties
of an exchange that a producer with
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index a4cbe9df8faa..69196936a079 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -323,6 +323,23 @@ support it on the path, the component lists the directory
as before. `readLock=c
modification time of a symbolic link itself, as the directory listing reported
them. See the
xref:components::sftp-component.adoc[SFTP component].
+=== camel-ftp - sftp uses remote file and directory names as given
+
+The `sftp` component now uses a remote file or directory name exactly as
given, also when it contains `*` or `?`.
+Before, the name was passed on to the JSch library, which reads `*` and `?` as
wildcards, so storing, retrieving,
+deleting, renaming, listing or changing to a directory could resolve the name
to another file or directory whose name
+matched, or fail when more than one name matched. With the default
`separator=UNIX`, a `\` in a name is a path
+separator, as before.
+
+If you relied on a wildcard to select a file or a directory:
+
+* A consumer with `useList=false` and a wildcard in `fileName`: list the
directory instead, and select the files with
+the `include` or `antInclude` option.
+* A producer with a wildcard in the `CamelFileName` header or the `fileName`
option: use the real name of the file.
+* A wildcard in the directory of the endpoint: use the real name of the
directory.
+
+See the xref:components::sftp-component.adoc[SFTP component].
+
=== camel-kafka - exactlyOnce: a rejected send that a doCatch catches skips
the record
With `exactlyOnce`, a Kafka send through the source producer after an earlier
send failed in the same transaction is