This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 9972ed33b1f4 CAMEL-26654: camel-azure-storage-datalake - getFile
writes only into an OutputStream body (#29268)
9972ed33b1f4 is described below
commit 9972ed33b1f41e4780be86fe28c2db2ea693a75f
Author: Federico Mariani <[email protected]>
AuthorDate: Thu Nov 5 20:50:15 2026 +0100
CAMEL-26654: camel-azure-storage-datalake - getFile writes only into an
OutputStream body (#29268)
getFile used message.getBody(OutputStream.class), a type conversion: a
java.io.File, a java.nio.file.Path or the GenericFile of a file consumer
converted to a stream that truncates the file, and the remote file was
then downloaded into it. Use the body only when it is an OutputStream,
as CAMEL-25933 did for getBlob in camel-azure-storage-blob; any other
body gets an InputStream of the file.
Co-Authored-By: Claude <[email protected]>
---
.../docs/azure-storage-datalake-component.adoc | 7 +-
.../docs/azure-storage-datalake-component.adoc | 7 +-
.../operations/DataLakeFileOperations.java | 9 +--
.../operations/DataLakeFileOperationTest.java | 89 ++++++++++++++++++++++
.../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc | 8 ++
5 files changed, 108 insertions(+), 12 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/azure-storage-datalake-component.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/azure-storage-datalake-component.adoc
index a21964ee4631..71c222d6f59d 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/azure-storage-datalake-component.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/azure-storage-datalake-component.adoc
@@ -150,7 +150,7 @@ For these operations, `accountName`, `fileSystemName` and
`fileName` options are
[width="100%", cols="10%,90%", options="header",]
|===
|Operation |Description
-|`getFile` | Get the contents of a file
+|`getFile` | Get the contents of a file. The content is written to the body
when the body is an `OutputStream`; any other body is replaced by an
`InputStream` of the content.
|`downloadToFile` | Download the entire file from the file system into a path
specified by fileDir.
|`downloadLink` | Generate a download link for the specified file using Shared
Access Signature (SAS).
The expiration time to be set for the link can be specified otherwise 1 hour
is taken as default.
@@ -435,8 +435,9 @@ from("direct:start")
.to("mock:results");
----
-Or if the body is not set, the operation will give an `InputStream`, given
that you have already registered for query acceleration
-in azure portal.
+Only a body that is an `OutputStream` is written to. Any other body, or no
body, is replaced by an `InputStream` of the file content,
+which can be processed further downstream. So a `File`, `Path` or the file of
a file consumer in the body is not written to:
+to save a file locally, use the `downloadToFile` operation, or a `file`
endpoint after `getFile`.
._Java-only: reads the file content as an `InputStream`_
[source,java]
diff --git
a/components/camel-azure/camel-azure-storage-datalake/src/main/docs/azure-storage-datalake-component.adoc
b/components/camel-azure/camel-azure-storage-datalake/src/main/docs/azure-storage-datalake-component.adoc
index a21964ee4631..71c222d6f59d 100644
---
a/components/camel-azure/camel-azure-storage-datalake/src/main/docs/azure-storage-datalake-component.adoc
+++
b/components/camel-azure/camel-azure-storage-datalake/src/main/docs/azure-storage-datalake-component.adoc
@@ -150,7 +150,7 @@ For these operations, `accountName`, `fileSystemName` and
`fileName` options are
[width="100%", cols="10%,90%", options="header",]
|===
|Operation |Description
-|`getFile` | Get the contents of a file
+|`getFile` | Get the contents of a file. The content is written to the body
when the body is an `OutputStream`; any other body is replaced by an
`InputStream` of the content.
|`downloadToFile` | Download the entire file from the file system into a path
specified by fileDir.
|`downloadLink` | Generate a download link for the specified file using Shared
Access Signature (SAS).
The expiration time to be set for the link can be specified otherwise 1 hour
is taken as default.
@@ -435,8 +435,9 @@ from("direct:start")
.to("mock:results");
----
-Or if the body is not set, the operation will give an `InputStream`, given
that you have already registered for query acceleration
-in azure portal.
+Only a body that is an `OutputStream` is written to. Any other body, or no
body, is replaced by an `InputStream` of the file content,
+which can be processed further downstream. So a `File`, `Path` or the file of
a file consumer in the body is not written to:
+to save a file locally, use the `downloadToFile` operation, or a `file`
endpoint after `getFile`.
._Java-only: reads the file content as an `InputStream`_
[source,java]
diff --git
a/components/camel-azure/camel-azure-storage-datalake/src/main/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperations.java
b/components/camel-azure/camel-azure-storage-datalake/src/main/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperations.java
index 117e18a85815..cd8dbc1bfb58 100644
---
a/components/camel-azure/camel-azure-storage-datalake/src/main/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperations.java
+++
b/components/camel-azure/camel-azure-storage-datalake/src/main/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperations.java
@@ -65,12 +65,9 @@ public class DataLakeFileOperations {
public DataLakeOperationResponse getFile(final Exchange exchange) throws
IOException {
final Message message = DataLakeUtils.getInMessage(exchange);
- final OutputStream outputStream;
- if (ObjectHelper.isEmpty(message)) {
- outputStream = null;
- } else {
- outputStream = message.getBody(OutputStream.class);
- }
+ // only an OutputStream body is written to, not a body that converts
to one (such as a File, which would be emptied)
+ final OutputStream outputStream
+ = message != null && message.getBody() instanceof OutputStream
os ? os : null;
final DataLakeFileClientWrapper fileClientWrapper =
getFileClientWrapper(exchange);
final FileRange fileRange = configurationProxy.getFileRange(exchange);
diff --git
a/components/camel-azure/camel-azure-storage-datalake/src/test/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperationTest.java
b/components/camel-azure/camel-azure-storage-datalake/src/test/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperationTest.java
index d41945dd20b7..3197eb022494 100644
---
a/components/camel-azure/camel-azure-storage-datalake/src/test/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperationTest.java
+++
b/components/camel-azure/camel-azure-storage-datalake/src/test/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperationTest.java
@@ -18,10 +18,15 @@ package
org.apache.camel.component.azure.storage.datalake.operations;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
+import java.io.OutputStream;
import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
import java.time.OffsetDateTime;
import com.azure.core.http.HttpHeaders;
@@ -30,6 +35,9 @@ import com.azure.storage.file.datalake.DataLakeFileClient;
import
com.azure.storage.file.datalake.models.DataLakeFileOpenInputStreamResult;
import com.azure.storage.file.datalake.models.DataLakeRequestConditions;
import com.azure.storage.file.datalake.models.FileRange;
+import com.azure.storage.file.datalake.models.FileReadAsyncResponse;
+import com.azure.storage.file.datalake.models.FileReadHeaders;
+import com.azure.storage.file.datalake.models.FileReadResponse;
import com.azure.storage.file.datalake.models.PathInfo;
import com.azure.storage.file.datalake.models.PathProperties;
import com.azure.storage.file.datalake.options.DataLakeFileInputStreamOptions;
@@ -37,12 +45,14 @@ import org.apache.camel.Exchange;
import org.apache.camel.component.azure.storage.datalake.DataLakeConfiguration;
import org.apache.camel.component.azure.storage.datalake.DataLakeConstants;
import
org.apache.camel.component.azure.storage.datalake.client.DataLakeFileClientWrapper;
+import org.apache.camel.component.file.FileConsumer;
import org.apache.camel.support.DefaultExchange;
import org.apache.camel.test.junit6.CamelTestSupport;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.extension.ExtendWith;
+import org.junit.jupiter.api.io.TempDir;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
@@ -54,8 +64,11 @@ import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyBoolean;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.ArgumentMatchers.same;
+import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
@@ -65,6 +78,9 @@ import static org.mockito.Mockito.when;
@ExtendWith(MockitoExtension.class)
public class DataLakeFileOperationTest extends CamelTestSupport {
+ private static final String LOCAL_CONTENT = "local file content";
+ private static final String REMOTE_CONTENT = "remote file content";
+
private DataLakeConfiguration configuration;
@Mock
@@ -104,6 +120,79 @@ public class DataLakeFileOperationTest extends
CamelTestSupport {
assertNotNull(secondResponse.getHeaders());
}
+ @Test
+ void testGetFileWithFileBodyDoesNotWriteIntoTheFile(@TempDir Path dir)
throws IOException {
+ // a File body converts to an OutputStream, but only an OutputStream
body is written to
+ final Path file = Files.writeString(dir.resolve("local.txt"),
LOCAL_CONTENT);
+
+ assertGetFileDoesNotWriteIntoTheFile(file.toFile(), file);
+ }
+
+ @Test
+ void testGetFileWithPathBodyDoesNotWriteIntoTheFile(@TempDir Path dir)
throws IOException {
+ final Path file = Files.writeString(dir.resolve("local.txt"),
LOCAL_CONTENT);
+
+ assertGetFileDoesNotWriteIntoTheFile(file, file);
+ }
+
+ @Test
+ void testGetFileWithGenericFileBodyDoesNotWriteIntoTheFile(@TempDir Path
dir) throws IOException {
+ // the body of an exchange of a file consumer
+ final Path file = Files.writeString(dir.resolve("local.txt"),
LOCAL_CONTENT);
+
+
assertGetFileDoesNotWriteIntoTheFile(FileConsumer.asGenericFile(dir.toString(),
file.toFile(), null, false), file);
+ }
+
+ private void assertGetFileDoesNotWriteIntoTheFile(Object body, Path file)
throws IOException {
+ when(client.openInputStream()).thenReturn(new
ByteArrayInputStream(REMOTE_CONTENT.getBytes(StandardCharsets.UTF_8)));
+ // a stream over the file given to the download would get the remote
content
+ lenient().when(client.downloadWithResponse(any(), any(), any(), any(),
anyBoolean(), any()))
+ .thenAnswer(invocation -> download(invocation.getArgument(0)));
+
+ final Exchange exchange = new DefaultExchange(context);
+ exchange.getIn().setBody(body);
+
+ final DataLakeOperationResponse response = new
DataLakeFileOperations(configuration, client).getFile(exchange);
+
+ assertEquals(LOCAL_CONTENT, Files.readString(file));
+ assertEquals(REMOTE_CONTENT,
+ new String(((InputStream) response.getBody()).readAllBytes(),
StandardCharsets.UTF_8));
+ verify(client, never()).downloadWithResponse(any(), any(), any(),
any(), anyBoolean(), any());
+ }
+
+ @Test
+ void testGetFileWithOutputStreamBodyWritesIntoTheStream() throws
IOException {
+ final ByteArrayOutputStream body = new ByteArrayOutputStream();
+ when(client.downloadWithResponse(same(body), any(), any(), any(),
anyBoolean(), any()))
+ .thenAnswer(invocation -> download(invocation.getArgument(0)));
+
+ final Exchange exchange = new DefaultExchange(context);
+ exchange.getIn().setBody(body);
+
+ final DataLakeOperationResponse response = new
DataLakeFileOperations(configuration, client).getFile(exchange);
+
+ assertSame(body, response.getBody());
+ assertEquals(REMOTE_CONTENT, body.toString(StandardCharsets.UTF_8));
+ verify(client, never()).openInputStream();
+ }
+
+ @Test
+ void testGetFileWithoutBodyReturnsAnInputStream() throws IOException {
+ when(client.openInputStream()).thenReturn(new
ByteArrayInputStream(REMOTE_CONTENT.getBytes(StandardCharsets.UTF_8)));
+
+ final DataLakeOperationResponse response
+ = new DataLakeFileOperations(configuration,
client).getFile(new DefaultExchange(context));
+
+ assertEquals(REMOTE_CONTENT,
+ new String(((InputStream) response.getBody()).readAllBytes(),
StandardCharsets.UTF_8));
+ verify(client, never()).downloadWithResponse(any(), any(), any(),
any(), anyBoolean(), any());
+ }
+
+ private static FileReadResponse download(OutputStream outputStream) throws
IOException {
+ outputStream.write(REMOTE_CONTENT.getBytes(StandardCharsets.UTF_8));
+ return new FileReadResponse(new FileReadAsyncResponse(null, 200, new
HttpHeaders(), null, new FileReadHeaders()));
+ }
+
@Test
void testDownloadToFileRejectsPathTraversalName() {
// a file name is influenced by whoever writes to the file system and
may contain ../ segments;
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index 8d76abb015b2..50ee81f51184 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -5084,6 +5084,14 @@ Remove the range header and options if your route needs
the entire file.
The streaming path also honors the `CamelAzureStorageDataLakeRequestCondition`
header, whether or not a range
is set. An ETag mismatch or an invalid lease can now fail a streaming read
that previously ignored the condition.
+=== camel-azure-storage-datalake - getFile writes only into an OutputStream
body
+
+The getFile operation now writes the file into the message body only when the
body is an `OutputStream`. A body that
+only converts to one, such as a `java.io.File`, a `java.nio.file.Path` or the
file of a file consumer, is replaced by an
+`InputStream` of the file, as any other body is. Previously such a file was
overwritten with the remote file, also with
+`noop=true` on the file consumer. To save a file locally, use the
downloadToFile operation, or a `file` endpoint after
+getFile.
+
=== camel-chatscript - the first message of a conversation is sent, and the
headers are kept
The first message of a ChatScript endpoint, and every message with
`resetChat=true`, used to start the conversation
