Federico Mariani created CAMEL-26654:
----------------------------------------
Summary: camel-azure-storage-datalake - getFile overwrites a File,
Path or GenericFile body with the remote file
Key: CAMEL-26654
URL: https://issues.apache.org/jira/browse/CAMEL-26654
Project: Camel
Issue Type: Bug
Components: camel-azure
Affects Versions: 4.18.5, 4.23.0, 4.22.2
Reporter: Federico Mariani
h3. Problem
The producer operation {{getFile}} decides whether to download into the message
body with {{message.getBody(OutputStream.class)}}:
{{components/camel-azure/camel-azure-storage-datalake/src/main/java/org/apache/camel/component/azure/storage/datalake/operations/DataLakeFileOperations.java}}
line 72 on main (commit 7978db3d42dd). That is a type conversion, not a type
check:
* a {{java.io.File}} or {{java.nio.file.Path}} body converts to a buffered
output stream that truncates the file ({{IOConverter.toOutputStream(File)}} and
{{toOutputStream(Path)}});
* a {{GenericFile}} body, which is what a {{file:}} consumer sets, converts the
same way through the fallback of {{GenericFileConverter}}.
{{getFile}} then downloads the Data Lake file into that stream. So
{{from("file:in").to("azure-storage-datalake:account/fs?fileName=a.txt&operation=getFile")}}
empties the consumed local file and replaces its content with the remote file,
instead of returning the remote file as an {{InputStream}}. The route was read
from source; the operation itself is measured below. The component
documentation says the download goes to "an {{OutputStream}} supplied as the
message body" ({{azure-storage-datalake-component.adoc}} lines 419 and 425).
CAMEL-25933 fixed the same defect in camel-azure-storage-blob {{getBlob}}
(commit 631b74151c65, main only): {{BlobOperations.java}} lines 106-108 now
only use a body that *is* an {{OutputStream}}. That change did not touch
camel-azure-storage-datalake. The datalake consumer is not affected, because
its exchange body is empty when it calls {{getFile}}.
h3. How to reproduce
A unit test (measured on main) calls {{DataLakeFileOperations.getFile}} with a
mocked {{DataLakeFileClientWrapper}}. The mock's {{downloadWithResponse}}
writes {{"remote"}} into the stream it is given, and its {{openInputStream()}}
returns {{"remote"}}:
{code:java}
Path local = Files.writeString(dir.resolve("consumed.txt"), "local file
content");
exchange.getIn().setBody(FileConsumer.asGenericFile(dir.toString(),
local.toFile(), null, false)); // or local.toFile(), or local
new DataLakeFileOperations(configuration, client).getFile(exchange);
Files.readString(local); // "remote"
{code}
||Body||Path taken by getFile||Local file afterwards||
|{{java.io.File}}|download into a {{BufferedOutputStream}} over the
file|{{remote}}|
|{{GenericFile}} (as set by a file consumer)|download into a
{{BufferedOutputStream}} over the file|{{remote}}|
|{{java.nio.file.Path}}|download into a {{BufferedOutputStream}} over the
file|{{remote}}|
|control: {{ByteArrayOutputStream}}|download into that stream (expected)|n/a|
|control: no body|{{openInputStream()}}, an {{InputStream}} is returned
(expected)|n/a|
h3. Proposed change
* Do what CAMEL-25933 did for blob: use the body only when it is an
{{OutputStream}} ({{message.getBody() instanceof OutputStream os ? os :
null}}). Otherwise return the {{InputStream}} from {{openInputStream}}, as for
any other body.
* Add tests with {{File}}, {{Path}} and {{GenericFile}} bodies, plus the
{{OutputStream}} and no-body controls.
* To save a file locally, users keep the {{downloadToFile}} operation (with
{{fileDir}}), or send the {{InputStream}} to a {{file:}} endpoint.
h3. Impact / upgrade
* Behaviour change for routes that relied on a {{File}}, {{Path}} or
{{GenericFile}} body being overwritten with the remote file. This needs an
entry in the 4.24 upgrade guide, worded like the existing
"camel-azure-storage-blob - getBlob writes only into an OutputStream body"
entry.
* Affected: 4.18.5, 4.22.2 and 4.23.0 (the same line at each release tag). The
line has been there since the component was created (CAMEL-15341).
* Only the producer operation {{getFile}} is affected.
* The blob fix (CAMEL-25933) is on main only. If this is backported to 4.18.x
and 4.22.x, the blob fix probably should be too.
h3. Related
* CAMEL-25933 (631b74151c65): the same fix for camel-azure-storage-blob
{{getBlob}}.
* CAMEL-26583: the latest change to the same {{getFile}} method (ranges on
streaming downloads).
* CAMEL-15341: the component's creation.
_Claude Code on behalf of Federico Mariani_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
