This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 09d11b37b2cb CAMEL-27267: camel-core - the binary data type keeps a
File or WrappedFile body instead of reading it into memory (#30472)
09d11b37b2cb is described below
commit 09d11b37b2cbccc733c768cb8022115910604574
Author: Claus Ibsen <[email protected]>
AuthorDate: Fri Nov 20 09:23:32 2026 +0100
CAMEL-27267: camel-core - the binary data type keeps a File or WrappedFile
body instead of reading it into memory (#30472)
The binary (application-octet-stream) data type kept a stream cache body
since CAMEL-27258, but still read a java.io.File or a WrappedFile body,
such as the GenericFile of the file and FTP consumers, into a byte[]. A
file is binary already, can be read again and may be large: keep it as it
is and only set the Content-Type, so a binary default on a file or object
sink Kamelet does not read every file into memory. transformerOut of the
@BodyTypes lists file, and the 4.24 upgrade guide note says so.
A java.nio.file.Path body is kept too, as the file carrier includes it.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
---
.../transformers/application-octet-stream.json | 3 +-
.../transformer/application-octet-stream.json | 3 +-
.../transformer/ByteArrayDataTypeTransformer.java | 19 ++++++---
.../ByteArrayDataTypeTransformerTest.java | 49 +++++++++++++++++++++-
.../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc | 15 +++----
5 files changed, 73 insertions(+), 16 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/transformers/application-octet-stream.json
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/transformers/application-octet-stream.json
index 5ca07d00c114..38d52fb8f03c 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/transformers/application-octet-stream.json
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/transformers/application-octet-stream.json
@@ -24,9 +24,10 @@
"carriers": [
"bytes",
"stream",
+ "file",
"null"
],
- "description": "A stream cache body is kept as it is and a null body
stays null; any other body becomes a byte array"
+ "description": "A stream cache or a file body is kept as it is and a
null body stays null; any other body becomes a byte array"
}
}
}
diff --git
a/core/camel-core-processor/src/generated/resources/META-INF/services/org/apache/camel/transformer/application-octet-stream.json
b/core/camel-core-processor/src/generated/resources/META-INF/services/org/apache/camel/transformer/application-octet-stream.json
index 5ca07d00c114..38d52fb8f03c 100644
---
a/core/camel-core-processor/src/generated/resources/META-INF/services/org/apache/camel/transformer/application-octet-stream.json
+++
b/core/camel-core-processor/src/generated/resources/META-INF/services/org/apache/camel/transformer/application-octet-stream.json
@@ -24,9 +24,10 @@
"carriers": [
"bytes",
"stream",
+ "file",
"null"
],
- "description": "A stream cache body is kept as it is and a null body
stays null; any other body becomes a byte array"
+ "description": "A stream cache or a file body is kept as it is and a
null body stays null; any other body becomes a byte array"
}
}
}
diff --git
a/core/camel-core-processor/src/main/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformer.java
b/core/camel-core-processor/src/main/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformer.java
index 7d734cbe40bc..ca3be6016b67 100644
---
a/core/camel-core-processor/src/main/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformer.java
+++
b/core/camel-core-processor/src/main/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformer.java
@@ -16,9 +16,13 @@
*/
package org.apache.camel.processor.transformer;
+import java.io.File;
+import java.nio.file.Path;
+
import org.apache.camel.Exchange;
import org.apache.camel.Message;
import org.apache.camel.StreamCache;
+import org.apache.camel.WrappedFile;
import org.apache.camel.spi.BodyType;
import org.apache.camel.spi.BodyType.Carrier;
import org.apache.camel.spi.BodyTypes;
@@ -28,23 +32,26 @@ import org.apache.camel.spi.Transformer;
/**
* Generic binary data type uses Camel message body converter mechanism to
convert content to byte array representation.
- * A stream cache body is binary already and can be read again, so it is kept
as it is: a large body, such as an object
- * read from a bucket, is not read into memory. Any other stream is read into
a byte array, as it may be read only once.
+ * A stream cache or a file body is binary already and can be read again, so
it is kept as it is: a large body, such as
+ * an object read from a bucket or a file to upload, is not read into memory.
Any other stream is read into a byte
+ * array, as it may be read only once.
*/
@DataTypeTransformer(name = "application-octet-stream")
@BodyTypes(transformerIn = @BodyType(value = { Carrier.TEXT, Carrier.BYTES,
Carrier.STREAM, Carrier.FILE, Carrier.NULL },
description = "A body Camel can convert
to a byte array: a number fails, and a null"
+ " body stays null"),
- transformerOut = @BodyType(value = { Carrier.BYTES, Carrier.STREAM,
Carrier.NULL },
- description = "A stream cache body is
kept as it is and a null body stays null;"
- + " any other body becomes
a byte array"))
+ transformerOut = @BodyType(value = { Carrier.BYTES, Carrier.STREAM,
Carrier.FILE, Carrier.NULL },
+ description = "A stream cache or a file
body is kept as it is and a null body"
+ + " stays null; any other
body becomes a byte array"))
public class ByteArrayDataTypeTransformer extends Transformer {
private static final Transformer DELEGATE = new
TypeConverterTransformer(byte[].class);
@Override
public void transform(Message message, DataType from, DataType to) throws
Exception {
- if (!(message.getBody() instanceof StreamCache)) {
+ Object body = message.getBody();
+ if (!(body instanceof StreamCache || body instanceof File || body
instanceof Path
+ || body instanceof WrappedFile)) {
DELEGATE.transform(message, from, to);
}
diff --git
a/core/camel-core/src/test/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformerTest.java
b/core/camel-core/src/test/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformerTest.java
index 6cd3e927987f..e51451c8a1d7 100644
---
a/core/camel-core/src/test/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformerTest.java
+++
b/core/camel-core/src/test/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformerTest.java
@@ -17,12 +17,17 @@
package org.apache.camel.processor.transformer;
import java.io.ByteArrayInputStream;
+import java.io.File;
import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
import java.util.List;
import org.apache.camel.CamelExecutionException;
import org.apache.camel.Exchange;
import org.apache.camel.StreamCache;
+import org.apache.camel.component.file.FileConsumer;
+import org.apache.camel.component.file.GenericFile;
import org.apache.camel.converter.stream.ByteArrayInputStreamCache;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.impl.engine.DefaultTransformerRegistry;
@@ -34,6 +39,7 @@ import org.apache.camel.spi.TransformerKey;
import org.apache.camel.support.DefaultExchange;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
@@ -98,6 +104,46 @@ public class ByteArrayDataTypeTransformerTest {
assertEquals("test3.txt", exchange.getMessage().getHeader("file"));
}
+ @Test
+ void shouldKeepFile(@TempDir Path dir) throws Exception {
+ // a file is binary already and can be read again: it is kept, so a
large file is not read into memory
+ File file = Files.writeString(dir.resolve("test5.txt"),
"Test5").toFile();
+ Exchange exchange = new DefaultExchange(camelContext);
+ exchange.getMessage().setBody(file);
+ transformer.transform(exchange.getMessage(), DataType.ANY,
DataType.ANY);
+
+ assertSame(file, exchange.getMessage().getBody());
+ assertEquals("application/octet-stream",
exchange.getMessage().getHeader(Exchange.CONTENT_TYPE));
+ assertEquals("Test5", exchange.getMessage().getBody(String.class));
+ }
+
+ @Test
+ void shouldKeepPath(@TempDir Path dir) throws Exception {
+ // a Path is a file too, and is kept the same way
+ Path path = Files.writeString(dir.resolve("test7.txt"), "Test7");
+ Exchange exchange = new DefaultExchange(camelContext);
+ exchange.getMessage().setBody(path);
+ transformer.transform(exchange.getMessage(), DataType.ANY,
DataType.ANY);
+
+ assertSame(path, exchange.getMessage().getBody());
+ assertEquals("application/octet-stream",
exchange.getMessage().getHeader(Exchange.CONTENT_TYPE));
+ assertEquals("Test7", exchange.getMessage().getBody(String.class));
+ }
+
+ @Test
+ void shouldKeepWrappedFile(@TempDir Path dir) throws Exception {
+ // a file of the file consumer is kept the same way
+ File file = Files.writeString(dir.resolve("test6.txt"),
"Test6").toFile();
+ GenericFile<File> wrapped = FileConsumer.asGenericFile(dir.toString(),
file, null, false);
+ Exchange exchange = new DefaultExchange(camelContext);
+ exchange.getMessage().setBody(wrapped);
+ transformer.transform(exchange.getMessage(), DataType.ANY,
DataType.ANY);
+
+ assertSame(wrapped, exchange.getMessage().getBody());
+ assertEquals("application/octet-stream",
exchange.getMessage().getHeader(Exchange.CONTENT_TYPE));
+ assertEquals("Test6", exchange.getMessage().getBody(String.class));
+ }
+
@Test
void shouldReadOtherInputStreamToBytes() throws Exception {
// a stream that is not a stream cache may be read only once: it
becomes a byte array every later step can read
@@ -130,7 +176,8 @@ public class ByteArrayDataTypeTransformerTest {
BodyTypes bodyTypes =
ByteArrayDataTypeTransformer.class.getAnnotation(BodyTypes.class);
assertEquals(List.of(Carrier.TEXT, Carrier.BYTES, Carrier.STREAM,
Carrier.FILE, Carrier.NULL),
List.of(bodyTypes.transformerIn().value()));
- assertEquals(List.of(Carrier.BYTES, Carrier.STREAM, Carrier.NULL),
List.of(bodyTypes.transformerOut().value()));
+ assertEquals(List.of(Carrier.BYTES, Carrier.STREAM, Carrier.FILE,
Carrier.NULL),
+ List.of(bodyTypes.transformerOut().value()));
for (Object body : List.of("Test",
"Test".getBytes(StandardCharsets.UTF_8))) {
Exchange exchange = new DefaultExchange(camelContext);
exchange.getMessage().setBody(body);
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index 0b0aa0eaa084..fdd37e2227b3 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -241,13 +241,14 @@ it used to get the body of the component.
When no transformer for the type is on the classpath, such as `json` with
neither `camel-jackson` nor `camel-jackson3`,
the message is handed on unchanged, as before, and a WARN is logged once.
-=== camel-core - the binary data type keeps a stream cache body
-
-The `application-octet-stream` (`binary`) data type transformer keeps a body
that is a stream cache as it is, and only
-sets the `Content-Type` header to `application/octet-stream`. Before, it read
the stream into a `byte[]`. A stream cache
-is binary already and can be read again, and a source Kamelet such as
`aws-s3-source` or `azure-storage-blob-source`
-declares `binary` and gives the object as a stream, which may be large. Any
other body, including an `InputStream` that is
-not a stream cache (stream caching turned off), is still converted to a
`byte[]`.
+=== camel-core - the binary data type keeps a stream cache or a file body
+
+The `application-octet-stream` (`binary`) data type transformer keeps a body
that is a stream cache, a `java.io.File`, a
+`Path` or a `WrappedFile` (such as the `GenericFile` of the file and FTP
consumers) as it is, and only sets the `Content-Type`
+header to `application/octet-stream`. Before, it read the body into a
`byte[]`. These bodies are binary already and can
+be read again, and they may be large: a source Kamelet such as `aws-s3-source`
or `azure-storage-blob-source` declares
+`binary` and gives the object as a stream, and a file can be uploaded without
reading it into memory. Any other body,
+including an `InputStream` that is not a stream cache (stream caching turned
off), is still converted to a `byte[]`.
=== camel-kamelet - each Kamelet EIP has a Kamelet route of its own