Croway commented on code in PR #30453:
URL: https://github.com/apache/camel/pull/30453#discussion_r4634896618


##########
components/camel-kamelet/src/main/java/org/apache/camel/component/kamelet/KameletProducer.java:
##########
@@ -126,6 +141,37 @@ public boolean process(Exchange exchange, AsyncCallback 
originalCallback) {
         }
     }
 
+    /**
+     * The output data type of the source Kamelet whose route the exchange is 
in, from the route property that the
+     * Kamelet YAML loader sets, or null when it has none.
+     */
+    private DataTypeProcessor sourceOutputType(Exchange exchange) throws 
Exception {
+        if (!sourceOutputTypeResolved) {
+            synchronized (this) {
+                if (!sourceOutputTypeResolved) {
+                    Route route = ExchangeHelper.getRoute(exchange);
+                    Object type = route != null ? 
route.getProperties().get(Kamelet.SOURCE_OUTPUT_TYPE_PROPERTY) : null;
+                    if (type != null) {
+                        DataTypeProcessor processor = new 
DataTypeProcessor(null, type.toString());
+                        
processor.setCamelContext(getEndpoint().getCamelContext());
+                        ServiceHelper.startService(processor);

Review Comment:
   **An output data type whose transformer is not on the classpath now fails 
every message**
   
   `DataTypeProcessor.doInit` throws when no transformer resolves, so 
`startService` throws here on the first message. `sourceOutputTypeResolved` 
stays `false`, so every later message repeats the lookup inside `synchronized 
(this)` and fails the same way. Before, `ContractAdvice.after` found no 
transformer from `java:<body class>` to the type, took the body as already of 
that type, and the message went through.
   
   In the Kamelet catalog, `slack-source` and `aws-ddb-streams-source` default 
to `json` (`application-json`), whose transformer is in camel-jackson / 
camel-jackson3, while both Kamelets declare only `camel:gson`. An output type 
with the `java:` scheme, such as `java:java.lang.String`, fails the same way, 
because the transformer registry does not lazy-load Java types.
   
   Suggestion: keep the old outcome (log a WARN once and hand the message on 
unchanged), or resolve the type when the route starts and fail there with the 
name of the Kamelet. Either way, cache the outcome so the lookup does not run 
again for each message.
   
   <details><summary>Reproducer (a test for <code>KameletLoaderTest</code>, 
needs <code>import org.apache.camel.ProducerTemplate</code>)</summary>
   
   ```java
       @Test
       void sourceKameletOutputTypeWithoutTransformerDoesNotFailMessages() 
throws Exception {
           // the transformer of the format is not on the classpath 
(camel-slack is not a test dependency)
           loadKamelets("""
                   apiVersion: camel.apache.org/v1
                   kind: Kamelet
                   metadata:
                     name: my-ce-source
                     labels:
                       camel.apache.org/kamelet.type: source
                   spec:
                     definition:
                       title: "My Source"
                     dataTypes:
                       out:
                         default: cloudevents
                         types:
                           cloudevents:
                             format: "slack:application-cloudevents"
                     template:
                       from:
                         uri: "direct:feed"
                         steps:
                           - to: "kamelet:sink"
                   """);
           loadRoutes("""
                   - from:
                       uri: "kamelet:my-ce-source"
                       steps:
                         - to: "mock:ce"
                   """);
   
           try (ProducerTemplate producer = context.createProducerTemplate()) {
               for (int i = 0; i < 2; i++) {
                   Exchange out = producer.send("direct:feed", e -> 
e.getMessage().setBody("hello"));
                   assertThat(out.getException()).isNull();
               }
           }
           assertThat(context.getEndpoint("mock:ce", 
MockEndpoint.class).getReceivedCounter()).isEqualTo(2);
       }
   ```
   
   PR head (028a8e6):
   
   ```
   org.opentest4j.AssertionFailedError:
   expected:
     null
    but was:
     java.lang.IllegalArgumentException: No transformer from: null 
to:slack:application-cloudevents
   ```
   
   main (5523f54): passes, `mock:ce` gets both messages. Without the assertion, 
on the PR head the second message fails the same way and `mock:ce` gets none. 
With `format: "java:java.lang.String"` and a body of `42`, main delivers the 
message and the PR head fails it with `No transformer from: null 
to:java:java.lang.String`.
   
   </details>



##########
core/camel-core-processor/src/main/java/org/apache/camel/processor/transformer/ByteArrayDataTypeTransformer.java:
##########
@@ -27,19 +29,25 @@
 
 /**
  * Generic binary data type uses Camel message body converter mechanism to 
convert content to byte array representation.
+ * A stream body is binary already, so it is kept as it is: a large body, such 
as an object read from a bucket, is not
+ * read into memory.
  */
 @DataTypeTransformer(name = "application-octet-stream")
 @BodyTypes(transformerIn = @BodyType(value = { Carrier.TEXT, Carrier.BYTES, 
Carrier.STREAM, Carrier.FILE },
                                      description = "A body Camel can convert 
to a byte array: a number or a null body"
                                                    + " fails"),
-           transformerOut = @BodyType(value = Carrier.BYTES, javaType = 
"byte[]"))
+           transformerOut = @BodyType(value = { Carrier.BYTES, Carrier.STREAM 
},
+                                      description = "A stream body is kept as 
it is; any other body becomes a byte"
+                                                    + " array"))
 public class ByteArrayDataTypeTransformer extends Transformer {
 
     private static final Transformer DELEGATE = new 
TypeConverterTransformer(byte[].class);
 
     @Override
     public void transform(Message message, DataType from, DataType to) throws 
Exception {
-        DELEGATE.transform(message, from, to);
+        if (!(message.getBody() instanceof InputStream)) {

Review Comment:
   **The `binary` transformer now keeps any `InputStream`, including one that 
can be read only once**
   
   This is the global `application-octet-stream` transformer, so the change 
reaches every `transformDataType`, route `inputType`/`outputType` and data type 
action with `binary`, not only source Kamelets. Before, a raw stream became a 
`byte[]` that every later step could read. Now, with stream caching off, the 
first step that reads the body consumes the stream, and the next step gets an 
empty body without any error.
   
   With stream caching on (the default), the body is a `StreamCache` and both 
versions deliver the payload. So the large-object case of this PR is already 
covered by `StreamCache` (resettable, spooled to disk when large). Suggestion: 
keep only a `StreamCache` body and still convert any other `InputStream` to a 
`byte[]`:
   
   ```java
   if (!(message.getBody() instanceof StreamCache)) {
       DELEGATE.transform(message, from, to);
   }
   ```
   
   (`shouldKeepInputStream` would then need a `StreamCache` body instead of a 
`ByteArrayInputStream`.)
   
   <details><summary>Reproducer (<code>core/camel-core</code>, 
<code>org.apache.camel.processor.transformer</code>)</summary>
   
   ```java
   package org.apache.camel.processor.transformer;
   
   import java.io.ByteArrayInputStream;
   import java.io.SequenceInputStream;
   import java.nio.charset.StandardCharsets;
   import java.util.Collections;
   import java.util.List;
   
   import org.apache.camel.ContextTestSupport;
   import org.apache.camel.builder.RouteBuilder;
   import org.junit.jupiter.api.Test;
   
   public class BinaryDataTypeOneShotStreamTest extends ContextTestSupport {
   
       @Test
       public void oneShotStreamIsStillReadableAfterBinary() throws Exception {
           getMockEndpoint("mock:out").expectedBodiesReceived("payload");
           template.sendBody("direct:start", "x");
           assertMockEndpointsSatisfied();
       }
   
       @Override
       protected RouteBuilder createRouteBuilder() {
           return new RouteBuilder() {
               @Override
               public void configure() {
                   from("direct:start").noStreamCaching()
                           // a stream that can be read only once (no 
mark/reset), as a bean or a component can give
                           .process(e -> e.getMessage().setBody(new 
SequenceInputStream(Collections.enumeration(
                                   List.of(new 
ByteArrayInputStream("payload".getBytes(StandardCharsets.UTF_8)))))))
                           .transformDataType("application-octet-stream")
                           .log("${body}")
                           .convertBodyTo(String.class)
                           .to("mock:out");
               }
           };
       }
   }
   ```
   
   PR head (028a8e6):
   
   ```
   java.lang.AssertionError: mock://out Body of message: 0. Expected: <payload> 
but was: <>
   ```
   
   main (5523f54): passes.
   
   </details>



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to