oscerd opened a new pull request, #29216: URL: https://github.com/apache/camel/pull/29216
Fixes the camel-aws2-sqs and camel-aws2-ddb part of [CAMEL-26300](https://issues.apache.org/jira/browse/CAMEL-26300), found by the runtime tests of the body types of the data type transformers (CAMEL-25928). CAMEL-26300 is fixed with one PR per component. Fixes the camel-aws2 items of [CAMEL-26300](https://issues.apache.org/jira/browse/CAMEL-26300), found by the runtime tests of the body types of the data type transformers (CAMEL-25928). ## 1. aws2-sqs:application-cloudevents builds the CloudEvent source from the receipt handle **Was:** `AWS2SQSCloudEventDataTypeTransformer` set `CamelCloudEventSource` to `aws.sqs.queue.` followed by the `CamelAwsSqsReceiptHandle` header. A receipt handle identifies one receive of a message and changes each time the message is received, so every event had a source of its own, and the source never named the queue. The consumer did not put the queue name on the message, so the transformer had nothing better to use. **Now:** the `aws2-sqs` consumer sets a new header, `CamelAwsSqsQueueName`, to the name of the queue it reads from (the name part of the ARN when the endpoint names the queue by its ARN), and the transformer builds the source from it: `aws.sqs.queue.<queue name>`, as `aws2-s3:application-cloudevents` builds its source from the bucket name header of the `aws2-s3` consumer. A message without the header gets no source from the transformer, also when it has a receipt handle, as a message without a receipt handle did before. The queue name travels in a header rather than being read from the from endpoint of the exchange because the output data type of a Kamelet runs in the route that consumes from the Kamelet: that route gets a copy of the exchange with the headers of the consumer, but with its own from endpoint. ## 2. aws2-ddb:application-x-struct says it gives a Json node, and fails on a stream or a file **Was:** the description of `Ddb2JsonStructDataTypeTransformer` said "Transforms DynamoDB record into a Json node", but it gives a JSON String: a String body is left as it is, and another body is written with Gson. A stream or a file body failed with Gson's `JsonIOException`, as Gson cannot reflect into the fields of the `java.io` classes on Java 17 and later. **Now:** the output stays a JSON String, and the description says so: "Transforms DynamoDB record into a Json String". A stream (an `InputStream`, a `Reader` or a stream cache) or a file (a `java.io.File`, a `Path` or the `GenericFile` of the file consumers) is read as text with the type converters, as the JSON it holds, the same way a String body is taken as it is. Gson is no longer given these bodies. When the content cannot be read, such as a file that does not exist, the transformer fails with a `CamelExecutionException` whose message does not hold the body. Other bodies, such as the `Record` of the `aws2-ddbstream` consumer, are written with Gson as before. The transformer keeps the String rather than giving a Jackson `JsonNode` as the `application-x-struct` transformers of camel-jackson do: it was written to replace a Gson marshal in the `aws-ddb-streams-source` Kamelet, whose default output is JSON text (`application/json`), camel-aws2-ddb has camel-jackson only as an optional dependency, and `google-sheets:application-x-struct` gives JSON Strings as well. ## Tests - `AWS2SQSCloudEventDataTypeTransformerRouteTest` (new) uses the in-memory SQS client of the module's tests. A route from an `aws2-sqs` consumer hands the message over `seda` to a second route, as the sink of a Kamelet hands a copy of the exchange to the route that consumes from the Kamelet, and the transformer runs there: the message has the `CamelAwsSqsQueueName` header and the source `aws.sqs.queue.MyQueue`. An endpoint that names the queue by its ARN gives the name part of the ARN in the header and in the source. - `AWS2SQSCloudEventDataTypeTransformerTest`: `shouldMapToCloudEvent` set the receipt handle to `myQueue` and expected `aws.sqs.queue.myQueue`, which was the bug. It now sets the queue name header and a receipt handle. A new test checks that a receipt handle alone gives no source. - `Ddb2JsonStructDataTypeTransformerTest`: the body types list the stream and file carriers. An `InputStream`, a `Reader` and a stream cache, and a `java.io.File`, a `Path` and the `GenericFile` of the file component, are read as the JSON text they hold. A file that does not exist fails with a `CamelExecutionException` whose message does not name the file. A `Record`, bytes, a number, a `Map` and a `List` still give a String, and a String is left as it is. Revert check: with the change to `src/main` reverted, 8 of the 8 new or changed tests fail, each on its assertion or with the old exception. - aws2-sqs: the new `Sqs2Constants.QUEUE_NAME` constant was kept so that the tests compile. The two route tests fail because the consumer sets no queue name header, `shouldMapToCloudEvent` gets the source `aws.sqs.queue.<receipt handle>`, and the receipt handle alone still gives a source. - aws2-ddb: the stream and file tests fail with Gson's `JsonIOException` ("Failed making field 'java.io.ByteArrayInputStream#buf' accessible"), the missing file with a `JsonIOException` instead of a `CamelExecutionException`, and the body types test on the carriers. Modules: camel-aws2-sqs 1586 tests and camel-aws2-ddb 99 tests, 0 failures. The integration tests that need LocalStack were not run. ## Docs and generated files - `CamelAwsSqsQueueName` is a new consumer header in `Sqs2Constants`, so it is in the regenerated `aws2-sqs.json`, its catalog copy, the header table of the component page, and the generated endpoint DSL. - The body types of `aws2-ddb:application-x-struct` now list the stream and file carriers and say that they are read as text; the regenerated `aws2-ddb-application-x-struct.json` and its catalog copy have the new description and body types. - The body types of `aws2-sqs:application-cloudevents` are unchanged: any body, left as it is. - Upgrade guide entry for 4.24, as the source of `aws2-sqs:application-cloudevents` changes and the consumer sets a new header. ## Not changed - The source is not read from the from endpoint of the exchange, which needs no new header but does not work for the output data type of a Kamelet, as explained above. A consumer header is also how the other AWS CloudEvent transformers get their source. - The source is the queue name, not the queue URL, to match `aws.s3.bucket.<bucket name>` of `aws2-s3:application-cloudevents`. - A `byte[]` body of `aws2-ddb:application-x-struct` is still written by Gson as a JSON array of numbers, and a `null` body still gives the String `null`: the issue covers only streams and files, and these bodies do not fail. Main only, as the other fixes of the CAMEL-24961 follow-ups. Main only, as the other fixes of the CAMEL-24961 follow-ups. _Claude Code on behalf of Andrea Cosentino (oscerd)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
