davsclaus opened a new pull request, #3297:
URL: https://github.com/apache/camel-kamelets/pull/3297

   Batch 1 (AWS) of #3117: a default data type on the AWS Kamelets whose body 
is fixed. Camel runs the transformer of a default data type on every message, 
so each default changes the body class or the `Content-Type`. The table shows 
what changes. It was checked with the Citrus tests and with `camel run`, on a 
local Camel main build that has CAMEL-27258.
   
   > **Draft:** the `aws-cloudtrail-source` and `aws-kinesis-source` tests need 
CAMEL-27258, which applies a source's output type before the consuming route. 
That fix was merged today, after the published Camel 4.24.0-SNAPSHOT that CI 
uses was built. This PR waits for a newer snapshot.
   
   ## What changes
   
   | Kamelet | in | out | What changes (before → after) |
   |---|---|---|---|
   | `aws-cloudtrail-source` | - | `json` (was `text`) | `byte[]`, no 
`Content-Type` → `byte[]` (JSON text), `application/json`. With CAMEL-27258 and 
the old `text` default it would have been `String`, `text/plain`. Adds 
`camel:jackson3`. |
   | `aws-kinesis-source` | - | `binary` | `byte[]`, no `Content-Type` → 
`byte[]`, `application/octet-stream`. With `useKclConsumers`: `ByteBuffer` → 
`byte[]`. |
   | `aws-kinesis-sink` | `binary` | - | `String` → `byte[]`, a stream cache is 
kept; `Content-Type` → `application/octet-stream`. A Map, a List or a number 
now fails. |
   | `aws-kinesis-firehose-sink` | `binary` | - | same as `aws-kinesis-sink` |
   | `aws-s3-streaming-upload-sink` | `binary` | - | same as `aws-kinesis-sink` 
|
   | `aws-lambda-sink` | `json` | - | JSON text `String` → JSON `byte[]`. A Map 
or a List is written as JSON. Plain text `hello` → `"hello"`. A number now 
fails. After the call the body is the `String` result as before, but 
`Content-Type` is now `application/json`. Adds `camel:jackson3`. |
   | `aws-eventbridge-sink` | `json` | - | as `aws-lambda-sink`. After the 
call: `PutEventsResponse`, `Content-Type` → `application/json`. Adds 
`camel:jackson3`. |
   | `aws-bedrock-text-sink` | `json` | - | as `aws-lambda-sink`. After the 
call: `String`, `Content-Type` → `application/json`. Adds `camel:jackson3`. |
   | `aws-bedrock-agent-runtime-sink` | `text` | - | bytes or a stream → 
`String`. After the call: `String`, `Content-Type` → `text/plain`. |
   | `aws-comprehend-sink` | `text` | - | bytes or a stream → `String`. After 
the call: the SDK List, `Content-Type` → `text/plain`. |
   | `aws-polly-sink` | `text` | - | bytes or a stream → `String`. After the 
call: the audio stream (`InputStreamCache`), `Content-Type` → `text/plain` (not 
the audio type: that is in `CamelAwsPollyContentType`). |
   | `aws-secrets-manager-sink` | `text` | - | bytes or a stream → `String`. 
After the call: `CreateSecretResponse`, `Content-Type` → `text/plain`. |
   | `aws-ses-sink` | `text` | - | bytes or a stream → `String`. After the 
call: `String`, `Content-Type` → `text/plain`. |
   | `aws-translate-action` | `text` | `text` | Input: text, bytes and a stream 
are all sent as the text. Output: `String` as before. `Content-Type`: the 
incoming one was kept → `text/plain`. |
   
   The `Input:`/`Output:` paragraphs of these Kamelets now say this.
   
   ## Not done
   - **`aws-ddb-sink`: no default.** The proposal had the 
`aws2-ddb:application-json` type here, with the template's own 
`transformDataType` step kept. A `camel run` against Floci showed a problem. 
The default runs at the start of the Kamelet, before the template sets the 
`operation` property. A body with an `operation` field, such as 
`{"operation":"PutItem","id":1}` with `operation=PutItem`, then fails in 
`Ddb2JsonDataTypeTransformer.checkBodyOperation`. Without the default this body 
is stored, because Camel accepts an `operation` field that names the configured 
operation. The default would also give the validator nothing. The YAML comment 
now gives this reason.
   - The other AWS Kamelets of the proposal wait on a Camel change: 
`aws-s3-sink` waits on C3, and the sinks that ignore the body wait on C1. The 
SQS and SNS sinks keep no default, because they forward `Content-Type` to AWS 
(D2).
   
   ## Tests
   - 
`AwsIT#awsKinesis+awsFirehose+awsS3+awsLambda+awsEventBridge+awsBedrock+awsDdb+awsComprehend+awsPolly+awsSecretsManager+awsSes+awsCloudTrail`:
 24 tests pass, run locally with Camel main + CAMEL-27258 in `~/.m2`.
   - The expected `X-Body-Type`/`X-Content-Type` values are updated.
   - The kinesis source and sink, eventbridge and ddb sink routes did not 
record the body. They now log the body class and `Content-Type`, and the tests 
verify that line.
   - The firehose and streaming-upload routes now send the body to the Citrus 
server as text, because Citrus reads an `application/octet-stream` body as a 
byte array.
   - `aws-translate-action` has no Citrus test here (#3290 adds one, see 
below). I checked it with `camel run` against a local stand-in, before and 
after the change.
   - I also checked the `aws-lambda-sink` Map, List, plain-text and number 
bodies with `camel run` against a stand-in.
   - `./mvnw clean install` from the root passes, run twice.
   
   ## Overlap with open PRs
   - **#3290** changes `aws-lambda-sink`, `aws-secrets-manager-sink` and 
`aws-translate-action`, and their tests. Its translate test checks only 
`X-Body-Type` (`String`), which this PR keeps. The lambda and secrets-manager 
test files may conflict on rebase; the expected `X-Content-Type` there becomes 
`ct=application/json` and `ct=text/plain`.
   - **#3285** changes `aws-lambda-sink` and `aws-bedrock-text-sink`, and adds 
`aws-lambda-sink-async` and `aws-bedrock-text-sink-converse` tests. Whichever 
of the two PRs merges second must expect `ct=application/json` in those new 
tests.
   - In this PR the edits to those Kamelets are limited to the `dataTypes` 
block, `camel:jackson3` and the `Input:`/`Output:` sentences.
   
   ## Upgrade note for #3190
   ```markdown
   - **Default data types on the AWS Kamelets (#NNNN):**
     - `aws-cloudtrail-source` changes from `text` to `json`. The record is 
JSON in `byte[]` with `Content-Type: application/json`, instead of a `String` 
with `text/plain`.
     - `aws-kinesis-source` (`out: binary`) sets `Content-Type: 
application/octet-stream`. With `useKclConsumers` the body is `byte[]` instead 
of a `ByteBuffer`.
     - `aws-kinesis-sink`, `aws-kinesis-firehose-sink` and 
`aws-s3-streaming-upload-sink` (`in: binary`) turn text into `byte[]` and set 
`Content-Type: application/octet-stream`. A stream is kept. A `Map`, a `List` 
or a number now fails: turn it into JSON text first, for example with 
`json-serialize-action`. AWS receives the same bytes as before.
     - `aws-lambda-sink`, `aws-eventbridge-sink` and `aws-bedrock-text-sink` 
(`in: json`) send JSON bytes and set `Content-Type: application/json`, which 
stays on the message after the sink. A `Map` or a `List` is now written as 
JSON. Text that is not a JSON object or array is sent as a JSON string (`hello` 
becomes `"hello"`). A number fails.
     - `aws-bedrock-agent-runtime-sink`, `aws-comprehend-sink`, 
`aws-polly-sink`, `aws-secrets-manager-sink` and `aws-ses-sink` (`in: text`) 
read the body as a `String` and set `Content-Type: text/plain`, which stays on 
the message after the sink. After `aws-polly-sink` it is not the type of the 
audio: use `CamelAwsPollyContentType`.
     - `aws-translate-action` (`in` and `out: text`) sets `Content-Type: 
text/plain` on the translated text. Before, the incoming `Content-Type` was 
kept.
   ```
   The CAMEL-27258 bullet in #3190 lists `aws-cloudtrail-source` under `text`. 
Move it to `json`.
   
   _Claude Code on behalf of Claus Ibsen_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   
   https://claude.ai/code/session_016Y4dDmBDsSWV117kccuNf9
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to