davsclaus opened a new pull request, #3309:
URL: https://github.com/apache/camel-kamelets/pull/3309

   **Stacked on #3299: merge after it; until then the diff also shows its 
commits.**
   
   **Draft: CI needs tonight's Camel 4.24.0-SNAPSHOT.** The defaults here 
depend on Camel changes merged today (see Needs). Tested locally against a 
snapshot built from Camel `main` with them.
   
   Part of #3117, batch 9c. Adds the default data types that waited for today's 
Camel data type changes. Camel runs the transformer of a default on every 
message, so each one was checked with `camel run`, before (the Kamelets of 
#3299) and after: the body class, `Content-Type` and the message data type, 
with String, byte[], stream, Map, List, POJO, `JsonNode`, plain text, a number 
and null bodies, in the Kamelet EIP and the `to:` forms.
   
   ## What changes
   
   | Kamelet | in | out | What changes (before → after) |
   |---|---|---|---|
   | `json-deserialize-action` | `binary` (kept) | `struct` | Body unchanged: a 
Jackson 3 `JsonNode`, `Content-Type` removed. The message now carries the 
`application-x-struct` data type, so the input type of the next Kamelet 
converts from it. |
   | `avro-deserialize-action` | `binary` (kept) | `struct` | As 
`json-deserialize-action`. |
   | `protobuf-deserialize-action` | `binary` (kept) | `struct` | As 
`json-deserialize-action`. |
   | `jsonata-action` | - | `struct` | Body unchanged (a Jackson 3 `JsonNode`); 
the headers are kept as they came. Adds the `camel:jackson3` dependency, which 
has the `struct` transformer. |
   | `avro-serialize-action` | - | `avro-binary` | No visible change: the Avro 
data and `Content-Type: application/avro` are kept as they are (CAMEL-27271). 
The Output text said bytes; it is a stream, fixed. |
   | `protobuf-serialize-action` | - | `protobuf-binary` | As 
`avro-serialize-action`, with `application/protobuf`. Its description said JSON 
text is passed on as it is; it fails (a `NullPointerException` from Jackson 
Protobuf, before and after), fixed. |
   | `fhir-source` | - | `json` | The Bundle: a stream → `byte[]`. 
`Content-Type: application/fhir+json` is kept (CAMEL-27271). Adds 
`camel:jackson3`. |
   | `ogcapi-features-action` | `text` (kept) | `json` | The collection: a 
`String` → `byte[]`. The `Content-Type` of the response, such as 
`application/geo+json`, is kept. With `split` and the Kamelet EIP, each 
feature: a stream → `byte[]`, `Content-Type: application/json`. With `split` 
and `to:` nothing changes (see below). |
   
   The type is `struct` (`application-x-struct`). With `camel-jackson` and 
`camel-jackson3` both on the classpath, the Jackson 3 transformer is used 
(checked with the transformer registry debug log).
   
   ### The chains from #3299 (fresh JVM)
   
   - `json-deserialize-action` → `value-to-key-action`: JSON bytes, 
`Content-Type: application/json`, data type `application-json`, as before. Also 
with `camel-jackson` added. New Citrus test 
`json/json-deserialize-chain-route`, which uses the Kamelet EIP.
   - `json-deserialize-action` → `jsonata-action` → `pdf-action`: the debug log 
shows `StringDataTypeTransformer for key application-x-struct/text-plain` for 
the input of `pdf-action`, so its `text` input runs. Before CAMEL-27289 this 
was the struct transformer.
   - Also checked after `json-deserialize-action`: `jslt-action` (`in: 
binary`), and `data-type-action` with `text-plain`, `application-json` and 
`application-octet-stream`: same body and `Content-Type` as before.
   - `avro-serialize-action` → `avro-deserialize-action` and the Protobuf pair: 
same `JsonNode` as before.
   
   ### Descriptions
   
   - `avro-serialize-action`, `protobuf-serialize-action`: the output is a 
stream, not bytes.
   - `protobuf-serialize-action`: JSON text fails (it was said to be passed on).
   - `fhir-source`, `ogcapi-features-action`: the output is JSON text in bytes.
   - `ogcapi-features-action`: `split` works only with the Kamelet EIP. Called 
with `to:` (as in a Pipe), the next step still gets the whole collection as 
text: the message that comes back never passes `kamelet:sink`, so its output 
type is not applied either. The description said `split` never changes what the 
next step gets; with the Kamelet EIP the next step gets each feature.
   
   ## Not done: `application-x-java-object` as input
   
   CAMEL-27272 lets `application-x-java-object` keep a Map or a List. As an 
input default it would still break input that these Kamelets take today, so it 
was left out. A YAML comment in each Kamelet gives the reason.
   
   - `jsonata-action`: JSON text, bytes, a stream and plain text fail with 
"Missing content class information". All of them work today.
   - `json-serialize-action`: text, bytes, a stream and a number fail. Today 
they are written as JSON.
   - `jolt-transformation-action`, in and out: as input, JSON text, bytes and a 
stream fail, which the Jolt component parses today. As output, it fails on the 
null body that a specification matching nothing gives (also a List, a POJO or a 
`JsonNode` input give null).
   - `avro-serialize-action`, `protobuf-serialize-action`: a Map or a 
`JsonNode` fails ("Unsupported schema type 'AVRO'") once an earlier 
`resolve-pojo-schema-action` has set `contentClass`. That works today. Input 
that fails today (text, bytes, a List, null) fails the same way.
   
   The sources in other open PRs follow when those merge: the GitHub 
commit/event sources (#3293), `mongodb-changes-stream-source` (#3283), 
`kafka-batch-source` (#3281) and `google-mail-source` (#3296).
   
   ## Tests
   
   - `CommonIT#json+transformation+geo`: 23 of 23 pass, with the new 
`json/json-deserialize-chain-route`.
   - `AvroIT`: 2 of 2. `ProtobufIT`: 2 of 2. `ServicesIT#fhir`: 2 of 2. 
`avro-data-type-test` and `protobuf-data-type-test` failed locally for #3299; 
with today's snapshot they pass.
   - `avro/` and `protobuf/` serdes routes now also record what the serializer 
gives and what the deserializer gives. They are checked with a second `verify`: 
`InputStreamCache ct=application/avro` (or `application/protobuf`), then 
`ObjectNode ct=`.
   - `fhir-source-route` and `ogcapi-features-action-route` now expect `[B`.
   - `./mvnw clean install` from the root, twice: green.
   
   ## Overlap
   
   #3299 is the base (the same deserialize actions, `ogcapi-features-action`). 
#3285 changes `nominatim-geocode-action`, which is not touched here.
   
   ## Needs
   
   Camel 4.24.0-SNAPSHOT with CAMEL-27270, CAMEL-27271, CAMEL-27272, 
CAMEL-27289 and CAMEL-27290 (all on `main` today; in tonight's snapshot). 
Without CAMEL-27289 a `struct` output is taken as the transformer for the next 
Kamelet's input, as #3299 found. Without CAMEL-27290 a source's output type is 
applied only after the consuming route has run. Without CAMEL-27271 
`avro-binary`/`protobuf-binary` need a schema, and `json` replaces 
`application/fhir+json`.
   
   ## Upgrade note for #3190 (Data types section)
   
   - **Default data types that the Camel 4.24 data type changes make possible 
(#XXXX):**
     - `json-deserialize-action`, `avro-deserialize-action`, 
`protobuf-deserialize-action` and `jsonata-action` give their parsed JSON as 
the `application-x-struct` data type. The body is the same Jackson 3 
`JsonNode`. `jsonata-action` now depends on `camel-jackson3`.
     - `avro-serialize-action` and `protobuf-serialize-action` give the 
`avro-binary` and `protobuf-binary` data types. The data and the `Content-Type` 
are unchanged.
     - `fhir-source` gives the Bundle as JSON text in bytes (it was a stream), 
with `Content-Type: application/fhir+json` as before, and now depends on 
`camel-jackson3`.
     - `ogcapi-features-action` gives the feature collection as JSON text in 
bytes (it was a `String`), with the `Content-Type` of the response. With 
`split` and the Kamelet EIP, each feature is bytes (it was a stream).
   
   _Claude Code on behalf of Claus Ibsen_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   
   https://claude.ai/code/session_016Y4dDmBDsSWV117kccuNf9
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to