davsclaus opened a new pull request, #3309:
URL: https://github.com/apache/camel-kamelets/pull/3309
**Stacked on #3299: merge after it; until then the diff also shows its
commits.**
**Draft: CI needs tonight's Camel 4.24.0-SNAPSHOT.** The defaults here
depend on Camel changes merged today (see Needs). Tested locally against a
snapshot built from Camel `main` with them.
Part of #3117, batch 9c. Adds the default data types that waited for today's
Camel data type changes. Camel runs the transformer of a default on every
message, so each one was checked with `camel run`, before (the Kamelets of
#3299) and after: the body class, `Content-Type` and the message data type,
with String, byte[], stream, Map, List, POJO, `JsonNode`, plain text, a number
and null bodies, in the Kamelet EIP and the `to:` forms.
## What changes
| Kamelet | in | out | What changes (before → after) |
|---|---|---|---|
| `json-deserialize-action` | `binary` (kept) | `struct` | Body unchanged: a
Jackson 3 `JsonNode`, `Content-Type` removed. The message now carries the
`application-x-struct` data type, so the input type of the next Kamelet
converts from it. |
| `avro-deserialize-action` | `binary` (kept) | `struct` | As
`json-deserialize-action`. |
| `protobuf-deserialize-action` | `binary` (kept) | `struct` | As
`json-deserialize-action`. |
| `jsonata-action` | - | `struct` | Body unchanged (a Jackson 3 `JsonNode`);
the headers are kept as they came. Adds the `camel:jackson3` dependency, which
has the `struct` transformer. |
| `avro-serialize-action` | - | `avro-binary` | No visible change: the Avro
data and `Content-Type: application/avro` are kept as they are (CAMEL-27271).
The Output text said bytes; it is a stream, fixed. |
| `protobuf-serialize-action` | - | `protobuf-binary` | As
`avro-serialize-action`, with `application/protobuf`. Its description said JSON
text is passed on as it is; it fails (a `NullPointerException` from Jackson
Protobuf, before and after), fixed. |
| `fhir-source` | - | `json` | The Bundle: a stream → `byte[]`.
`Content-Type: application/fhir+json` is kept (CAMEL-27271). Adds
`camel:jackson3`. |
| `ogcapi-features-action` | `text` (kept) | `json` | The collection: a
`String` → `byte[]`. The `Content-Type` of the response, such as
`application/geo+json`, is kept. With `split` and the Kamelet EIP, each
feature: a stream → `byte[]`, `Content-Type: application/json`. With `split`
and `to:` nothing changes (see below). |
The type is `struct` (`application-x-struct`). With `camel-jackson` and
`camel-jackson3` both on the classpath, the Jackson 3 transformer is used
(checked with the transformer registry debug log).
### The chains from #3299 (fresh JVM)
- `json-deserialize-action` → `value-to-key-action`: JSON bytes,
`Content-Type: application/json`, data type `application-json`, as before. Also
with `camel-jackson` added. New Citrus test
`json/json-deserialize-chain-route`, which uses the Kamelet EIP.
- `json-deserialize-action` → `jsonata-action` → `pdf-action`: the debug log
shows `StringDataTypeTransformer for key application-x-struct/text-plain` for
the input of `pdf-action`, so its `text` input runs. Before CAMEL-27289 this
was the struct transformer.
- Also checked after `json-deserialize-action`: `jslt-action` (`in:
binary`), and `data-type-action` with `text-plain`, `application-json` and
`application-octet-stream`: same body and `Content-Type` as before.
- `avro-serialize-action` → `avro-deserialize-action` and the Protobuf pair:
same `JsonNode` as before.
### Descriptions
- `avro-serialize-action`, `protobuf-serialize-action`: the output is a
stream, not bytes.
- `protobuf-serialize-action`: JSON text fails (it was said to be passed on).
- `fhir-source`, `ogcapi-features-action`: the output is JSON text in bytes.
- `ogcapi-features-action`: `split` works only with the Kamelet EIP. Called
with `to:` (as in a Pipe), the next step still gets the whole collection as
text: the message that comes back never passes `kamelet:sink`, so its output
type is not applied either. The description said `split` never changes what the
next step gets; with the Kamelet EIP the next step gets each feature.
## Not done: `application-x-java-object` as input
CAMEL-27272 lets `application-x-java-object` keep a Map or a List. As an
input default it would still break input that these Kamelets take today, so it
was left out. A YAML comment in each Kamelet gives the reason.
- `jsonata-action`: JSON text, bytes, a stream and plain text fail with
"Missing content class information". All of them work today.
- `json-serialize-action`: text, bytes, a stream and a number fail. Today
they are written as JSON.
- `jolt-transformation-action`, in and out: as input, JSON text, bytes and a
stream fail, which the Jolt component parses today. As output, it fails on the
null body that a specification matching nothing gives (also a List, a POJO or a
`JsonNode` input give null).
- `avro-serialize-action`, `protobuf-serialize-action`: a Map or a
`JsonNode` fails ("Unsupported schema type 'AVRO'") once an earlier
`resolve-pojo-schema-action` has set `contentClass`. That works today. Input
that fails today (text, bytes, a List, null) fails the same way.
The sources in other open PRs follow when those merge: the GitHub
commit/event sources (#3293), `mongodb-changes-stream-source` (#3283),
`kafka-batch-source` (#3281) and `google-mail-source` (#3296).
## Tests
- `CommonIT#json+transformation+geo`: 23 of 23 pass, with the new
`json/json-deserialize-chain-route`.
- `AvroIT`: 2 of 2. `ProtobufIT`: 2 of 2. `ServicesIT#fhir`: 2 of 2.
`avro-data-type-test` and `protobuf-data-type-test` failed locally for #3299;
with today's snapshot they pass.
- `avro/` and `protobuf/` serdes routes now also record what the serializer
gives and what the deserializer gives. They are checked with a second `verify`:
`InputStreamCache ct=application/avro` (or `application/protobuf`), then
`ObjectNode ct=`.
- `fhir-source-route` and `ogcapi-features-action-route` now expect `[B`.
- `./mvnw clean install` from the root, twice: green.
## Overlap
#3299 is the base (the same deserialize actions, `ogcapi-features-action`).
#3285 changes `nominatim-geocode-action`, which is not touched here.
## Needs
Camel 4.24.0-SNAPSHOT with CAMEL-27270, CAMEL-27271, CAMEL-27272,
CAMEL-27289 and CAMEL-27290 (all on `main` today; in tonight's snapshot).
Without CAMEL-27289 a `struct` output is taken as the transformer for the next
Kamelet's input, as #3299 found. Without CAMEL-27290 a source's output type is
applied only after the consuming route has run. Without CAMEL-27271
`avro-binary`/`protobuf-binary` need a schema, and `json` replaces
`application/fhir+json`.
## Upgrade note for #3190 (Data types section)
- **Default data types that the Camel 4.24 data type changes make possible
(#XXXX):**
- `json-deserialize-action`, `avro-deserialize-action`,
`protobuf-deserialize-action` and `jsonata-action` give their parsed JSON as
the `application-x-struct` data type. The body is the same Jackson 3
`JsonNode`. `jsonata-action` now depends on `camel-jackson3`.
- `avro-serialize-action` and `protobuf-serialize-action` give the
`avro-binary` and `protobuf-binary` data types. The data and the `Content-Type`
are unchanged.
- `fhir-source` gives the Bundle as JSON text in bytes (it was a stream),
with `Content-Type: application/fhir+json` as before, and now depends on
`camel-jackson3`.
- `ogcapi-features-action` gives the feature collection as JSON text in
bytes (it was a `String`), with the `Content-Type` of the response. With
`split` and the Kamelet EIP, each feature is bytes (it was a stream).
_Claude Code on behalf of Claus Ibsen_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_016Y4dDmBDsSWV117kccuNf9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]