[
https://issues.apache.org/jira/browse/CAMEL-24961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18131334#comment-18131334
]
Claus Ibsen commented on CAMEL-24961:
-------------------------------------
Status 2026-10-24: everything so far is merged on main.
*Merged PRs* (each tested locally against current main before it was merged):
* The tooling: [#28246|https://github.com/apache/camel/pull/28246] (the YAML
validator reads the body types of the catalog instead of its own tables) and
[#28247|https://github.com/apache/camel/pull/28247] (the component, data format
and language pages show them in a _Message Body_ section, and {{camel catalog
doc}} prints them).
* The body types, in 14 PRs:
[#28250|https://github.com/apache/camel/pull/28250],
[#28251|https://github.com/apache/camel/pull/28251],
[#28275|https://github.com/apache/camel/pull/28275],
[#28276|https://github.com/apache/camel/pull/28276],
[#28282|https://github.com/apache/camel/pull/28282],
[#28291|https://github.com/apache/camel/pull/28291],
[#28298|https://github.com/apache/camel/pull/28298],
[#28302|https://github.com/apache/camel/pull/28302],
[#28306|https://github.com/apache/camel/pull/28306],
[#28309|https://github.com/apache/camel/pull/28309],
[#28314|https://github.com/apache/camel/pull/28314],
[#28327|https://github.com/apache/camel/pull/28327],
[#28340|https://github.com/apache/camel/pull/28340] and
[#28343|https://github.com/apache/camel/pull/28343].
*What the catalog on main has:*
* Data formats: 56 of 56.
* Languages: 16 of 29. Not yet: datasonnet, hl7terser, jactl, java, js, ognl,
openfeature, python, python3, quickjs, semantic, spel and typesafe-ai.
* Components: 160 of 413, among them every AWS component. They cover 98.1% of
the endpoint uses in the human-written routes.
* EIPs: not started.
* aws-parameter-store has its body types in its own JSON, and reaches the
catalog when [#28349|https://github.com/apache/camel/pull/28349] (CAMEL-26093)
registers the module.
*Bugs found while checking the body types at runtime:* 115 bug JIRAs
(CAMEL-25924 to CAMEL-26105, one per component).
* 101 are in progress, in open PRs: the batches
[#28353|https://github.com/apache/camel/pull/28353],
[#28355|https://github.com/apache/camel/pull/28355],
[#28358|https://github.com/apache/camel/pull/28358],
[#28359|https://github.com/apache/camel/pull/28359],
[#28363|https://github.com/apache/camel/pull/28363],
[#28364|https://github.com/apache/camel/pull/28364],
[#28365|https://github.com/apache/camel/pull/28365],
[#28372|https://github.com/apache/camel/pull/28372],
[#28373|https://github.com/apache/camel/pull/28373] and
[#28376|https://github.com/apache/camel/pull/28376], and the single fixes
[#28308|https://github.com/apache/camel/pull/28308],
[#28310|https://github.com/apache/camel/pull/28310],
[#28312|https://github.com/apache/camel/pull/28312],
[#28330|https://github.com/apache/camel/pull/28330],
[#28331|https://github.com/apache/camel/pull/28331],
[#28344|https://github.com/apache/camel/pull/28344] and
[#28349|https://github.com/apache/camel/pull/28349]. The batches are the larger
changes and wait for reviews.
* 4 are resolved, and 10 are still open.
* Some descriptions on main change when the batches merge:
[#28363|https://github.com/apache/camel/pull/28363] updates those of jolt,
jsonata and stub.
*Next:*
* The remaining 253 components, by their use in routes: djl, azure-cosmosdb,
azure-storage-datalake, the kubernetes, google and twitter families.
* The 13 languages listed above.
* The EIPs.
_Claude Code on behalf of davsclaus_
> camel-catalog: the body format and Java carrier each component, data format
> and language consumes and produces
> --------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24961
> URL: https://issues.apache.org/jira/browse/CAMEL-24961
> Project: Camel
> Issue Type: Improvement
> Components: camel-catalog
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> Phase C of CAMEL-24844. CAMEL-11132 (2017) is the original ask and stays the
> umbrella; this issue is the executable spec for it.
> The catalog describes every option and every header of a component - {{file}}
> carries 17 headers with a javaType - and says nothing about the body. So a
> person, an IDE, the camel-jbang-mcp catalog tools and the YAML validator
> cannot answer the question that is asked most: *what is the body after this
> step*. The validator's body-presence walk (CAMEL-24844 phase A) has to carry
> its own hardcoded rules for that reason, which is the wrong place for them.
> h3. What to add
> Per component, data format, language and EIP, in the JSON the catalog already
> generates, two dimensions:
> * the *format* - what the payload is, with the Kamelet data type names so a
> Kamelet and a plain component read the same: json, xml, csv, text, binary,
> avro...
> * the *carrier* - the Java type the format arrives in: text, bytes, stream,
> file, map, list, pojo, null. This is what the failures are about: a JSON body
> as a {{Map}} and as text look identical in a description and behave
> differently in {{${body.orderId}}}, {{jsonpath}} and Groovy.
> Both for input and output, and where the type depends on an option
> ({{unmarshalType}}, {{charset}}, stream caching) the metadata names the
> option rather than picking one value.
> h3. Every entry is truthful: four values, no guesses
> A half-filled field is worse than an empty one - a consumer cannot tell
> "unknowable" from "not done yet". So all 407 components get an entry, and the
> value is one of:
> # *derived* - read from the component's own source at its {{setBody}} call
> site, recorded with file and line as provenance so a reviewer checks it in
> seconds. Measured: *322 of 407* components have a {{setBody}} call site in
> their package, and the argument is nearly always a local or field whose
> declared type is in the same file ({{result}} 519, {{response}} 191, {{body}}
> 191, {{out}} 53). This is a reading task, not a guessing task.
> # *unchanged* - the component does not replace the body ({{browse}},
> {{bean-validator}}, {{crypto}} which writes a header). A fact, not an absence.
> # *same as <scheme>* - delegating components state the reference instead of
> repeating it: {{activemq}}, {{activemq6}}, {{amqp}} to jms; {{class}} to
> bean; {{cron}} to quartz; the {{debezium-*}} family to debezium-common.
> # *unknown* - the genuinely dynamic ones, named as such. This is the case
> CAMEL-11132 describes: a component whose type comes from its configuration or
> from the remote system's schema.
> The 85 components with no {{setBody}} in their package fall almost entirely
> into 2 and 3, so *unknown* should end up a small set rather than the residue.
> h3. How it is verified
> # source-derived, with the citation, for the 322;
> # the message history of the reference runs (camel-jbang-examples and the doc
> examples) records the body class per step, so a derived carrier that
> disagrees with what actually flowed fails the check - this is what catches a
> systematic error in the derivation itself, and it is the runtime half of
> CAMEL-24844 (merged in 4.23.0) earning its keep;
> # a reviewer pass over the entries that are neither derived nor confirmed.
> h3. Scope note
> The routes people actually write concentrate hard - over 1712 real YAML files
> (camel-jbang-examples, camel-examples, camel-kamelets, benchmark output)
> there are 122 distinct schemes and the top 10 cover 89.5% of uses, with
> {{json}} at 1214 of 1317 data format uses. That concentration says where to
> *start* and where the benchmark can measure an effect. It does not say where
> to stop: the catalog is a public contract and is filled completely.
> h3. What it unlocks
> * the validator carries the type from step to step and suggests the step that
> fixes it - {{convertBodyTo}}, {{unmarshal}}, {{transformDataType}} - instead
> of only reporting (CAMEL-24844 phase D);
> * the catalog tools and the component documentation answer "what is the body
> here" from metadata rather than prose;
> * the rules leave the validator's source and become data every tool shares.
> Prior art to reuse rather than invent: the Kamelet spec already has
> {{dataTypes:}} for in, out and error (19 of 250 Kamelets declare it today),
> and CAMEL-18698 shipped the data type transformer registry and the
> {{transformDataType}} EIP in 4.0 as the conversion mechanism.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
