[
https://issues.apache.org/jira/browse/CAMEL-24961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18131417#comment-18131417
]
Claus Ibsen commented on CAMEL-24961:
-------------------------------------
Status: the last 253 components of the catalog now have body types, in 22 pull
requests against main (one per family of components, each independent of the
others):
#28395 HTTP servers, #28393 databases, #28394 messaging, #28398 NoSQL and
search, #28396 hazelcast, #28397 ignite and caches, #28399 and #28400
kubernetes and openshift, #28401 langchain4j, spring-ai, djl, #28402 model
servers, vector databases and IBM Watson, #28403 google workspace and azure,
#28404 google cloud and the vaults, #28409 alibaba, dapr, consul, #28406 huawei
cloud and openstack, #28410 and #28412 SaaS, #28411 IoT, #28413 network and
RPC, #28414 ops, #28416 security components, #28417 files and miscellaneous,
#28418 debezium and workflow.
With them merged, every component, data format and language in the catalog has
body types; the EIPs have none yet.
Each component was checked at runtime with throwaway tests (embedded servers,
the test-infra containers, local fakes; never a real cloud service), sending
text, bytes, a stream, a file, a Map, a POJO and no body, and logging the class
of the body after each step. About 35 doc pages that were wrong about the body
are corrected in the PRs; the YAML validator of the check also caught two doc
examples that could not work (azure-cosmosdb, alibaba-eventbridge).
The PRs list the bugs found along the way, per component, about 440 in about
200 components (not fixed in these PRs). The JIRAs for them, one per component,
will follow.
_Claude Code on behalf of davsclaus_
> camel-catalog: the body format and Java carrier each component, data format
> and language consumes and produces
> --------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24961
> URL: https://issues.apache.org/jira/browse/CAMEL-24961
> Project: Camel
> Issue Type: Improvement
> Components: camel-catalog
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> Phase C of CAMEL-24844. CAMEL-11132 (2017) is the original ask and stays the
> umbrella; this issue is the executable spec for it.
> The catalog describes every option and every header of a component - {{file}}
> carries 17 headers with a javaType - and says nothing about the body. So a
> person, an IDE, the camel-jbang-mcp catalog tools and the YAML validator
> cannot answer the question that is asked most: *what is the body after this
> step*. The validator's body-presence walk (CAMEL-24844 phase A) has to carry
> its own hardcoded rules for that reason, which is the wrong place for them.
> h3. What to add
> Per component, data format, language and EIP, in the JSON the catalog already
> generates, two dimensions:
> * the *format* - what the payload is, with the Kamelet data type names so a
> Kamelet and a plain component read the same: json, xml, csv, text, binary,
> avro...
> * the *carrier* - the Java type the format arrives in: text, bytes, stream,
> file, map, list, pojo, null. This is what the failures are about: a JSON body
> as a {{Map}} and as text look identical in a description and behave
> differently in {{${body.orderId}}}, {{jsonpath}} and Groovy.
> Both for input and output, and where the type depends on an option
> ({{unmarshalType}}, {{charset}}, stream caching) the metadata names the
> option rather than picking one value.
> h3. Every entry is truthful: four values, no guesses
> A half-filled field is worse than an empty one - a consumer cannot tell
> "unknowable" from "not done yet". So all 407 components get an entry, and the
> value is one of:
> # *derived* - read from the component's own source at its {{setBody}} call
> site, recorded with file and line as provenance so a reviewer checks it in
> seconds. Measured: *322 of 407* components have a {{setBody}} call site in
> their package, and the argument is nearly always a local or field whose
> declared type is in the same file ({{result}} 519, {{response}} 191, {{body}}
> 191, {{out}} 53). This is a reading task, not a guessing task.
> # *unchanged* - the component does not replace the body ({{browse}},
> {{bean-validator}}, {{crypto}} which writes a header). A fact, not an absence.
> # *same as <scheme>* - delegating components state the reference instead of
> repeating it: {{activemq}}, {{activemq6}}, {{amqp}} to jms; {{class}} to
> bean; {{cron}} to quartz; the {{debezium-*}} family to debezium-common.
> # *unknown* - the genuinely dynamic ones, named as such. This is the case
> CAMEL-11132 describes: a component whose type comes from its configuration or
> from the remote system's schema.
> The 85 components with no {{setBody}} in their package fall almost entirely
> into 2 and 3, so *unknown* should end up a small set rather than the residue.
> h3. How it is verified
> # source-derived, with the citation, for the 322;
> # the message history of the reference runs (camel-jbang-examples and the doc
> examples) records the body class per step, so a derived carrier that
> disagrees with what actually flowed fails the check - this is what catches a
> systematic error in the derivation itself, and it is the runtime half of
> CAMEL-24844 (merged in 4.23.0) earning its keep;
> # a reviewer pass over the entries that are neither derived nor confirmed.
> h3. Scope note
> The routes people actually write concentrate hard - over 1712 real YAML files
> (camel-jbang-examples, camel-examples, camel-kamelets, benchmark output)
> there are 122 distinct schemes and the top 10 cover 89.5% of uses, with
> {{json}} at 1214 of 1317 data format uses. That concentration says where to
> *start* and where the benchmark can measure an effect. It does not say where
> to stop: the catalog is a public contract and is filled completely.
> h3. What it unlocks
> * the validator carries the type from step to step and suggests the step that
> fixes it - {{convertBodyTo}}, {{unmarshal}}, {{transformDataType}} - instead
> of only reporting (CAMEL-24844 phase D);
> * the catalog tools and the component documentation answer "what is the body
> here" from metadata rather than prose;
> * the rules leave the validator's source and become data every tool shares.
> Prior art to reuse rather than invent: the Kamelet spec already has
> {{dataTypes:}} for in, out and error (19 of 250 Kamelets declare it today),
> and CAMEL-18698 shipped the data type transformer registry and the
> {{transformDataType}} EIP in 4.0 as the conversion mechanism.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
