[
https://issues.apache.org/jira/browse/CAMEL-24844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18123387#comment-18123387
]
Claus Ibsen commented on CAMEL-24844:
-------------------------------------
h2. Design note: knowing the body type
h3. The problem
Every tool that helps someone write a route asks the same question: what is the
body after this step? The hard cases are the normal Camel use cases: plain
components, a file, an HTTP call, a Kafka topic, a JMS queue, a bean. There,
the body type depends on component options, on what the other system sent, and
on type converters. Only the running route knows it for certain. That is why
people (and models) meet it as runtime errors, such as:
* {{No such property: sku for class: java.lang.Byte}}
* {{Key: amount not found in bean: GenericFile\[...\]}}
* {{No type converter available to convert from type: LinkedHashMap to
InputStream}}
A local model met it in 19 of 156 attempts in the round-2 benchmark.
*Not the way:* asking the route author to declare types. Kamelet {{dataTypes}},
route {{inputType}}/{{outputType}} and {{transformDataType}} exist. They take
strong discipline to keep accurate on every route, and normal Camel users do
not use them: only 19 of 262 kamelets declare data types. Kamelets and
transformers are newer concepts, used mostly with Camel K. Whatever we build
has to work for a route that declares nothing.
So: the runtime is the truth, and the tools should get it from there. Static
metadata is the best-effort complement for the steps the runtime has not run
yet.
h3. 1. Runtime first: errors say what the body is, and where it came from
The message history now records the body type at every step
({{MessageHistory.getBodyType()}}, merged for 4.23). When a step fails on the
body's type, the error can say three things: what the body is, which step made
it so, and the step that fixes it. For example:
bq. {{$\{body\[amount\]\}}} failed: the body is a GenericFile (the file as
read), unchanged since {{from: file:inbox}} (invoices.yaml:30). Unmarshal it
first ({{unmarshal: json}}) to read its fields.
Today each place writes its own partial hint:
* the Groovy hint (CAMEL-25330)
* the null-body hint for marshal/unmarshal (CAMEL-25285)
* the Map-key hint of the bean expression
* the jsonpath/jq null hint (CAMEL-24838)
Proposal: one helper in camel-support that builds this sentence from the
exchange and its message history. It is used by the expression failures
(simple/bean OGNL, Groovy, jsonpath), the type converter failure and
{{InvalidPayloadException}}. With message history off, it still names the
body's type.
This helps every route, every component, every user, with no metadata at all.
It comes after the run, but in a dev loop (camel run --dev, or a model through
camel-jbang-mcp) the run is seconds away.
h3. 2. Observed types feed the authoring tools
In dev mode the app has already run most steps. The body type each step saw is
in the message history, and the camel-jbang history tool already shows it per
step. Proposal:
* *The validator takes observed types as facts.* Where its static walk says
"unknown" (after kafka, a bean, an HTTP call), the last observed type for that
node fills the gap. The walk in PR 27367 then reports {{$\{body\[x\]\}}} on a
body that was seen to be a String, whatever component produced it.
* *An MCP tool answers "what was the body at each step"* for the running app,
so a model reads it before it writes the next step, instead of guessing.
This covers exactly what the user ran, and only that. Untried paths stay
unknown, which is honest.
h3. 3. Catalog metadata: the static default, verified at runtime
For steps that have not run yet (a model writing a new route), the catalog
gives the default answer per component, data format and language. A coarse
*carrier* is enough: {{text}}, {{bytes}}, {{stream}}, {{file}}, {{map}},
{{list}}, {{pojo}}, {{null}}, {{unchanged}}, {{any}}, {{unknown}}. Plus a
*format* where it is known (json, xml, csv, text, binary).
The option-dependent cases are the rule in normal Camel, so they are part of
the shape:
{code:json}
"bodyTypes": {
"consumerOut": { "carrier": "text", "dependsOn": [ "valueDeserializer" ] },
"producerOut": { "carrier": "unchanged" }
}
{code}
* {{dependsOn}}: the answer holds while those options keep their defaults;
changed, it is {{unknown}}.
* {{byOption}}: maps values to answers, for example JMS {{jmsMessageType}} Text
→ text, Bytes → bytes, Map → map.
The source is a {{@BodyTypes}} annotation on the endpoint, data format and
language classes. The existing build plugins carry it into the catalog JSON,
the catalog API and the component pages ("Consumer body: a String, unless
valueDeserializer is set").
Why this can work now, when CAMEL-330 (2008) and CAMEL-11132 (2017) were
abandoned: they needed 350 components filled in by hand. Now an AI pass
proposes the values, and the runtime checks them. A reference route per
component is run with message history on, and the recorded type is compared
with the metadata, in CI. A component nobody verified ships without the field:
no field means {{unknown}}, never a guess.
h3. Content-type aware payloads (CAMEL-25334) prevent the most common mistakes
CAMEL-25334 (4.24.0) proposes a different approach. The body stays text
together with its content type. Groovy and simple read JSON fields straight
from that text, and a structured result is written back as text. This removes
the most common collision before anyone has to explain it: a field read on JSON
that was never unmarshalled. The two complement each other:
* The content-type mode prevents the mistakes it covers.
* Parts 1 to 3 explain whatever is left: binary payloads, component-specific
types, a bean that returns a POJO, routes that run in classic mode.
The benchmark can measure both together: count body type collisions per series
with the mode off and on.
h3. Order
# *Runtime errors:* the shared helper, and the existing hints moved onto it.
Smallest step, helps every user today.
# *Observed types:* in the validator, and an MCP tool for the running app.
# *Catalog metadata:* a pilot of the components the routes in our corpus start
from (file, ftp/sftp, http, platform-http, kafka, jms, sql, aws2-sqs,
paho-mqtt5, mongodb, rest), the json/jacksonXml/csv/yaml data formats and the
main languages. Verified at runtime, then the rest of the catalog by AI pass
plus review, group by group.
Measured each time on the benchmark: the body type collisions per series.
h3. Later, separately
Kamelets and data type transformers are not well tested or documented today.
With AI there is a chance to overhaul them, and the catalog metadata would give
them correct data types for free. That is its own piece of work, and nothing
here waits for it.
h3. Not in scope
Schemas of the payload (the fields inside the JSON), headers, and Java/XML DSL
routes in the static walk. The runtime parts (1 and 2) work for every DSL.
_Claude Code on behalf of davsclaus_
> Body type flow: the validator carries the body's Java type from step to step
> and warns where a step assumes text; the message history's recorded body
> types feed the same check at runtime
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24844
> URL: https://issues.apache.org/jira/browse/CAMEL-24844
> Project: Camel
> Issue Type: Improvement
> Components: camel-core, camel-jbang, camel-yaml-dsl
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.23.0
>
>
> Camel carries the payload as the natural Java object: a GenericFile from the
> file consumer, a Map or List after {{unmarshal json}}, a byte[] or a
> StreamCache after {{marshal}}, an InputStream from http. Integration people
> think in text and bytes, and write routes as if the body were a String. Every
> failure in the round-2 local-model benchmark that was not a YAML shape error
> was that collision, and a person writing the same route gets the same runtime
> exception:
> * {{${body.orderId}}} after {{unmarshal: json}} (the body is a Map):
> MethodNotFoundException, in five examples. The runtime message now says "the
> value is a Map: a key is read with [orderId]".
> * {{body.email}} in a Groovy expression on a GenericFile, before any
> conversion: MissingPropertyException.
> * {{jsonpath}} on a body that is already a Map after unmarshal, or on a null
> body on a timer route (CAMEL-24838).
> * {{${body}}} after {{unmarshal: json}} treated as JSON text: it is the Map's
> toString ({{{id=ORD-1001, ...}}}).
> * {{new ByteArrayInputStream(body)}} in Groovy after {{marshal}}: with stream
> caching on (the Camel CLI default) the body is a StreamCache, not a byte[]:
> "could not find matching constructor". Same route, different Java type
> depending on a runtime setting.
> Measured with the CLI on 4.23.0-SNAPSHOT: logging {{${body}}} prints readable
> text at every stage (GenericFile, Map, marshal output, InputStream with
> stream caching on, twice); only with
> {{camel.main.streamCachingEnabled=false}} does a log step consume an
> InputStream and leave every later step an empty body.
> The object model must stay: it is why a Map flows into a bean and a stream
> into a file without copies. What can change is that Camel knows the type at
> every step and says nothing until the step that assumes text explodes.
> Proposal, in two halves that share one rule set (which step produces which
> type, which expressions and steps need which type):
> # *Static, in {{camel validate yaml}} and the camel-jbang-mcp validation
> tool.* Walk the route and carry the body type forward from the catalog:
> {{from: file}} gives GenericFile, {{unmarshal: json}} gives Map/List (or the
> unmarshalType), {{marshal}} gives byte[]/StreamCache, {{unmarshal:
> jacksonXml}} gives Map, {{split}} on a List gives an element, {{setBody:
> constant}} gives String, {{to: http}} gives InputStream, {{convertBodyTo}}
> gives its type, a bean gives its return type when it can be resolved. At each
> step check the expression against the type and say it in the words of the
> text world, with the line: "after unmarshal json the body is a Map:
> ${body.orderId} fails, write ${body[orderId]}"; "jsonpath needs the JSON
> text, move it before the unmarshal or use simple on the Map"; "the body is
> the file (GenericFile): convert it with convertBodyTo String or unmarshal it
> before the Groovy expression reads body.email"; "after marshal the body is
> bytes (a cached stream when stream caching is on): a Groovy script gets it
> with exchange.message.getBody(byte[])". Unknown types (a bean with no source)
> stop the flow silently, no false positives.
> # *At runtime, from the message history.* The message history already
> records, per step, what went through, including the class of the body at each
> previous step. Use it in two places: (a) the exception message of a failing
> expression or bean invocation can say "the body reaching this step was a
> java.util.LinkedHashMap (set by unmarshal at line 12)", the fact a person
> otherwise gets from a debugger; (b) {{camel trace}} and the camel-jbang-mcp
> tools that show a message's history can show the body type per step, so the
> flow is visible on a running app, and the static half can be checked against
> it (a recorded history of the reference run is the ground truth for the
> catalog's type rules).
> The static half is where most of the value is (it reaches the file before it
> runs); the runtime half makes the rule set honest and gives the "what is the
> body here" answer on a live app.
> Context: the round-2 local-model benchmark on the camel-jbang-examples ladder
> (2026-09-19); the pattern was in aggregator, order-lines, csv-to-json,
> groovy, openapi-client, filter-and-multicast, json-transform.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
