[
https://issues.apache.org/jira/browse/CAMEL-25334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen reassigned CAMEL-25334:
-----------------------------------
Assignee: Claus Ibsen
> payloadMode=json for low-code: inside the route the body is JSON data
> (JsonObject / JsonArray), at the edges JSON text
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25334
> URL: https://issues.apache.org/jira/browse/CAMEL-25334
> Project: Camel
> Issue Type: New Feature
> Components: camel-core, camel-groovy
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> Low-code users, and AI models writing routes, think of the payload as JSON
> and write data mappings in Groovy and simple accordingly. Camel carries the
> native Java type of each component instead - a GenericFile, a byte[], an
> InputStream, a Map - and that type changes during routing and from component
> to component. The gap shows up as runtime errors:
> * Groovy {{body.find \{ it.sku == headers.sku \}}} on JSON that was not
> unmarshalled: {{No such property: sku for class: java.lang.Byte}}
> * simple {{$\{body[amount]\}}} on the file consumer's body: {{Key: amount not
> found in bean: GenericFile[...]}}
> * a mapping that returns a Map, sent to an http producer: no type converter
> from LinkedHashMap to InputStream
> * a jsonpath result stored in a header as a LinkedList, which disappears when
> the message crosses a JMS broker
> Since then the hints of CAMEL-25330, CAMEL-25746 and CAMEL-25749 say what to
> do in each of these cases, but nothing does it: on main {{$\{body.sku\}}},
> {{$\{body[sku]\}}} and Groovy {{body.sku}} on JSON text all fail with
> "unmarshal it first".
> Raymond Meester described the same from the Fluxygen low-code platform on
> Camel, where users write Groovy and simple without knowing Java:
> [Text-orientated vs. Object-orientated in
> integration|https://camel.zulipchat.com/#narrow/channel/257298-camel/topic/.E2.9C.94.20Text-orientated.20vs.2E.20Object-orientated.20in.20integration/with/568024059].
> In the local-model benchmark, 19 of 156 attempts in one series died on such
> a body type collision.
> h3. Design: payloadMode
> An opt-in mode where the route works on the payload as JSON, instead of the
> various Java types that change during routing or from different components.
> It is for low-code users and AI-built routes; classic routes keep the native
> body types, and the mode is not meant for routes that want the lowest
> overhead.
> *The switch*, like stream caching: {{payloadMode}} with the values
> {{default}} (today's behaviour) and {{json}}, on the CamelContext
> ({{camel.main.payloadMode = json}}) and per route, so a route can opt in or
> out. An enum leaves room for other formats (XML) later.
> *Inside the route the body is JSON data*: a {{JsonObject}} or {{JsonArray}}
> of camel-util-json ({{JsonObject}} is a LinkedHashMap, so the field order is
> kept; numbers are BigDecimal). So with no change to the languages or EIPs:
> * Groovy: {{body.sku}}, {{body.items.each \{ ... \}}}, {{body.find \{ ...
> \}}} work
> * simple: {{$\{body.sku\}}} and {{$\{body[sku]\}}} work (the bean OGNL reads
> a key of a Map)
> * split of a JsonArray gives its elements, each a JsonObject; aggregating
> them gives a List, which becomes a JsonArray
> *Which bodies become JSON data*: the mode works where stream caching does,
> when the consumer hands the message to the route and after every step, so one
> rule covers the consumer's body, the reply of a producer and the result of
> any step:
> ||body||in json mode||
> |Map, List (Groovy, process, sql, split, aggregate)|becomes JsonObject /
> JsonArray|
> |String, byte[], stream, file with a Content-Type of application/json or
> *+json|parsed|
> |the same without a Content-Type|parsed when the file name ends with .json
> (MimeTypeHelper) or the text starts with \{ or [|
> |another Content-Type (text/csv, application/xml, image/png ...)|as it is|
> |JSON-looking text that does not parse ({{[INFO] started}})|as it is (logged
> at DEBUG): the mode never fails a message|
> |a stream that stream caching spooled to disk|as it is: the spool threshold
> is the size limit, no new option|
> |other Java objects (SDK objects, entities, JsonNode)|as they are (a JSON
> writer SPI, for example from camel-jackson, can come later)|
> A step that produces JSON text, including {{convertBodyTo: String}}, has it
> parsed back into data; the text inside the route is {{$\{bodyAs(String)\}}},
> or {{body.toJson()}} in Groovy.
> *At the edges it is JSON text*:
> * new type converters from JsonObject / JsonArray to String, byte[] and
> InputStream, so http, file, kafka and the other producers that read text,
> bytes or a stream get the JSON text
> * producers that take a Map as their API use the data: sql named parameters
> ({{:#sku}}), mongodb documents
> * producers that would turn a Map into something that is not the JSON text
> are fixed: JMS / SJMS send a JsonObject as a TextMessage, not a MapMessage
> (other producers to be surveyed)
> * header values that are a JsonObject, JsonArray, Map or List become JSON
> text before a producer (not the Camel* headers), so they survive a JMS or
> Kafka hop
> * direct / seda to another route pass the data as it is
> *What users see is JSON text*: logs, tracing, the backlog tracer, the dev
> consoles, Camel TUI and the MCP tools show the JSON text of the body (in
> MessageHelper, or through a small display SPI).
> h3. Plan
> # camel-core: the payloadMode option (camel-api, camel-main, the model and
> the YAML / XML DSL), the hook at the consumer and after every step with the
> rule above, the converters to text, the header values before a producer, the
> display, a user manual page, and tests that run the cases of the problem list
> in both modes.
> # components: JMS / SJMS send a JsonObject as a TextMessage, and a survey of
> the producers that turn a Map into something else.
> # tooling: the body flow of the validator, the catalog and Camel TUI / MCP
> know payloadMode (in json mode the body of a file consumer is JSON data).
> # Run the local-model benchmark with the mode off and on, and compare the
> body type collisions and the pass rate.
> Related: CAMEL-24844 (body type flow), CAMEL-25330 (the Groovy hint this
> replaces with behaviour in json mode), CAMEL-25329 (marshal json writes JSON
> text as it is).
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
