Claus Ibsen created CAMEL-25532:
-----------------------------------
Summary: camel-jbang-mcp - lenient input and anchored edits, so a
model adds or changes a few EIPs without getting the YAML layout right
Key: CAMEL-25532
URL: https://issues.apache.org/jira/browse/CAMEL-25532
Project: Camel
Issue Type: Improvement
Components: camel-jbang
Reporter: Claus Ibsen
Fix For: 4.24.0
In the local-model benchmark (the stepwise examples ladder, qwen3.6 35B through
the camel-jbang-mcp server), most failed writes are about the layout of the
YAML, not about Camel. In the r7 run and a later k=1 run: 979 calls of
camel_write_file / camel_edit_file, 238 refused by the validator, 172 of them
(72%) only for layout: a key at the wrong level (steps: next to split:,
parameters: next to to:), unquoted braces ({{placeholder}}, {sku}), a plain
value where the DSL wants an expression (handled: true), otherwise: as a list,
an option of redeliveryPolicy written on onException. Of those, about 61 are
YAML syntax/indentation and about 95 are the shape of the DSL. A model that
gets one of these wrong often writes the same YAML again and again until it
runs out of tool calls.
The stepwise case is the common one: the model adds or changes a few EIPs in an
existing route, it does not write the whole file. Today it does so with
find/replace, and the replace text must reproduce the exact indentation of the
place it goes to.
Proposal:
1. Lenient input: accept the shapes models write (as observed in the benchmark
traces) and store canonical YAML. Each rule is structural and has one reading
only (never rename by similarity: the cronService / level quick fixes of #27666
show where that goes); an ambiguity is an error with both readings; the answer
reports what was expanded. Applies to camel_write_file and camel_edit_file, and
can serve camel validate --fix. JSON is accepted as it is valid YAML. The file
on disk stays canonical, so the next read shows the model the right form.
2. Anchored edits in camel_edit_file: the model gives where (a line number plus
the text of that line, or a selector such as "to kafka:orders" / "log in route
check-stock") and a fragment of a few EIPs at any indentation; the server
resolves the anchor to a node (the text is the anchor, the line number breaks
ties: in 79 route files 67% of the lines that carry a value are unique in their
file), re-indents the fragment to the node, applies the lenient rules in that
context, validates and splices it in, leaving the rest of the file byte for
byte. Modes: at (set options of the node), before / after / inside (insert
EIPs). A miss or a tie answers with the candidate lines.
3. camel_get_files can return numbered lines, so line anchors and the
validator's "Line N:" messages meet.
4. Measure each with a harness arm against the r9 run (Oct 10 2026), with
qwen3.6 and a weaker local model.
Analysis and local experiments follow as comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
