[
https://issues.apache.org/jira/browse/CAMEL-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118134#comment-18118134
]
Claus Ibsen commented on CAMEL-24909:
-------------------------------------
>From the s11 benchmark series (5 runs of the four HTTP rungs, 157 edit calls):
>25 of them answered not-found. The traces show three causes, all now handled:
* 4 calls passed a find whose JSON escapes were never turned back into
characters, so it held a literal \\n where the file has a newline. The tool now
retries the find (and the replacement) read the way it was meant.
* When the lines match at another indentation (the trimmed match), the
replacement used to be put in exactly as written, which landed valid YAML at
the wrong depth - part of the 36 invalid results. It is now re-indented to the
indentation the file has at that place.
* 16 misses came with no nearest block at all, because the model composed the
snippet from the shape it had in mind. A miss on a file of at most 400 lines
now hands the file back in fileContent, so the next attempt copies the text
instead of composing it again.
In PR https://github.com/apache/camel/pull/26750 with tests. A new 5-run series
(s12) measures the effect against s11 (28/65 steps, 43%).
> camel-jbang - an edit tool for the authoring set: replace a snippet instead
> of rewriting the whole file
> -------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24909
> URL: https://issues.apache.org/jira/browse/CAMEL-24909
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang
> Reporter: Claus Ibsen
> Priority: Major
> Fix For: 4.23.0
>
>
> {{camel_write_file}} takes the complete content of a file, so adding one step
> to a route means re-emitting every line of it. In the camel-jbang-mcp
> stepwise benchmark that is where a local model corrupts lines it never meant
> to touch.
> Evidence (connect/http-client, three focused runs after CAMEL-24906): the
> example's starting file contains
> {code}
> expression: "$[?(@.sku == '${header.sku}')]"
> {code}
> which the model must reproduce while adding a split to a *different* route in
> the same file. In 9 of 9 attempts it wrote
> {code}
> expression: "$[?(@.sku == '${header.sku}')]""
> {code}
> 30 refused writes in three runs, all the same line. The validator's message
> is now exact ("the value of expression ends with two double quotes; remove
> the extra one and write the line as ..."), and the model reads it correctly,
> saying "I'm still writing `...` with two double quotes at the end", and then
> emits the same bytes again. It cannot copy that line; it can write it: in
> connect/stock-api, where the same expression is written fresh rather than
> copied, it passed.
> So the missing tool is a targeted edit: replace a unique snippet with new
> text, or insert steps into a named route, validate the result and write it,
> instead of the whole file. Every coding agent has one (Claude Code's Edit
> takes old_string/new_string and refuses when it is not unique). It removes
> this class of corruption, and for a 60-line file it is a fraction of the
> tokens.
> Proposal: {{camel_edit_file}} (directory, file, find, replace, optional
> count) in the shared authoring set, next to {{camel_write_file}}: the find
> text must occur exactly once, the result is validated as the write is, and
> the answer says which lines changed. The reload outcome comes back as it does
> for a write (CAMEL-24859).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
