[
https://issues.apache.org/jira/browse/CAMEL-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18114841#comment-18114841
]
Claus Ibsen commented on CAMEL-24698:
-------------------------------------
Status 2026-09-13 (the work of the benchmark series, split into PRs):
Merged to main:
* https://github.com/apache/camel/pull/26369 camel-jbang-mcp: runtime,
camelVersion, platformBom, filter, label and limit optional on the catalog
tools; a non-version means the default catalog; EIP names in any case
* https://github.com/apache/camel/pull/26370 camel_eval_expression resolves
property placeholders
* https://github.com/apache/camel/pull/26371 camel-catalog: the Simple bean
function validates without a registry
Open:
* https://github.com/apache/camel/pull/26362 camel-yaml-dsl-validator: schema
errors say what to write (step, list, property, expression and script hints,
required expressions, the file's shape)
* https://github.com/apache/camel/pull/26368 runtime messages: missing
expression, xslt and file null body, groovy missing property, POJO aggregation
strategy null body, timer header metadata
To open, in order (each on the previous): the CAMEL-24695 optional-arguments
follow-up; the camel_catalog_sample tool (CAMEL-24693); camel validate yaml and
camel_validate_yaml_dsl running the catalog checks; the write-time checks
(endpoints, Simple, headers, bean references, Java, XSLT/XML,
application.properties) with SourceValidator split into one class per check.
Still to do after that: the table-driven form of the validator hints (the
tidy-up noted above), and CAMEL-24709 for the data-driven class-not-found hint.
> camel-jbang: follow-ups from the AI agent benchmarks (optional MCP args, CLI
> validate parity, eval placeholders, validator hints)
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24698
> URL: https://issues.apache.org/jira/browse/CAMEL-24698
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang, camel-yaml-dsl
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
>
> Follow-ups from benchmarking AI agents (a frontier model and a 22 GB local
> model) editing Camel integrations through the Camel MCP tools, 2026-09-11/12.
> The bugs the benchmarks found first are tracked separately (CAMEL-24692,
> CAMEL-24693, CAMEL-24694, CAMEL-24695); this issue collects the remaining,
> smaller items so none of them is lost. Each can be fixed on its own.
> h3. 1. Older camel-jbang-mcp wrappers declare every argument required
> 69 of the 82 tools in camel-jbang-mcp mark every parameter as required in
> their {{@ToolArg}} declarations, so the Quarkus MCP server rejects any call
> that omits an optional one with "Missing required argument", and the
> documented defaults cannot be used. Example: {{camel_catalog_component_doc}}
> with only {{name}} fails with "Missing required argument: component".
> CAMEL-24695 fixed this for the eleven shared authoring tools and added a test
> that keeps their flags equal to the shared descriptors; the older wrappers
> ({{CatalogTools}}, {{DiagnoseTools}} except {{camel_error_diagnose}},
> {{RuntimeTools}}, security, migration, OpenAPI, dependency tools) need
> {{required = false}} on every optional argument. A small model sends only the
> arguments it needs, so today it cannot use most of these tools.
> h3. 2. camel validate yaml does not check Simple expressions
> The CLI {{camel validate yaml}} validates the YAML DSL schema only. The Camel
> TUI's save-time validation and the shared {{camel_validate_source}} tool
> (CAMEL-24695, {{SourceValidator}} in camel-jbang-core) also check endpoint
> URIs, Simple expressions and {{camel.*}} properties. A frontier model shipped
> {{${padding(n)}}} (not a Simple function) past the CLI validator; the runtime
> caught it. {{camel validate yaml}} should use the shared {{SourceValidator}}
> so all three entry points report the same problems.
> h3. 3. Expression evaluation does not resolve property placeholders
> {{tui_eval_expression}} (now the shared {{camel_eval_expression}}) evaluated
> {{${body} >= {{hot.threshold}}}} as written and reported a parse error, while
> the same expression works in a route because placeholders are resolved before
> Simple is parsed. The evaluator should resolve placeholders from the running
> integration's properties (or from the project's application.properties when
> evaluating locally) before evaluating, or say clearly that a placeholder was
> left unresolved. Re-check after CAMEL-24692.
> h3. 4. Validator error messages should say what to do
> The YAML DSL validator returns the raw schema error. Two cases from the
> benchmark where the message stopped a small model and would slow a person:
> * "log: property 'loggerName' is not defined in the schema" should add "did
> you mean logName" (the edit-distance suggestion from CAMEL-24666 already
> exists in the catalog).
> * "route: property 'onException' is not defined in the schema" should add
> that onException is a top-level entry, a sibling of route, and show a
> five-line sample once CAMEL-24693 ships them.
> camel-yaml-dsl-validator already depends on camel-catalog, and the YAML DSL
> model (CAMEL-24362) knows which entries are top-level and which nest where,
> so this is a post-processing step over the networknt errors, no schema change.
> h3. Origin
> Rows 12 to 15 of the defects tally kept with the benchmark harness. Related:
> CAMEL-24692, CAMEL-24693, CAMEL-24694, CAMEL-24695, CAMEL-24666, CAMEL-24362.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
