[
https://issues.apache.org/jira/browse/CAMEL-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18114639#comment-18114639
]
Claus Ibsen edited comment on CAMEL-24698 at 9/13/26 9:55 AM:
--------------------------------------------------------------
Item 6, found in run 4 of the benchmark (2026-09-12): a route with
aggregationStrategy: myAggregator (or bean: {ref: x}, process: {ref: x}, a
strategy or processor ref anywhere) validates, and fails at startup with
"Cannot find AggregationStrategy in Registry with name: myAggregator" because
no bean of that name is declared. A model writes the Java class next to the
YAML and assumes the file name is the bean name. The shared validator
(camel_validate_source, camel validate yaml) could cross-check every ref-style
option in the file against the beans: entries of the same file and
#class:/#type: values, and report "bean 'myAggregator' is not declared: add it
under - beans: (name, type: #class:com.example.MyAggregator), or use
#class:com.example.MyAggregator directly" as a warning (beans can also come
from other files or the registry, so not an error). The aggregator example
failed on this in three of four runs, three attempts each.
was (Author: davsclaus):
Item 6, found in run 4 of the benchmark (2026-09-12): a route with
aggregationStrategy: myAggregator (or bean: {ref: x}, process: {ref: x}, a
strategy or processor ref anywhere) validates, and fails at startup with
"Cannot find AggregationStrategy in Registry with name: myAggregator" because
no bean of that name is declared. A model writes the Java class next to the
YAML and assumes the file name is the bean name. The shared validator
(camel_validate_source, camel validate yaml, the TUI) could cross-check every
ref-style option in the file against the beans: entries of the same file and
#class:/#type: values, and report "bean 'myAggregator' is not declared: add it
under - beans: (name, type: #class:com.example.MyAggregator), or use
#class:com.example.MyAggregator directly" as a warning (beans can also come
from other files or the registry, so not an error). The aggregator example
failed on this in three of four runs, three attempts each.
> camel-jbang: follow-ups from the AI agent benchmarks (optional MCP args, CLI
> validate parity, eval placeholders, validator hints)
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24698
> URL: https://issues.apache.org/jira/browse/CAMEL-24698
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang, camel-yaml-dsl
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
>
> Follow-ups from benchmarking AI agents (a frontier model and a 22 GB local
> model) editing Camel integrations through the Camel MCP tools, 2026-09-11/12.
> The bugs the benchmarks found first are tracked separately (CAMEL-24692,
> CAMEL-24693, CAMEL-24694, CAMEL-24695); this issue collects the remaining,
> smaller items so none of them is lost. Each can be fixed on its own.
> h3. 1. Older camel-jbang-mcp wrappers declare every argument required
> 69 of the 82 tools in camel-jbang-mcp mark every parameter as required in
> their {{@ToolArg}} declarations, so the Quarkus MCP server rejects any call
> that omits an optional one with "Missing required argument", and the
> documented defaults cannot be used. Example: {{camel_catalog_component_doc}}
> with only {{name}} fails with "Missing required argument: component".
> CAMEL-24695 fixed this for the eleven shared authoring tools and added a test
> that keeps their flags equal to the shared descriptors; the older wrappers
> ({{CatalogTools}}, {{DiagnoseTools}} except {{camel_error_diagnose}},
> {{RuntimeTools}}, security, migration, OpenAPI, dependency tools) need
> {{required = false}} on every optional argument. A small model sends only the
> arguments it needs, so today it cannot use most of these tools.
> h3. 2. camel validate yaml does not check Simple expressions
> The CLI {{camel validate yaml}} validates the YAML DSL schema only. The Camel
> TUI's save-time validation and the shared {{camel_validate_source}} tool
> (CAMEL-24695, {{SourceValidator}} in camel-jbang-core) also check endpoint
> URIs, Simple expressions and {{camel.*}} properties. A frontier model shipped
> {{${padding(n)}}} (not a Simple function) past the CLI validator; the runtime
> caught it. {{camel validate yaml}} should use the shared {{SourceValidator}}
> so all three entry points report the same problems.
> h3. 3. Expression evaluation does not resolve property placeholders
> {{tui_eval_expression}} (now the shared {{camel_eval_expression}}) evaluated
> {{${body} >= {{hot.threshold}}}} as written and reported a parse error, while
> the same expression works in a route because placeholders are resolved before
> Simple is parsed. The evaluator should resolve placeholders from the running
> integration's properties (or from the project's application.properties when
> evaluating locally) before evaluating, or say clearly that a placeholder was
> left unresolved. Re-check after CAMEL-24692.
> h3. 4. Validator error messages should say what to do
> The YAML DSL validator returns the raw schema error. Two cases from the
> benchmark where the message stopped a small model and would slow a person:
> * "log: property 'loggerName' is not defined in the schema" should add "did
> you mean logName" (the edit-distance suggestion from CAMEL-24666 already
> exists in the catalog).
> * "route: property 'onException' is not defined in the schema" should add
> that onException is a top-level entry, a sibling of route, and show a
> five-line sample once CAMEL-24693 ships them.
> camel-yaml-dsl-validator already depends on camel-catalog, and the YAML DSL
> model (CAMEL-24362) knows which entries are top-level and which nest where,
> so this is a post-processing step over the networknt errors, no schema change.
> h3. Origin
> Rows 12 to 15 of the defects tally kept with the benchmark harness. Related:
> CAMEL-24692, CAMEL-24693, CAMEL-24694, CAMEL-24695, CAMEL-24666, CAMEL-24362.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
