[
https://issues.apache.org/jira/browse/CAMEL-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18114771#comment-18114771
]
Claus Ibsen commented on CAMEL-24698:
-------------------------------------
Tidy-up to do before the validator-hint branches go upstream (noted 2026-09-13
after benchmark run 18):
The hints in camel-yaml-dsl-validator (YamlValidator) and camel-jbang-core
(SourceValidator) are a mix of data-driven and hardcoded rules. Data-driven:
did-you-mean from the schema's own properties, top-level entries, language
keys, step names, resilience4j options (read from the schema at init), and 17
catalog lookups in SourceValidator (component/endpoint options, main
configuration groups, component headers, beans by interface). Hardcoded: 18
hint branches on 22 location patterns and 14 literal sets/maps in YamlValidator
(beans shape, name vs id, bean: as a language, header name as key, steps at
route level, otherwise/when without steps, log component options on the log
EIP, circuitBreaker options, exception class as key, expression examples per
EIP, script languages), and 12 sets/maps in SourceValidator including a
17-entry table of invented endpoint options, the required interface per option,
the configuration group ranking and the common exchange headers. Every rule has
a test (83 + 72).
Plan:
1. Move the per-mistake rules into a resource file in the validator module (a
JSON list of location pattern + unknown name or keyword + message); Java keeps
the generic mechanics (closest name, schema lookups, noise filters). Adding a
hint then needs no code and the wording can be reviewed as text.
2. Replace three lists by lookups: "X is an option of the <component>, not of
the <EIP>" for any EIP sharing a name with a component; the invented-options
table by "is it an option of another component or of the EIP of that name" from
the catalog; the EXPRESSION_REQUIRED set by the schema's required (redundant
since CAMEL-24707).
3. Keep the one-off wordings about the file itself (//DEPS, the ... marker, the
beans shape).
About a day of work, one tidy-up commit per branch, after the benchmark series.
> camel-jbang: follow-ups from the AI agent benchmarks (optional MCP args, CLI
> validate parity, eval placeholders, validator hints)
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24698
> URL: https://issues.apache.org/jira/browse/CAMEL-24698
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang, camel-yaml-dsl
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
>
> Follow-ups from benchmarking AI agents (a frontier model and a 22 GB local
> model) editing Camel integrations through the Camel MCP tools, 2026-09-11/12.
> The bugs the benchmarks found first are tracked separately (CAMEL-24692,
> CAMEL-24693, CAMEL-24694, CAMEL-24695); this issue collects the remaining,
> smaller items so none of them is lost. Each can be fixed on its own.
> h3. 1. Older camel-jbang-mcp wrappers declare every argument required
> 69 of the 82 tools in camel-jbang-mcp mark every parameter as required in
> their {{@ToolArg}} declarations, so the Quarkus MCP server rejects any call
> that omits an optional one with "Missing required argument", and the
> documented defaults cannot be used. Example: {{camel_catalog_component_doc}}
> with only {{name}} fails with "Missing required argument: component".
> CAMEL-24695 fixed this for the eleven shared authoring tools and added a test
> that keeps their flags equal to the shared descriptors; the older wrappers
> ({{CatalogTools}}, {{DiagnoseTools}} except {{camel_error_diagnose}},
> {{RuntimeTools}}, security, migration, OpenAPI, dependency tools) need
> {{required = false}} on every optional argument. A small model sends only the
> arguments it needs, so today it cannot use most of these tools.
> h3. 2. camel validate yaml does not check Simple expressions
> The CLI {{camel validate yaml}} validates the YAML DSL schema only. The Camel
> TUI's save-time validation and the shared {{camel_validate_source}} tool
> (CAMEL-24695, {{SourceValidator}} in camel-jbang-core) also check endpoint
> URIs, Simple expressions and {{camel.*}} properties. A frontier model shipped
> {{${padding(n)}}} (not a Simple function) past the CLI validator; the runtime
> caught it. {{camel validate yaml}} should use the shared {{SourceValidator}}
> so all three entry points report the same problems.
> h3. 3. Expression evaluation does not resolve property placeholders
> {{tui_eval_expression}} (now the shared {{camel_eval_expression}}) evaluated
> {{${body} >= {{hot.threshold}}}} as written and reported a parse error, while
> the same expression works in a route because placeholders are resolved before
> Simple is parsed. The evaluator should resolve placeholders from the running
> integration's properties (or from the project's application.properties when
> evaluating locally) before evaluating, or say clearly that a placeholder was
> left unresolved. Re-check after CAMEL-24692.
> h3. 4. Validator error messages should say what to do
> The YAML DSL validator returns the raw schema error. Two cases from the
> benchmark where the message stopped a small model and would slow a person:
> * "log: property 'loggerName' is not defined in the schema" should add "did
> you mean logName" (the edit-distance suggestion from CAMEL-24666 already
> exists in the catalog).
> * "route: property 'onException' is not defined in the schema" should add
> that onException is a top-level entry, a sibling of route, and show a
> five-line sample once CAMEL-24693 ships them.
> camel-yaml-dsl-validator already depends on camel-catalog, and the YAML DSL
> model (CAMEL-24362) knows which entries are top-level and which nest where,
> so this is a post-processing step over the networknt errors, no schema change.
> h3. Origin
> Rows 12 to 15 of the defects tally kept with the benchmark harness. Related:
> CAMEL-24692, CAMEL-24693, CAMEL-24694, CAMEL-24695, CAMEL-24666, CAMEL-24362.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
