[
https://issues.apache.org/jira/browse/CAMEL-25541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Mariani updated CAMEL-25541:
-------------------------------------
Description:
Three findings from an evaluation of coding agents (Claude Code and IBM Bob) on
Mule-to-Camel migration tasks with camel-jbang-mcp 4.22.1 concern the same
answer, the component branch of {{camel_catalog_doc}}
({{dsl/camel-jbang/camel-jbang-core/.../commands/ai/CatalogDocs.java}}), and
are fixed together here. CAMEL-25540 and CAMEL-25542 were merged into this
issue.
h3. 1. The option list has no bound (15-35 K characters by default)
{{putOptions}} ({{CatalogDocs.java:1753-1781}}) adds every option the
{{OptionScope}} accepts; {{common}} only drops deprecated and advanced options,
so activemq still has 88 options (about 38 K of JSON), file 75 (35 K), smtp 76
(24 K), http 60 (19 K). Recorded sizes with 4.22.1:
{noformat}
camel_catalog_component_doc activemq 32 959 chars (includeOptions=all: 84
212, rejected by the client)
camel_catalog_component_doc file 31 423
camel_catalog_component_doc smtp 20 611
camel_component_properties activemq 123 893 "exceeds maximum allowed tokens"
camel_component_properties platform-http 14 996, called 35 times to find the
HTTP port, which is camel.server.* (CAMEL-25533)
{noformat}
121 of 192 sessions got at least one result over 20 000 characters; Claude Code
rejected 4 and wrote 48 to a file instead of showing them to the model.
h3. 2. Nothing matches, and the answer says nothing
25 of 81 calls with {{optionsFilter}} came back with {{"matchedOptions": 0}}
and empty lists, and the agents read that as "unsupported": on smtp with
{{optionsFilter: tls}} (15 sessions) several decided "no TLS support"; on xslt
with {{optionsFilter: param}} the agents concluded XSLT takes no parameters and
one moved to FreeMarker; on activemq {{redeliver}} found nothing. On main the
filter also searches descriptions, groups and labels, which does not close it:
smtp + {{tls}} returns only {{useJavaMailSessionPropertiesFromHeaders}}, the
option whose own description warns against this use, while the supported way
({{mail.smtp.starttls.enable=true}} as a URI parameter, which the catalog knows
as the multi-value prefix option {{additionalJavaMailProperties}}) is not
shown; xslt + {{param}} still matches nothing. (A third case,
{{schemaValidation}} with {{includeOptions: common}} finding nothing because
the option is {{advanced}}, is fixed on main: the filter now searches all
options.)
Names that still do not resolve on main: the page names {{camel_catalog_docs}}
returns ({{wireTap-eip}}, {{jackson-dataformat}}) end in "Artifact not found"
because the {{-eip}}/{{-component}}/{{-dataformat}}/{{-language}} suffix is not
stripped ({{catalogDoc}}, {{:187-300}}); {{retry}} resolves to
{{optimisticLockRetryPolicy}} by a title word ({{eipByTerm}}, {{:431-440}},
{{suggestEipNames(term, 1)}}) instead of pointing at the error handler;
{{camel_catalog_example_file}} with an unknown example answers "Example not
found: X" without the names ({{ExampleTools.java:113}}).
h3. 3. The page sections that decide correctness are never in the answer
{{docExcerpt}} ({{CatalogDocs.java:138}}, {{DOC_EXCERPT_BUDGET = 1400}})
carries the first {{==}} section only. Three behaviours that decide whether a
route is correct live further down the page and were never seen by any session:
* xslt: "By default, all headers and variables are added as parameters"
(section "Getting Usable Parameters into the XSLT", line 129 of 442). One
session rewrote the transformation in FreeMarker.
* mail: the Subject/From/To headers are ignored unless
{{useHeaderSubject}}/{{useHeaderFrom}}/{{useHeaderRecipients}} is true (section
"Opting out of header overrides", line ~300). 11 sessions sent a mail with an
empty subject. The option description itself is CAMEL-25534.
* platform-http under {{camel run}}: {{camel.server.port}} needs
{{camel.server.enabled=true}}. Only {{platform-http-main.adoc}} says so and
{{platform-http-component.adoc}} has no {{xref}} to it, so {{docPages}}
({{componentDocPages}}, {{:1432}}) is empty. 5 of 10 sessions on one task
listened on 8080 after 15-47 calls each.
h3. Fix
# *Bound the default answer*: without {{optionsFilter}}, return the required
options plus the first N common options (N about 25), with {{truncated: true}},
the total count and a hint ("optionsFilter=<word> finds one; includeOptions=all
with a limit pages the rest"). Alternatively names only (name, type, default)
when no filter is given, descriptions for filtered calls. Optionally a hard cap
on any catalog tool result (16 000 characters) with a truncation marker.
# *Say when nothing matches*: {{"note": "no option matches 'tls'"}} plus the
closest option names by edit distance (the catalog has the suggester since
CAMEL-24666) with their group; the multi-value prefix options of the component
({{additionalJavaMailProperties}}: "any mail.* URI parameter is passed to
JavaMail"), also when the filter matched something.
# *Make the page discoverable*: return the page's section titles ({{"sections":
[...]}}, with the ones whose text contains the filter term first) and a way to
fetch one ({{docPage=<section>}}), so a model that sees an empty filter can
still find the section that answers it. Optionally a short curated {{notes}}
list per component for defaults that silently change behaviour (mail
useHeader*, xslt header parameters), returned regardless of {{optionsFilter}}.
# *Resolve the names the tools themselves hand out*: strip a {{-component}},
{{-dataformat}}, {{-language}} or {{-eip}} suffix and look up again
(case-insensitive); do not resolve a bare word such as {{retry}} to an EIP by a
title word alone, return it as a suggestion with the error handler entries (or
add {{retry}}/{{redelivery}} as aliases of the error handler); list the closest
example names on "Example not found".
# *platform-http-component.adoc*: add a Camel Main / {{camel run}} entry to
"Platform HTTP Provider" (the first section, so it lands in the excerpt) with
{{camel.server.enabled=true}} + {{camel.server.port}}, and
{{xref:others:platform-http-main.adoc[...]}}, which {{componentDocPages}}
already turns into {{docPage=main}}.
# *Drop {{camel_component_properties}}* rather than add {{filter}}/{{limit}} to
it: its description says endpoint options "can be set as component-level
defaults via the same prefix", which is false
({{AbstractCamelCatalog.validateConfigurationProperty}} checks
{{camel.component.<scheme>.*}} against the component options only, {{:1221}}),
so the tool emits keys that {{camel_configuration_validate}} and {{camel run}}
reject. A {{configKey}} on each component option in {{camel_catalog_doc}}
covers the valid half.
Related: CAMEL-25539 (EIP/language answer: placement and YAML shapes),
CAMEL-25534, CAMEL-25533, CAMEL-25040 (the excerpt), CAMEL-25283
({{docPages}}), CAMEL-23475 (the scopes). Merged here: CAMEL-25540, CAMEL-25542.
h3. Added 2026-10-14: full-key lookup and token numbers from the evaluation
* {{camel_catalog_doc name=server}} reaches {{mainOptionsGroup}}
({{CatalogDocs.java:606}}) and answers the {{camel.server.*}} keys, but the
forms an agent actually asks for do not: a full key ({{camel.server.port}}
contains a dot after the group, so {{mainOptionsGroup}} returns null), a
component plus a word ({{platform-http}} + {{optionsFilter: port}}), and
{{camel.jbang.*}} ({{catalog.jbangModel()}} is not consulted by any tool).
Accept a full key and answer the group with the matching option first, and
route {{camel.jbang.*}} to the jbang model. In the evaluation the port question
alone cost 30 knowledge searches, 12 {{camel_component_properties
platform-http}} calls (15 K each, none naming it) and {{Run.class}} disassembly
in 35 plain-arm trials.
* Token numbers (Claude Code, 45 Haiku 5.5 trials in the MCP arm, 414 M
tokens): about 9% of all tokens are re-reads of MCP results on later model
calls, 4% of them from the 45 results over 20 000 characters (39 of those from
{{camel_catalog_component_doc}}, 4 from {{camel_component_properties}}); 125 of
564 results, 21% of all result bytes, were spilled to a file by the client and
read back with grep. Repeated identical calls: 9%. Sonnet made 0-2 MCP calls
per trial, so for it the bound changes nothing; CAMEL-25646 does.
_Claude Code on behalf of Croway_
was:
Three findings from an evaluation of coding agents (Claude Code and IBM Bob) on
Mule-to-Camel migration tasks with camel-jbang-mcp 4.22.1 concern the same
answer, the component branch of {{camel_catalog_doc}}
({{dsl/camel-jbang/camel-jbang-core/.../commands/ai/CatalogDocs.java}}), and
are fixed together here. CAMEL-25540 and CAMEL-25542 were merged into this
issue.
h3. 1. The option list has no bound (15-35 K characters by default)
{{putOptions}} ({{CatalogDocs.java:1753-1781}}) adds every option the
{{OptionScope}} accepts; {{common}} only drops deprecated and advanced options,
so activemq still has 88 options (about 38 K of JSON), file 75 (35 K), smtp 76
(24 K), http 60 (19 K). Recorded sizes with 4.22.1:
{noformat}
camel_catalog_component_doc activemq 32 959 chars (includeOptions=all: 84
212, rejected by the client)
camel_catalog_component_doc file 31 423
camel_catalog_component_doc smtp 20 611
camel_component_properties activemq 123 893 "exceeds maximum allowed tokens"
camel_component_properties platform-http 14 996, called 35 times to find the
HTTP port, which is camel.server.* (CAMEL-25533)
{noformat}
121 of 192 sessions got at least one result over 20 000 characters; Claude Code
rejected 4 and wrote 48 to a file instead of showing them to the model.
h3. 2. Nothing matches, and the answer says nothing
25 of 81 calls with {{optionsFilter}} came back with {{"matchedOptions": 0}}
and empty lists, and the agents read that as "unsupported": on smtp with
{{optionsFilter: tls}} (15 sessions) several decided "no TLS support"; on xslt
with {{optionsFilter: param}} the agents concluded XSLT takes no parameters and
one moved to FreeMarker; on activemq {{redeliver}} found nothing. On main the
filter also searches descriptions, groups and labels, which does not close it:
smtp + {{tls}} returns only {{useJavaMailSessionPropertiesFromHeaders}}, the
option whose own description warns against this use, while the supported way
({{mail.smtp.starttls.enable=true}} as a URI parameter, which the catalog knows
as the multi-value prefix option {{additionalJavaMailProperties}}) is not
shown; xslt + {{param}} still matches nothing. (A third case,
{{schemaValidation}} with {{includeOptions: common}} finding nothing because
the option is {{advanced}}, is fixed on main: the filter now searches all
options.)
Names that still do not resolve on main: the page names {{camel_catalog_docs}}
returns ({{wireTap-eip}}, {{jackson-dataformat}}) end in "Artifact not found"
because the {{-eip}}/{{-component}}/{{-dataformat}}/{{-language}} suffix is not
stripped ({{catalogDoc}}, {{:187-300}}); {{retry}} resolves to
{{optimisticLockRetryPolicy}} by a title word ({{eipByTerm}}, {{:431-440}},
{{suggestEipNames(term, 1)}}) instead of pointing at the error handler;
{{camel_catalog_example_file}} with an unknown example answers "Example not
found: X" without the names ({{ExampleTools.java:113}}).
h3. 3. The page sections that decide correctness are never in the answer
{{docExcerpt}} ({{CatalogDocs.java:138}}, {{DOC_EXCERPT_BUDGET = 1400}})
carries the first {{==}} section only. Three behaviours that decide whether a
route is correct live further down the page and were never seen by any session:
* xslt: "By default, all headers and variables are added as parameters"
(section "Getting Usable Parameters into the XSLT", line 129 of 442). One
session rewrote the transformation in FreeMarker.
* mail: the Subject/From/To headers are ignored unless
{{useHeaderSubject}}/{{useHeaderFrom}}/{{useHeaderRecipients}} is true (section
"Opting out of header overrides", line ~300). 11 sessions sent a mail with an
empty subject. The option description itself is CAMEL-25534.
* platform-http under {{camel run}}: {{camel.server.port}} needs
{{camel.server.enabled=true}}. Only {{platform-http-main.adoc}} says so and
{{platform-http-component.adoc}} has no {{xref}} to it, so {{docPages}}
({{componentDocPages}}, {{:1432}}) is empty. 5 of 10 sessions on one task
listened on 8080 after 15-47 calls each.
h3. Fix
# *Bound the default answer*: without {{optionsFilter}}, return the required
options plus the first N common options (N about 25), with {{truncated: true}},
the total count and a hint ("optionsFilter=<word> finds one; includeOptions=all
with a limit pages the rest"). Alternatively names only (name, type, default)
when no filter is given, descriptions for filtered calls. Optionally a hard cap
on any catalog tool result (16 000 characters) with a truncation marker.
# *Say when nothing matches*: {{"note": "no option matches 'tls'"}} plus the
closest option names by edit distance (the catalog has the suggester since
CAMEL-24666) with their group; the multi-value prefix options of the component
({{additionalJavaMailProperties}}: "any mail.* URI parameter is passed to
JavaMail"), also when the filter matched something.
# *Make the page discoverable*: return the page's section titles ({{"sections":
[...]}}, with the ones whose text contains the filter term first) and a way to
fetch one ({{docPage=<section>}}), so a model that sees an empty filter can
still find the section that answers it. Optionally a short curated {{notes}}
list per component for defaults that silently change behaviour (mail
useHeader*, xslt header parameters), returned regardless of {{optionsFilter}}.
# *Resolve the names the tools themselves hand out*: strip a {{-component}},
{{-dataformat}}, {{-language}} or {{-eip}} suffix and look up again
(case-insensitive); do not resolve a bare word such as {{retry}} to an EIP by a
title word alone, return it as a suggestion with the error handler entries (or
add {{retry}}/{{redelivery}} as aliases of the error handler); list the closest
example names on "Example not found".
# *platform-http-component.adoc*: add a Camel Main / {{camel run}} entry to
"Platform HTTP Provider" (the first section, so it lands in the excerpt) with
{{camel.server.enabled=true}} + {{camel.server.port}}, and
{{xref:others:platform-http-main.adoc[...]}}, which {{componentDocPages}}
already turns into {{docPage=main}}.
# *Drop {{camel_component_properties}}* rather than add {{filter}}/{{limit}} to
it: its description says endpoint options "can be set as component-level
defaults via the same prefix", which is false
({{AbstractCamelCatalog.validateConfigurationProperty}} checks
{{camel.component.<scheme>.*}} against the component options only, {{:1221}}),
so the tool emits keys that {{camel_configuration_validate}} and {{camel run}}
reject. A {{configKey}} on each component option in {{camel_catalog_doc}}
covers the valid half.
Related: CAMEL-25539 (EIP/language answer: placement and YAML shapes),
CAMEL-25534, CAMEL-25533, CAMEL-25040 (the excerpt), CAMEL-25283
({{docPages}}), CAMEL-23475 (the scopes). Merged here: CAMEL-25540, CAMEL-25542.
_Claude Code on behalf of Croway_
> camel-jbang-mcp - camel_catalog_doc component answer: bounded option lists,
> an explicit answer when nothing matches, and the page sections that decide
> correctness
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25541
> URL: https://issues.apache.org/jira/browse/CAMEL-25541
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang-mcp
> Affects Versions: 4.22.1
> Reporter: Federico Mariani
> Priority: Major
>
> Three findings from an evaluation of coding agents (Claude Code and IBM Bob)
> on Mule-to-Camel migration tasks with camel-jbang-mcp 4.22.1 concern the same
> answer, the component branch of {{camel_catalog_doc}}
> ({{dsl/camel-jbang/camel-jbang-core/.../commands/ai/CatalogDocs.java}}), and
> are fixed together here. CAMEL-25540 and CAMEL-25542 were merged into this
> issue.
> h3. 1. The option list has no bound (15-35 K characters by default)
> {{putOptions}} ({{CatalogDocs.java:1753-1781}}) adds every option the
> {{OptionScope}} accepts; {{common}} only drops deprecated and advanced
> options, so activemq still has 88 options (about 38 K of JSON), file 75 (35
> K), smtp 76 (24 K), http 60 (19 K). Recorded sizes with 4.22.1:
> {noformat}
> camel_catalog_component_doc activemq 32 959 chars (includeOptions=all: 84
> 212, rejected by the client)
> camel_catalog_component_doc file 31 423
> camel_catalog_component_doc smtp 20 611
> camel_component_properties activemq 123 893 "exceeds maximum allowed
> tokens"
> camel_component_properties platform-http 14 996, called 35 times to find
> the HTTP port, which is camel.server.* (CAMEL-25533)
> {noformat}
> 121 of 192 sessions got at least one result over 20 000 characters; Claude
> Code rejected 4 and wrote 48 to a file instead of showing them to the model.
> h3. 2. Nothing matches, and the answer says nothing
> 25 of 81 calls with {{optionsFilter}} came back with {{"matchedOptions": 0}}
> and empty lists, and the agents read that as "unsupported": on smtp with
> {{optionsFilter: tls}} (15 sessions) several decided "no TLS support"; on
> xslt with {{optionsFilter: param}} the agents concluded XSLT takes no
> parameters and one moved to FreeMarker; on activemq {{redeliver}} found
> nothing. On main the filter also searches descriptions, groups and labels,
> which does not close it: smtp + {{tls}} returns only
> {{useJavaMailSessionPropertiesFromHeaders}}, the option whose own description
> warns against this use, while the supported way
> ({{mail.smtp.starttls.enable=true}} as a URI parameter, which the catalog
> knows as the multi-value prefix option {{additionalJavaMailProperties}}) is
> not shown; xslt + {{param}} still matches nothing. (A third case,
> {{schemaValidation}} with {{includeOptions: common}} finding nothing because
> the option is {{advanced}}, is fixed on main: the filter now searches all
> options.)
> Names that still do not resolve on main: the page names
> {{camel_catalog_docs}} returns ({{wireTap-eip}}, {{jackson-dataformat}}) end
> in "Artifact not found" because the
> {{-eip}}/{{-component}}/{{-dataformat}}/{{-language}} suffix is not stripped
> ({{catalogDoc}}, {{:187-300}}); {{retry}} resolves to
> {{optimisticLockRetryPolicy}} by a title word ({{eipByTerm}}, {{:431-440}},
> {{suggestEipNames(term, 1)}}) instead of pointing at the error handler;
> {{camel_catalog_example_file}} with an unknown example answers "Example not
> found: X" without the names ({{ExampleTools.java:113}}).
> h3. 3. The page sections that decide correctness are never in the answer
> {{docExcerpt}} ({{CatalogDocs.java:138}}, {{DOC_EXCERPT_BUDGET = 1400}})
> carries the first {{==}} section only. Three behaviours that decide whether a
> route is correct live further down the page and were never seen by any
> session:
> * xslt: "By default, all headers and variables are added as parameters"
> (section "Getting Usable Parameters into the XSLT", line 129 of 442). One
> session rewrote the transformation in FreeMarker.
> * mail: the Subject/From/To headers are ignored unless
> {{useHeaderSubject}}/{{useHeaderFrom}}/{{useHeaderRecipients}} is true
> (section "Opting out of header overrides", line ~300). 11 sessions sent a
> mail with an empty subject. The option description itself is CAMEL-25534.
> * platform-http under {{camel run}}: {{camel.server.port}} needs
> {{camel.server.enabled=true}}. Only {{platform-http-main.adoc}} says so and
> {{platform-http-component.adoc}} has no {{xref}} to it, so {{docPages}}
> ({{componentDocPages}}, {{:1432}}) is empty. 5 of 10 sessions on one task
> listened on 8080 after 15-47 calls each.
> h3. Fix
> # *Bound the default answer*: without {{optionsFilter}}, return the required
> options plus the first N common options (N about 25), with {{truncated:
> true}}, the total count and a hint ("optionsFilter=<word> finds one;
> includeOptions=all with a limit pages the rest"). Alternatively names only
> (name, type, default) when no filter is given, descriptions for filtered
> calls. Optionally a hard cap on any catalog tool result (16 000 characters)
> with a truncation marker.
> # *Say when nothing matches*: {{"note": "no option matches 'tls'"}} plus the
> closest option names by edit distance (the catalog has the suggester since
> CAMEL-24666) with their group; the multi-value prefix options of the
> component ({{additionalJavaMailProperties}}: "any mail.* URI parameter is
> passed to JavaMail"), also when the filter matched something.
> # *Make the page discoverable*: return the page's section titles
> ({{"sections": [...]}}, with the ones whose text contains the filter term
> first) and a way to fetch one ({{docPage=<section>}}), so a model that sees
> an empty filter can still find the section that answers it. Optionally a
> short curated {{notes}} list per component for defaults that silently change
> behaviour (mail useHeader*, xslt header parameters), returned regardless of
> {{optionsFilter}}.
> # *Resolve the names the tools themselves hand out*: strip a {{-component}},
> {{-dataformat}}, {{-language}} or {{-eip}} suffix and look up again
> (case-insensitive); do not resolve a bare word such as {{retry}} to an EIP by
> a title word alone, return it as a suggestion with the error handler entries
> (or add {{retry}}/{{redelivery}} as aliases of the error handler); list the
> closest example names on "Example not found".
> # *platform-http-component.adoc*: add a Camel Main / {{camel run}} entry to
> "Platform HTTP Provider" (the first section, so it lands in the excerpt) with
> {{camel.server.enabled=true}} + {{camel.server.port}}, and
> {{xref:others:platform-http-main.adoc[...]}}, which {{componentDocPages}}
> already turns into {{docPage=main}}.
> # *Drop {{camel_component_properties}}* rather than add {{filter}}/{{limit}}
> to it: its description says endpoint options "can be set as component-level
> defaults via the same prefix", which is false
> ({{AbstractCamelCatalog.validateConfigurationProperty}} checks
> {{camel.component.<scheme>.*}} against the component options only,
> {{:1221}}), so the tool emits keys that {{camel_configuration_validate}} and
> {{camel run}} reject. A {{configKey}} on each component option in
> {{camel_catalog_doc}} covers the valid half.
> Related: CAMEL-25539 (EIP/language answer: placement and YAML shapes),
> CAMEL-25534, CAMEL-25533, CAMEL-25040 (the excerpt), CAMEL-25283
> ({{docPages}}), CAMEL-23475 (the scopes). Merged here: CAMEL-25540,
> CAMEL-25542.
> h3. Added 2026-10-14: full-key lookup and token numbers from the evaluation
> * {{camel_catalog_doc name=server}} reaches {{mainOptionsGroup}}
> ({{CatalogDocs.java:606}}) and answers the {{camel.server.*}} keys, but the
> forms an agent actually asks for do not: a full key ({{camel.server.port}}
> contains a dot after the group, so {{mainOptionsGroup}} returns null), a
> component plus a word ({{platform-http}} + {{optionsFilter: port}}), and
> {{camel.jbang.*}} ({{catalog.jbangModel()}} is not consulted by any tool).
> Accept a full key and answer the group with the matching option first, and
> route {{camel.jbang.*}} to the jbang model. In the evaluation the port
> question alone cost 30 knowledge searches, 12 {{camel_component_properties
> platform-http}} calls (15 K each, none naming it) and {{Run.class}}
> disassembly in 35 plain-arm trials.
> * Token numbers (Claude Code, 45 Haiku 5.5 trials in the MCP arm, 414 M
> tokens): about 9% of all tokens are re-reads of MCP results on later model
> calls, 4% of them from the 45 results over 20 000 characters (39 of those
> from {{camel_catalog_component_doc}}, 4 from {{camel_component_properties}});
> 125 of 564 results, 21% of all result bytes, were spilled to a file by the
> client and read back with grep. Repeated identical calls: 9%. Sonnet made 0-2
> MCP calls per trial, so for it the bound changes nothing; CAMEL-25646 does.
> _Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
