Claus Ibsen created CAMEL-25676:
-----------------------------------
Summary: camel ask: confirm risky tool calls, and fit the tools to
the model
Key: CAMEL-25676
URL: https://issues.apache.org/jira/browse/CAMEL-25676
Project: Camel
Issue Type: Improvement
Components: camel-jbang
Reporter: Claus Ibsen
Fix For: 4.24.0
{{camel ask}} runs the shared AI tools of camel-jbang-core ({{ToolRegistry}},
the tools the Camel MCP server serves) against a running integration, with a
local (Ollama) or a hosted model. It has two gaps: it asks before nothing, and
it sends every tool to every model.
h3. 1. No confirmations
The developer tools section of the security model (security-model.adoc) sets
the guardrails for Camel's AI assistants: each file write goes through a
confirmation with the diff, the developer confirms a call that reaches beyond
the integration, may destroy data or runs code, and an HTTP request goes only
to the integration's own server. {{camel ask}} has none of the confirmations:
{{ToolContext}} has no confirmation hook, and {{AskTools}} runs each call as it
comes.
* {{camel_write_file}}, {{camel_edit_file}} and its own {{write_file}} write
files without asking. {{write_file}} writes anywhere under the current
directory, without the YAML and properties validation of {{camel_write_file}}.
* {{send_message}} sends to any endpoint, {{execute_sql}} runs any statement,
{{eval_expression}} and {{camel_eval_expression}} run expressions in languages
that run code, such as Groovy, and {{stop_application}} stops the integration.
* {{cli_exec}} runs any camel command; its description only asks the model to
confirm with the user first.
* {{http_request}} already reaches only the integration's own server.
Proposal, the same for local and hosted models:
* Ask before the same calls, by the same rules: a message to an endpoint the
integration does not have (its routes, its own endpoints and the components the
catalog marks internal, such as direct:, seda: and log:, stay inside), a
message body read from a file outside the project folder, SQL other than a
single SELECT, SHOW or DESCRIBE, a row update, and an expression in a language
that runs code. Move these rules ({{ToolConfirms}}) to camel-jbang-core next to
the shared tools, so every client that runs the tools applies the same ones.
* Show the diff of each write and ask. Drop {{write_file}}:
{{camel_write_file}} and {{camel_edit_file}} cover it, with validation.
* Ask before {{stop_application}}, the {{camel_control}} actions that stop or
restart routes or the integration, and a {{cli_exec}} command that changes
something (such as run, stop, cmd and infra run). Read-only commands (get,
catalog, version, help) run without asking.
* Without a terminal to ask on (one-shot use in a script, stdin not a TTY),
refuse such a call with a message the model can pass on, unless an option
allows it explicitly, for example {{--yes}}.
* Add {{camel ask}} to the developer tools section of the security model.
h3. 2. Every tool to every model
{{AskTools}} sends all 68 shared tools plus its own 6 ({{cli_list_commands}},
{{cli_command_help}}, {{cli_exec}}, {{list_files}}, {{read_file}},
{{write_file}}) with every request: 74 tool definitions. Several do the same
job, for example {{get_errors}} and {{camel_get_errors}}, {{eval_expression}}
and {{camel_eval_expression}}, {{catalog_component_doc}} and
{{camel_catalog_doc}}, {{read_file}} and {{camel_get_files}}. The system prompt
is generic: nothing on the YAML DSL shapes or Simple, and it points the model
to a stop_route tool that does not exist.
Proposal:
* Local providers ({{LlmClient.isLocalEndpoint}}) get the core tools
({{ToolDescriptor.isCore}}) and the tool groups of the selected integration
({{get_tool_groups}} works them out). Hosted providers get the full set,
without the duplicates. An option chooses (auto, core, full), auto by default.
* Use the authoring guidelines the shared tools are written for (YAML step
shapes, Simple operators, the edit loop of camel_get_files, camel_edit_file and
camel_write_file), shared with the other clients instead of a prompt of its
own, and drop the stop_route line.
* Allow 25 tool-calling rounds per question instead of 10, and check the Simple
expressions in the answer ({{AnswerChecks}}), as the edit tools check the ones
they write.
h3. Measured
Oct 15 2026: qwen3.6:35b-mlx on Ollama 0.40.3 on a Mac mini, the
fail-well/error-handling example running, four questions about it, each in a
fresh conversation and asked once. Compared with the same model given the core
tool set (25 tools) and the authoring guidelines:
|| ||core set (25 tools)||camel ask (74 tools)||
|first request of a question|5.2k tokens|9.5k tokens|
|time for the four questions|68 s|89 s|
|model requests|17|11|
|tokens for the four questions|122k|145k|
|tool calls|16|12|
Both got the declined order, the one recorded error and the retry settings
right. Both named the slowest route but not the reason (two redeliveries of one
order, one second apart). camel ask also said that one order failed and was
parked, while it was charged on its third attempt. A hosted model was not
measured: the full tool set suits it better, and the confirmations apply to
both.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
