[
https://issues.apache.org/jira/browse/CAMEL-25676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen reassigned CAMEL-25676:
-----------------------------------
Assignee: Claus Ibsen
> camel ask: confirm risky tool calls, and fit the tools to the model
> -------------------------------------------------------------------
>
> Key: CAMEL-25676
> URL: https://issues.apache.org/jira/browse/CAMEL-25676
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> {{camel ask}} runs the shared AI tools of camel-jbang-core ({{ToolRegistry}},
> the tools the Camel MCP server serves) against a running integration, with a
> local (Ollama) or a hosted model. It has two gaps: it asks before nothing,
> and it sends every tool to every model.
> h3. 1. No confirmations
> The developer tools section of the security model (security-model.adoc) sets
> the guardrails for Camel's AI assistants: each file write goes through a
> confirmation with the diff, the developer confirms a call that reaches beyond
> the integration, may destroy data or runs code, and an HTTP request goes only
> to the integration's own server. {{camel ask}} has none of the confirmations:
> {{ToolContext}} has no confirmation hook, and {{AskTools}} runs each call as
> it comes.
> * {{camel_write_file}}, {{camel_edit_file}} and its own {{write_file}} write
> files without asking. {{write_file}} writes anywhere under the current
> directory, without the YAML and properties validation of {{camel_write_file}}.
> * {{send_message}} sends to any endpoint, {{execute_sql}} runs any statement,
> {{eval_expression}} and {{camel_eval_expression}} run expressions in
> languages that run code, such as Groovy, and {{stop_application}} stops the
> integration.
> * {{cli_exec}} runs any camel command; its description only asks the model to
> confirm with the user first.
> * {{http_request}} already reaches only the integration's own server.
> Proposal, the same for local and hosted models:
> * Ask before the same calls, by the same rules: a message to an endpoint the
> integration does not have (its routes, its own endpoints and the components
> the catalog marks internal, such as direct:, seda: and log:, stay inside), a
> message body read from a file outside the project folder, SQL other than a
> single SELECT, SHOW or DESCRIBE, a row update, and an expression in a
> language that runs code. Move these rules ({{ToolConfirms}}) to
> camel-jbang-core next to the shared tools, so every client that runs the
> tools applies the same ones.
> * Show the diff of each write and ask. Drop {{write_file}}:
> {{camel_write_file}} and {{camel_edit_file}} cover it, with validation.
> * Ask before {{stop_application}}, the {{camel_control}} actions that stop or
> restart routes or the integration, and a {{cli_exec}} command that changes
> something (such as run, stop, cmd and infra run). Read-only commands (get,
> catalog, version, help) run without asking.
> * Without a terminal to ask on (one-shot use in a script, stdin not a TTY),
> refuse such a call with a message the model can pass on, unless an option
> allows it explicitly, for example {{--yes}}.
> * Add {{camel ask}} to the developer tools section of the security model.
> h3. 2. Every tool to every model
> {{AskTools}} sends all 68 shared tools plus its own 6 ({{cli_list_commands}},
> {{cli_command_help}}, {{cli_exec}}, {{list_files}}, {{read_file}},
> {{write_file}}) with every request: 74 tool definitions. Several do the same
> job, for example {{get_errors}} and {{camel_get_errors}}, {{eval_expression}}
> and {{camel_eval_expression}}, {{catalog_component_doc}} and
> {{camel_catalog_doc}}, {{read_file}} and {{camel_get_files}}. The system
> prompt is generic: nothing on the YAML DSL shapes or Simple, and it points
> the model to a stop_route tool that does not exist.
> Proposal:
> * Local providers ({{LlmClient.isLocalEndpoint}}) get the core tools
> ({{ToolDescriptor.isCore}}) and the tool groups of the selected integration
> ({{get_tool_groups}} works them out). Hosted providers get the full set,
> without the duplicates. An option chooses (auto, core, full), auto by default.
> * Use the authoring guidelines the shared tools are written for (YAML step
> shapes, Simple operators, the edit loop of camel_get_files, camel_edit_file
> and camel_write_file), shared with the other clients instead of a prompt of
> its own, and drop the stop_route line.
> * Allow 25 tool-calling rounds per question instead of 10, and check the
> Simple expressions in the answer ({{AnswerChecks}}), as the edit tools check
> the ones they write.
> h3. Measured
> Oct 15 2026: qwen3.6:35b-mlx on Ollama 0.40.3 on a Mac mini, the
> fail-well/error-handling example running, four questions about it, each in a
> fresh conversation and asked once. Compared with the same model given the
> core tool set (25 tools) and the authoring guidelines:
> || ||core set (25 tools)||camel ask (74 tools)||
> |first request of a question|5.2k tokens|9.5k tokens|
> |time for the four questions|68 s|89 s|
> |model requests|17|11|
> |tokens for the four questions|122k|145k|
> |tool calls|16|12|
> Both got the declined order, the one recorded error and the retry settings
> right. Both named the slowest route but not the reason (two redeliveries of
> one order, one second apart). camel ask also said that one order failed and
> was parked, while it was charged on its third attempt. A hosted model was not
> measured: the full tool set suits it better, and the confirmations apply to
> both.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
