allthingssecurity opened a new pull request, #29047:
URL: https://github.com/apache/camel/pull/29047
# Description
CAMEL-26454 (related to CAMEL-26330 and CAMEL-26436): the MCP tool group of
`camel-chaos`, the step "The MCP tool
group" of "Order of work" in `design/chaos.adoc`. It builds on #28788, now
merged (the `chaos` dev console, the `chaos`
action of the CLI connector, `camel cmd chaos-*` and `camel get chaos`).
An AI agent working on an integration through the Camel MCP server (or
`camel ask`) can now try a route against
failures itself: add a fault, run the route, read what the fault did, take
it away. Two parts of the design are
narrowed: the AI panel of the TUI gets only a guidance line, no chaos tools
(see below), and step 6 of the design
(route lifecycle faults) is still to come. No upgrade-guide entry:
everything is new in the unreleased 4.24.
| Tool | `readOnlyHint` | `destructiveHint` | In `ToolConfirms` |
|---|---|---|---|
| `camel_runtime_chaos_list` | `true` | `false` | no |
| `camel_runtime_chaos_report` | `true` | `false` | no |
| `camel_runtime_chaos_add` | `false` | `true` | yes |
| `camel_runtime_chaos_change` | `false` | `true` | yes |
| `camel_runtime_chaos_remove` | `false` | `false` | no |
| `camel_runtime_chaos_clear` | `false` | `false` | no |
- **camel-jbang-mcp** (`RuntimeTools`, `@McpSecured`): the six tools, thin
wrappers like the other runtime tools: they
resolve `nameOrPid` with `RuntimeService` and delegate to the shared
`ToolRegistry`, which sends the `chaos` action of
the CLI connector (add, change, remove, clear) or reads the chaos console
from the status (list, report), over the
same files as the other runtime tools; nothing new is opened in the
integration.
- **Shared `ToolRegistry`** (also used by `camel ask`): `chaos_list`,
`chaos_report`, `chaos_add`, `chaos_change`,
`chaos_remove`, `chaos_clear`, with the same marks. The list and the
report read the status again up to five times,
50 ms apart, when it is being rewritten: the connector rewrites the file
every second (not atomically), and a read in
that moment found no status. A follow-up to write it atomically is drafted
below.
- **`ToolConfirms.chaos(command, integration, id, rule)`**: adding or
changing a rule is a call the user confirms first,
with the rule shown; null for the other four. `camel ask` asks before
`chaos_add` and `chaos_change`
(`AskTools.confirmation`).
- **A `chaos` runtime tool group** (`ToolGroup.CHAOS`, `AppFeatures.chaos()`
/ `chaosOn()`, `ToolGroups`): it loads
when the status has the chaos console (camel-chaos in the application),
with chaos on or off, and its guidance line
says which. It comes last, so the other groups keep their place. `camel
ask` maps it to the shared tools. The AI
panel of the TUI gets the group without tools: only a line that failures
and delays may be injected on purpose, so
its model does not hunt for a bug behind them.
- Docs: the MCP server page (a "Chaos" section with the tools and their
hints, the `chaos` row of the tool groups, an
example prompt), the TUI local-models page (the group), the chaos page
("From an AI agent (MCP)", with the catalog
copy), the security model (`camel ask` asks before adding or changing a
chaos rule; the opt-in row names the tools),
and `design/chaos.adoc` (the step is done; remove/clear and the TUI as
built).
## Remove and clear: not destructive
As in the design table: removing a rule only brings the integration back
towards normal and the report keeps every
removed rule, as `camel_runtime_trace` (tracing on and off) is not
destructive either; so `read-write` offers them and
an agent can always take its faults away.
## Security
- The tools change rules only through the `chaos` action of the CLI
connector (#28788), and the list and the report
only read the status. The action works only when chaos is switched on
(profile `dev` or `camel.chaos.enabled=true`,
never profile `prod`) and never creates a `ChaosManager`, so the MCP
server cannot switch chaos on.
- Gating (all opt-in, as for the other runtime tools): with
`camel.mcp.security.enabled=true`, `read-only` offers
list and report, `read-write` adds remove and clear, `admin` offers all
six, and with audit on `McpAccessFilter`
logs a tool it does not offer. `RuntimeTools` is `@McpSecured`, so with
the security layer on the interceptor audits
each call (with `include-arguments`, the default, the rule text is in the
audit log, as a SQL statement is) and, with
`camel.mcp.security.redaction.enabled=true`, passes the answer through
`McpSecretRedactor`. A refusal message is not
redacted, as for every tool.
- `openWorldHint=false` on all six: they only reach the selected integration.
- Not changed here: with the development opt-in `camel.jbang.mcp=true`,
`JbangDevMcpServer` (camel-mcp-server) puts every
shared `ToolRegistry` tool on the local management HTTP server, now
`chaos_add` and `chaos_change` too, and runs them
without asking, as it already runs `stop_route`, `execute_sql` and
`camel_eval_expression`. The chaos page, the
design and the security model now say so. Follow-up drafted below.
## Tests
- `RuntimeToolsChaosTest` (camel-jbang-mcp, 7 tests, `@Isolated`): the tools
against a running integration: a
`CamelContext` with camel-chaos and the real `LocalCliConnector` in the
test JVM, reached over the connector's files
in a temporary home, as the MCP server reaches any integration (only the
process lookup is faked: the real one never
takes the JVM of the tools for an integration). Every tool (ids, versions,
the fault in place, the list after each
change, the report with `r1 v1 replaced`, `r2 v1 removed`, `r1 v2
removed`); a rule that is not valid is refused with
exactly the message of the parser of `camel.chaos.rules`, and nothing
changes; an unknown id; chaos off (no profile)
and profile `prod`: every change refused with the integration's reason,
the list says why; a missing argument is said
before any lookup; an integration that is not found or not the only one is
refused by the real lookup (two status
files of live processes).
- `RuntimeToolsChaosSecurityTest` (camel-jbang-mcp, 3 tests): the hints of
each tool, the same marks in the shared
registry, exactly the destructive tools in `ToolConfirms` (with the rule
as the target), what each access level
offers (`AccessLevel.permits`, as `RuntimeToolsTest` does for the HTTP
tools), and `@McpSecured`.
`RuntimeToolsTest.theNewWrappersDelegateToRegistryTools` names the six
shared tools.
- camel-jbang-core: `AppFeaturesTest` (+1), `ToolGroupsTest` (+2),
`ToolConfirmsTest` (+1), `ToolRegistryTest` (+4:
marks, arguments, the chaos section of the status, the re-read while the
status is written), `AskToolsTest` (+3:
`camel ask` asks before add and change, not before the others; the group
names the shared tools; the longest names
first). camel-jbang-plugin-tui: `AiPanelToolGroupsTest` (+1); the guidance
points to the `chaos` section of
`tui_get_status`.
Runs (offline, Maven on an aggregator of the needed modules; logs kept):
- camel-chaos 262/0, camel-cli-connector `LocalCliConnector*Test` 39/0,
camel-jbang-core (the AI, ask, registry and
chaos command tests) 228/0, camel-jbang-mcp whole module 535 with 1 error:
`TransformToolsTest.transformXmlToJava`,
`NoClassDefFoundError: WasmExpression` from an old snapshot jar of the
offline build (the wasm language was removed in
CAMEL-25677), not related; CI runs it.
- 20 runs each in its own JVM, all passed: `RuntimeToolsChaosTest` 20/20
(before the status re-read it failed 6 of 20),
`RuntimeToolsChaosSecurityTest`, `ChaosActivationTest`,
`ToolRegistryTest`, `ToolConfirmsTest`, `ToolGroupsTest`,
`AppFeaturesTest`, `AskToolsTest`.
- Negative controls: with the main code of #28788 the new MCP tests do not
compile (29 errors: the tools do not exist);
26 mutants of the new code (hints, the wrappers, the registry tools, the
status re-read, `ToolConfirms`, `camel ask`,
the group, the placeholder check) each make a test fail.
- camel-jbang-plugin-tui cannot be built offline here (its tamboui jars are
missing), so `TuiToolGroups` and the new
`AiPanelToolGroupsTest` case ran as a javac copy against camel-jbang-core;
CI is their first Maven run.
Also fixed on the way, as this PR rewrites the map:
`AskTools.sharedToolNames` replaced the names in the iteration order
of the map, which changes from one JVM to the next, and `camel_runtime_sql`
is the start of
`camel_runtime_sql_trace`, so the `camel ask` prompt said
`execute_sql_trace` in some runs. It now replaces the longest
names first
(`AskToolsTest.aGroupToolThatStartsWithTheNameOfAnotherIsNamedRight`). Main
only: the map came with
CAMEL-25676, which is not on 4.22.x or 4.23.x.
The chaos plugin fix for tooling placeholders (`RouteDiagramToolsTest` and
`camel export` with camel-chaos on the classpath) is now part of #28788, where
CI found it.
## Follow-ups (drafted, not filed; I can file them if you agree)
- camel-mcp-server: the development MCP tools of `camel run --mcp` should
carry the hints of the shared tools, or leave
out the calls of `ToolConfirms`.
- camel-cli-connector: write the status file atomically (temporary file and
`ATOMIC_MOVE`, as `ActionClient` writes
action files); then the re-read in `ToolRegistry.chaosSection` can go.
## Open questions
- The TUI's AI panel has no chaos tools here (its group only adds the line).
Do you want `tui_chaos_*` tools in the
panel too, with the confirmation dialog for add and change, in a follow-up?
- The chaos group loads its six tool schemas also when chaos is off (its
line says it is off). I kept one tool list per
group, so the fingerprint does not depend on the switch. OK?
_Claude Code on behalf of allthingssecurity_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
