[
https://issues.apache.org/jira/browse/CAMEL-26760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18141665#comment-18141665
]
Claus Ibsen commented on CAMEL-26760:
-------------------------------------
Progress (night of 2026-11-09/10). All on main now:
* step 5b: camel cmd chaos-experiment, chaos-explore-start/finish (CAMEL-26786,
#29498), and the camel ask / MCP tools (#29499)
* step 6: the playbook as the MCP prompt camel_chaos_explore, and camel cmd
chaos-explore (CAMEL-26787, #29501)
* step 7: the report and a replayable scenario per failed experiment in
test/chaos (CAMEL-26788, #29514)
* from the recorded demo runs: an experiment fails on messages lost at the dead
letter channel (CAMEL-26789, #29512); an experiment that cannot tell does not
pass (faults never fired, a route without messages, failures outside its
expectations, a failed expectation behind a dead letter channel) (CAMEL-26790,
#29517)
* feeding: a feed that cannot reach its route says why, and a feed to a real
system is warned (CAMEL-26792, #29519); HTTP routes are fed with HTTP requests
and the traffic counts the status codes (CAMEL-26793, #29521); file routes get
files their filter takes (CAMEL-26795, #29523)
Open: step 8 (the Explore view on the chaos screen of the TUI, on a local
branch for review), the nightly eval, and the expectation for duplicates
(CAMEL-26791).
_Claude Code on behalf of davsclaus_
> camel-chaos - stage 3: let there be chaos, an AI agent explores chaos rules
> on a project by itself and reports back
> -------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26760
> URL: https://issues.apache.org/jira/browse/CAMEL-26760
> Project: Camel
> Issue Type: New Feature
> Components: camel-jbang
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> The user tells an AI agent "let there be chaos" on a Camel project. Within a
> time budget, the agent runs the project, tries chaos rules by itself, sends
> traffic, reads the report, errors and log, and comes back with a report: what
> it tried, what held, what broke, and scenarios to replay.
> Most of the pieces are on main from stage 1 and 2 (CAMEL-26330, CAMEL-26607):
> chaos suggest and the tried table (what to try), chaos check (safe to add),
> expectations (a verdict that does not depend on the model), scenarios with a
> seed (replay and CI gate), and the MCP tools. The agent that wrote the chaos
> example already did this by hand, and found CAMEL-26734. This ticket makes it
> a feature.
> h3. What is missing
> # *A playbook.* An MCP prompt {{camel_chaos_explore}} (budget, focus, traffic
> allowed) next to the prompts in {{PromptDefinitions}}, with the same text
> behind {{camel ask}} and a key on the TUI chaos screen. Loop: baseline
> without rules (errors already there are not blamed on chaos), suggest, pick a
> kind not tried, state the expectation, check, add with a short until=, send
> traffic, wait for the verdict, read, clear, next. Single faults first, then
> pairs ("two failures meet").
> # *Traffic.* Timer routes feed themselves, platform-http can be fed with
> curl, but Kafka, JMS, AMQP etc. cannot. Stage 1: run with {{--stub=remote}}
> by default and feed the stubbed consumers with {{camel cmd send}}, which
> needs {{--repeat}} / {{--interval}} (today it sends one message), also on
> {{camel_runtime_send}}. The suggestions say per route how to feed it. Stubbed
> messages are not real Kafka, but close enough to test the error handling.
> # *Safety.* The faults are local, the traffic is not: by default remote
> endpoints are stubbed, and the agent asks before it sends to anything real.
> One consent covers the whole budget, instead of a confirm per rule.
> camel-chaos enforces the deadline: rules are capped to the end of the
> exploration, and cleared when it ends. The agent never edits the routes:
> hints only.
> # *One experiment per call.* {{chaos_experiment}} (rule(s), traffic, max
> wait) returns a compact result when the expectations are decided: verdict,
> per route delta, new errors, faults fired. About 10 large calls instead of
> 100 small ones, which matters for small local models.
> # *A journal and a report.* camel-chaos keeps a journal of the exploration:
> baseline, each experiment with its hypothesis, rules, seed and outcome. The
> report is text for the CLI, JSON for tools and markdown in
> {{chaos/report-<time>.md}}. Each finding is classified (route gap: no
> breaker, no DLC, not idempotent, hand-off loses; or suspected Camel bug),
> with evidence and a hint, and experiments worth keeping are saved as
> {{chaos/explore-*.chaos}} scenarios.
> # *TUI.* An Explore mode on the chaos screen: budget dialog, the experiments
> live, and the report at the end.
> h3. Stage 2 (later)
> * Realistic traffic with Citrus ({{camel test run}}): real brokers through
> test-infra, message shapes from a schema or samples.
> * A non-AI explore ({{--no-ai}}): the top suggestions in order, each with its
> own expectation and traffic, for CI and as a floor for small models.
> * Several integrations at once (a hand-off over a real broker).
> h3. Open questions
> * Do chaos rules on {{endpoint=kafka:...}} match the stubbed endpoint, and
> does {{camel cmd send}} to a stubbed consumer feed it? To verify first.
> * CLI shape: {{camel cmd chaos-explore --budget=10m}} (uses the AI of camel
> ask) or only {{camel ask "let there be chaos"}}.
> _Claude Code on behalf of Claus Ibsen_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
