Claus Ibsen created CAMEL-26330:
-----------------------------------
Summary: camel-chaos - turn on faults in a running Camel
application, live, and report how the routes cope
Key: CAMEL-26330
URL: https://issues.apache.org/jira/browse/CAMEL-26330
Project: Camel
Issue Type: New Feature
Components: camel-jbang, camel-main
Reporter: Claus Ibsen
Assignee: shashank
Fix For: 4.24.0
A chaos feature for Camel developers and AI tools: run a Camel application in
dev mode, turn on faults while it runs, and see how the routes cope, in real
time. A report then says, per route, how many messages completed, failed, went
to the error handler, were redelivered or were lost, with PASS or FAIL.
It starts from Shashank's proposal and prototype:
[PROPOSAL.md|https://github.com/allthingssecurity/camel/blob/camel-chaos-prototype/components/camel-chaos/PROPOSAL.md]
on the camel-chaos-prototype branch. The prototype reproduces two real bugs on
the code from before their fixes (CAMEL-25684, CAMEL-25682).
*Switched on by the classpath*
* With camel-chaos on the classpath, its hooks are installed and stay idle
until a rule is added. Rules are added, changed and removed live, as in dev
mode ({{camel run --dev}}).
* Without camel-chaos on the classpath, as in production, nothing of it is
there.
* With profile {{prod}} it stays off, as an extra guard.
* Options {{camel.chaos.*}} through camel-main, as for {{camel.debug.*}} and
{{camel.trace.*}}: for example the seed, and rules to install at startup.
*Hooks at all levels*
The prototype wraps only {{to}}. Chaos needs hooks at every level:
* Every send to any endpoint: {{to}}, {{toD}}, wireTap, enrich, recipientList,
multicast and a ProducerTemplate. Chaos must be able to stop a send, delay it
or change its message. The best place is probably the component, as {{camel run
--stub}} does it: the component resolver of camel run can wrap a component
instead of replacing it with a stub. Then every producer and consumer of the
component goes through chaos, and the processors of the routes stay as they
are. The event notifier ({{ExchangeSendingEvent}}) fires for every send but
cannot stop it, because the exception of a notifier is logged and ignored. The
interceptSendToEndpoint wrapper can stop a send, but it adds processors to the
routes.
* Every processing step: for example small random delays at each step.
* The message: header values with random data, the body in an unusual charset
(characters outside US-ASCII), an input stream that ends too soon.
* Consumer polls: the poll strategy of a {{ScheduledPollConsumer}} can be
changed at runtime, and its begin, commit and rollback run around every poll.
* The route lifecycle: stall a route (suspend, then resume), and remove a route
and add it again (the shape of CAMEL-26286 to CAMEL-26305).
A failure must be able to have a real type, for example an HTTP status code
with its headers, so that the {{onException}} of the route is tested. Rules end
by themselves, after N calls or a duration. A seed makes a run repeatable.
*The report*
Per route: completed, failed, handled by the error handler, redelivered, lost,
and a verdict. Expectations ("this route completes N more messages") find
messages that were lost before they became an exchange. The verdict must not
call a message lost when its consumer redelivers it (for example the file
consumer, or transacted JMS), because an AI tool acts on what the report says.
*Control*
* A {{chaos}} dev console, which the CLI connector uses: {{camel cmd chaos}},
and any other client.
* AI tools: dedicated tools in the camel-jbang-mcp server (add a fault, report,
clear), in a runtime tool group that is offered only when the application has
camel-chaos.
*Time*
The clock and the sleeps go through {{TimeSource}} (camel-util). It is the
system clock, so a person sees the faults in real time. camel-test-dst can run
the same rules in virtual time, but chaos does not need DST. The Javadoc of
{{TimeSource}} says it is internal, so the design document should say that
camel-chaos may use it.
*How to start*
A pull request with a short design document in {{design/}}, like
{{design/payload-mode.adoc}} (CAMEL-25334,
[#28457|https://github.com/apache/camel/pull/28457]). The code follows in its
own pull requests. The video, the models (TLA+, Lean, Z3) and the demo scripts
of the prototype are not needed in the Camel repository.
_Filed by Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
