This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 5a452ec7987a CAMEL-26330: design - camel-chaos, faults turned on in a
running Camel application (#28539)
5a452ec7987a is described below
commit 5a452ec7987a7af500063f3734d183fdf429e0c9
Author: allthingssecurity <[email protected]>
AuthorDate: Tue Oct 27 17:35:35 2026 +0530
CAMEL-26330: design - camel-chaos, faults turned on in a running Camel
application (#28539)
Design document design/chaos.adoc for camel-chaos and its entry in the
AGENTS.md design index.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
AGENTS.md | 1 +
design/chaos.adoc | 468 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 469 insertions(+)
diff --git a/AGENTS.md b/AGENTS.md
index e78bcca7c4d2..5a85c2332565 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -315,6 +315,7 @@ jump straight to implementation after reading the issue
description and the curr
- **MDC / Logging** (MDC propagation, logging context):
[`design/mdc.adoc`](design/mdc.adoc)
- **Headers** (naming conventions, constants, upgrade policy):
[`design/headers.adoc`](design/headers.adoc)
- **Cross-cutting features** (onException, interceptors, onCompletion,
route configurations and how they bind to routes):
[`design/cross-cutting-binding.adoc`](design/cross-cutting-binding.adoc)
+ - **Chaos** (proposed; `camel-chaos`: faults turned on in a running
application, the send and step hooks in camel-core, `camel.chaos.*` options,
the report): [`design/chaos.adoc`](design/chaos.adoc)
- **Java DSL parsing** (reading Java routes into the model without
compiling them, `LwJavaParser`, its safety rules):
[`design/java-dsl-parser.adoc`](design/java-dsl-parser.adoc)
5. **Understand the broader context**: If the issue involves a module that
replaced or deprecated
another (e.g., `camel-opentelemetry2` replacing `camel-opentelemetry`),
understand *why* the
diff --git a/design/chaos.adoc b/design/chaos.adoc
new file mode 100644
index 000000000000..51c3b2a894f7
--- /dev/null
+++ b/design/chaos.adoc
@@ -0,0 +1,468 @@
+---
+title: Chaos
+authors:
+ - "@allthingssecurity"
+reviewers: ["@davsclaus"]
+approvers: []
+creation-date: 2026-10-26
+last-updated: 2026-10-27
+status: proposed
+see-also:
+ - "CAMEL-26330"
+ - "CAMEL-26331"
+ - "CAMEL-26338"
+ - "CAMEL-25911"
+ - "CAMEL-25684"
+ - "CAMEL-25682"
+ - "CAMEL-26286"
+replaces: []
+superseded-by: []
+---
+
+== Summary
+
+`camel-chaos` lets a developer, or an AI tool, turn on faults in a running
Camel application and see how its routes
+cope. A rule such as "fail 30% of the sends to `kafka:orders` for one minute"
or "stall route `poller` for 20 s" is
+added while the application runs, for example in dev mode (`camel run --dev`).
A report then says per route how many
+exchanges completed, failed, were handled by the error handler or were
redelivered, whether the consumer keeps a
+failed message, and, for a rule with an expectation, how many messages were
lost, with a verdict.
+
+With the `camel-chaos` JAR and profile `dev` (the default of `camel run`), or
`camel.chaos.enabled=true`, its hooks
+are installed and stay idle until a rule is added. Without the JAR, as in
production, nothing of it is there. Faults
+run in real time, and the same rules can run inside a `camel-test-dst` world
in virtual time.
+
+Chaos needs two small changes in camel-core: a hook for every send to an
endpoint (CAMEL-26331) and a hook for every
+step, inside the error handler of the step (CAMEL-26338). Both are general
extension points, not chaos-specific: their
+API names no chaos type, so other features and tests can use them. The other
levels use SPIs that exist today.
+
+== Motivation
+
+Routes are written for the happy path, and their error handling is rarely
exercised before production. Questions such
+as "what happens to the messages in flight when the broker is slow for 30 s",
"does this `onException` really match the
+error the HTTP endpoint returns" or "does the consumer resume after its route
was suspended" are answered today by
+hand-written tests, one route and one fault at a time.
+
+A prototype (CAMEL-26330 links it), which hooks only `to`, reproduced
CAMEL-25684 (camel-slack, a route suspend in the
+middle of a poll) and CAMEL-25682 (camel-resilience4j, a timeout while the
task writes its result) on the code from
+before their fixes. The recent "remove a route" bugs (CAMEL-26286,
CAMEL-26287, CAMEL-26288, CAMEL-26302 to
+CAMEL-26305: one route goes, and another route loses the cache, channel or
session it shares) have a shape a chaos
+rule can produce on demand.
+
+AI tools that write and change routes need the same: a way to try a route
against failures and read a short result.
+
+== Goals
+
+- Out of the box chaos for any Camel application in development: faults on
sends, on the messages that enter a route,
+ on consumer polls, on processing steps and on the route lifecycle.
+- Switched on by the classpath with profile `dev` or
`camel.chaos.enabled=true`, idle until a rule is added; rules
+ added, changed and removed live.
+- Configuration through `camel.chaos.*` options of camel-main, as for
`camel.debug.*` and `camel.trace.*`.
+- A report per route with counts and a verdict, readable by a person and by a
tool.
+- Control from the CLI (`camel cmd chaos-add` and the other `chaos-*`
commands, `camel get chaos`) and the
+ camel-jbang MCP server; a dev console shows the rules and the report.
+- Real time by default; the same rules in virtual time inside a DST world.
+- A seed fixes the random draws of the rules. The same report again needs a
DST world (see Time).
+
+== Non-goals
+
+- Production use. Chaos fails closed, and stays off with profile `prod` even
when enabled.
+- Custom fault types written by users. The first version has a fixed set; a
plugin SPI (a JAR with its own faults) can
+ come later.
+- Faults outside Camel: network partitions, killing processes, disk or clock
faults of the host. Tools outside Camel
+ do that; this is about what Camel does with the failures it sees.
+- The video, the models (TLA+, Lean, Z3) and the demo scripts of the
prototype. They stay on the prototype branch.
+
+== Proposal
+
+=== Switched on by the classpath
+
+`camel-chaos` registers a `ContextServicePlugin`
(`META-INF/services/org.apache.camel.spi.ContextServicePlugin`).
+The `CamelContext` loads the plugins found on the classpath while it is built,
before the routes are created.
+
+In camel-main the profile and the `camel.chaos.*` options are set after the
plugins are loaded, so the plugin reads
+them when the context initializes (`LifecycleStrategy.onContextInitializing`),
which is still before the routes are
+created. The profile comes from `ExtendedCamelContext.getProfile()`.
+
+The plugin installs the hooks below only with profile `dev`, the default of
`camel run`, or with
+`camel.chaos.enabled=true` (a plain `Main`, tests, DST worlds), and never with
`prod`. Otherwise it fails closed. That
+includes no profile, the usual case for a plain `Main`, Spring Boot and
Quarkus: the profile is then null, not `prod`
+(`ProfileConfigurer.configureMain` returns early), and a `camel.chaos.rules`
left in `application.properties` does
+nothing, unless `camel.chaos.enabled=true` is left there too. When it installs
the hooks, it logs a WARN that chaos is
+active. Each hook asks the `ChaosManager` whether a rule matches; with no
rules that is one volatile read, and the call
+goes on unchanged.
+
+=== Getting it
+
+- `camel run --chaos` adds the `camel:chaos` dependency, as `--observe` adds
`camel:observability-services` in
+ `Run.java`; `--dep camel:chaos` works too.
+- A plain `Main` or a test adds the `camel-chaos` dependency and sets
`camel.chaos.enabled=true`.
+- Spring Boot and Quarkus: the plugin works with any `CamelContext` and
installs the hooks with Camel profile `dev`.
+ The options are camel-main only, so rules come from the CLI connector; a
starter or an extension can bind
+ `camel.chaos.*` later.
+
+=== Options
+
+[cols="1,1,3"]
+|===
+|Option |Default |Description
+
+|`camel.chaos.enabled` |`false`, `true` with profile `dev` |Install the hooks:
`true` for a plain `Main`, tests and
+DST worlds, ignored with profile `prod`. They do nothing until a rule is added.
+|`camel.chaos.seed` | |Seed of the random draws of the rules; without it a
seed is picked and shown in the report.
+|`camel.chaos.rules` | |Rules to add at startup, in the text form of Rule
syntax, separated by `;` (marked
+`insecure:dev`, see Risks).
+|`camel.chaos.maxDuration` |`10m` |Upper bound for every rule: a rule without
`until` ends after it, and a
+`duration` of `stall` or `lifecycle` may not be longer (such a rule is
rejected when it is added).
+|`camel.chaos.reportOnShutdown` |`true` |Log the report when the context stops.
+|===
+
+They are a `ChaosConfigurationProperties` group in camel-main, as
`DebuggerConfigurationProperties` is for
+`camel.debug.*`, so the catalog and tooling know them.
+
+=== Rules
+
+A rule has a target, a fault, how often it fires, and when it ends:
+
+[source]
+----
+target: route=<id or pattern> | endpoint=<uri pattern> | component=<name> |
step=<processor id pattern>
+at: send | poll | entry | step (optional, see Target and hooks)
+fault: error | delay | drop | corrupt | lifecycle (stall) (parameters: see
Rule syntax)
+rate: a probability (0.3), or every Nth match
+until: a number of matches, a duration, or removed by hand
+----
+
+Rules are added, changed, listed and removed live; a rule gets an id when it
is added, which change and remove take.
+A rule ends by itself after its count or duration (at most
`camel.chaos.maxDuration`), so a forgotten rule cannot keep
+breaking a dev instance. Rules stay when the routes are reloaded in dev mode.
+
+==== Rule syntax
+
+A rule is one line: the target, the fault and its parameters as `name=value`,
then `at`, `rate` and `until`.
+`rate=0.3` is a probability and `rate=every:5` fires on every 5th match;
without `rate` every match fires. `until=100`
+is a number of matches and `until=2m` a duration; without `until` the rule
ends after `camel.chaos.maxDuration`. A
+value with spaces is quoted. Where a rule fires is in Target and hooks below.
+
+A rule can also carry an expectation, `expect=<route id>:<count>` with an
optional `within=<time>` (default: the rule's
+`until` duration, else `camel.chaos.maxDuration`): the route must complete at
least that many more exchanges after the
+rule is added and before that time is up. For example
+`endpoint=kafka:orders* error rate=0.3 until=1m expect=orders:100 within=2m`.
See The report for how it is checked.
+
+[cols="1,2,3"]
+|===
+|Fault |Parameters |Effect
+
+|`error`
+|`exception=<class>`, `message=<text>`; for the HTTP components
`status=<code>`, `header.<name>=<value>` and
+`body=<text>`
+|Throws that exception. With `status` it is the exception the HTTP component
throws for that status; when the endpoint
+has `throwExceptionOnFailure=false`, chaos returns the response instead, as
the real producer does: the status in the
+`CamelHttpResponseCode` header, the `header.*` values and the `body` (empty
without it). All parameters are optional:
+without `exception` (and without `status`) the fault throws a
`ChaosException`, a `RuntimeCamelException` of
+camel-chaos, with the `message` or a default one, on any endpoint. A rule
whose exception class is not found, is not
+a `Throwable`, or has neither a constructor taking a `String` message nor a
no-argument one is rejected when it is
+added, with an error that names the class.
+
+|`delay`
+|`duration=<time>`, such as `500ms` or `2s`
+|Waits that long before the send, the step or the poll.
+
+|`drop`
+|none
+|A send that reports success without calling the producer, or a skipped poll.
+
+|`corrupt`
+|`mode=headers\|charset\|truncate`
+|Changes the message: header values with random data, a body in an unusual
charset (characters outside US-ASCII), or
+an `InputStream` body that ends too soon. `charset` applies to a body that is
text or converts to text, and `truncate`
+to an `InputStream` or stream cache body; on another body the message is left
as it is and the report counts the rule
+as skipped for that message, so a fault that did nothing is visible.
+
+|`lifecycle`
+|`action=suspend\|stop\|remove`, `duration=<time>`
+|Suspends, stops or removes the target route, then resumes it, starts it or
adds its definition again after the
+duration. `stall duration=<time>` is short for `lifecycle action=suspend
duration=<time>`. Both take only a `route=`
+target; a rule with another target is rejected when it is added.
+|===
+
+`stall` and `lifecycle` act once on the route when the rule is added; `rate`
and `until` do not apply to them, and
+their `duration` is bounded by `camel.chaos.maxDuration`.
+
+==== Target and hooks
+
+Each target has these hooks, and each hook offers some of the faults:
+
+[cols="2,3,3"]
+|===
+|Target |Hooks (`at=`) |Faults
+
+|`endpoint=`, `component=`
+|The sends to a matching endpoint (`send`) and the polls of its consumer
(`poll`).
+|`send`: `error`, `delay`, `drop`, `corrupt`. `poll`: `error`, `delay`, `drop`.
+
+|`route=`
+|The messages that enter the route (`entry`), and the route lifecycle.
+|`entry`: `corrupt`. Lifecycle: `stall`, `lifecycle`.
+
+|`step=`
+|The matching steps (`step`).
+|`error`, `delay`.
+|===
+
+A rule fires at every hook of its target that offers its fault; `at=` narrows
it to one (`stall` and `lifecycle`
+take no `at`). A rule whose `at` is not a hook of its target, or whose fault
no hook of its target (after `at`) offers,
+is rejected when it is added, with an error that names the hooks of the
target. A rule that matches nothing, such as a
+poll rule for a consumer that does not poll on a schedule (Kafka, JMS), fires
0 times, as the report shows. So:
+
+- `endpoint=file:inbox* drop` (or `error`) fires at both hooks: a poll of the
`file:inbox` consumer is skipped, and a
+ send to `file:inbox` reports success without writing. `at=poll` or `at=send`
keeps one. With `corrupt` the rule
+ fires at the sends only.
+- `route=orders error` and `route=orders delay` are rejected: the entry of a
route offers only `corrupt`. `route=`
+ does not reach the sends and steps of the route; they are targeted with
`endpoint=`, `component=` or `step=`. One
+ kind of target per hook keeps a rule, and its counts in the report, in one
kind of place.
+- `component=kafka` matches the sends to `kafka:` endpoints and the polls of
its consumers (none: they do not poll
+ on a schedule), not the entry of a route that starts from Kafka, which is
`route=`.
+
+In `application.properties`:
+
+[source,properties]
+----
+camel.chaos.seed = 42
+camel.chaos.rules = \
+ endpoint=kafka:orders* error exception=java.net.ConnectException
message="broker down" rate=0.3 until=1m;\
+ endpoint=https:inventory* error status=503 header.Retry-After=5 rate=every:4
until=50;\
+ route=poller stall duration=20s
+----
+
+With the CLI, against the running integration `myapp` (see Control):
+
+[source,bash]
+----
+camel cmd chaos-add myapp --rule='endpoint=kafka:orders* delay duration=2s
rate=0.2 until=5m'
+camel cmd chaos-add myapp --rule='endpoint=file:inbox* drop at=poll rate=0.5
until=2m'
+camel cmd chaos-add myapp --rule='route=orders corrupt mode=truncate rate=0.1
until=20'
+camel cmd chaos-add myapp --rule='route=orders lifecycle action=remove
duration=10s'
+camel get chaos myapp --report
+----
+
+=== Hooks at all levels
+
+[cols="1,2,2"]
+|===
+|Level |Hook |Faults
+
+|Sends to any endpoint
+|The send hook of camel-core (CAMEL-26331): one helper that the six places
that fire `ExchangeSendingEvent` share
+(`SendProcessor`, `DefaultProducerCache` twice, `EventNotifierCallback`,
`MulticastProcessor`,
+`EventNotifierProducer`), with a hook before and after the producer.
+|`error` (the producer is not called), `delay`, `drop` (a send that reports
success), `corrupt` (the message sent).
+
+|Messages that enter a route
+|A `RoutePolicy`, added to every route by a `RoutePolicyFactory`. Its
`onExchangeBegin` sees every exchange that
+enters a route and can change the message.
+|`corrupt`: header values with random data, a body in an unusual charset, an
`InputStream` that ends too soon. Not
+`drop`: the exchange already belongs to its consumer, and dropping it there
would make the consumer commit a message
+the route never processed (a file consumer deletes the file), a loss caused by
chaos rather than by the route. `drop`
+is at the sends and the polls instead.
+
+|Consumer polls
+|The `PollingConsumerPollStrategy` of a `ScheduledPollConsumer`, which can be
changed at runtime with
+`setPollStrategy`. Its `begin`, `commit` and `rollback` run around every poll.
Chaos wraps the consumer's own strategy
+while a rule is active and puts it back when the rule ends.
+|A failed poll (`error`), a slow poll (`delay`), a skipped poll (`drop`).
+
+|Every step
+|Inside the error handler of the step (CAMEL-26338).
+|`delay` or `error` at a step, raised where the error handler calls the step,
as for a send. (The check that skips a
+processor disabled with `camel cmd disable-processor` runs before the error
handler, so the hook is not there.)
+
+|Route lifecycle
+|The `RouteController` (suspend, resume, stop, start), and
`CamelContext.removeRoute` with the route definition added
+again.
+|`stall` (suspend, then resume); `lifecycle` (suspend, stop or remove the
route, then bring it back).
+|===
+
+The send hook covers every send whatever started it: `to`, `toD`, `wireTap`,
`enrich`, `recipientList`, `multicast`
+and a `ProducerTemplate`. The components stay the real ones (a
`KafkaComponent` stays a `KafkaComponent`), and the
+routes get no extra processors. For a send or a step of a route the fault is
raised inside the error handler, so it is
+a real failure for `onException`, redelivery and the dead letter channel.
+
+A fault has a real type where it matters, so that the `onException` of the
route is what gets tested: for the HTTP
+components, the exception the component throws for a status code
(`HttpOperationFailedException` of camel-http-base,
+`NettyHttpOperationFailedException` for camel-netty-http) with the status code
and headers given in the rule (or the
+response, with `throwExceptionOnFailure=false`); for any other endpoint, the
exception class and message given in the
+rule, or a `ChaosException` when the rule gives none.
+
+=== Time
+
+The `delay` fault sleeps through `TimeSource` (camel-util, CAMEL-25911), and
the report reads its times through it.
+The timed actions (the resume or restart after `stall` or `lifecycle`, the end
of a rule) are tasks on a scheduled
+executor of the `ExecutorServiceManager`, not threads of their own or a
`java.util.Timer`. By default that is the
+system clock, so a person sees a 20 s stall last 20 s.
+
+The seed fixes the sequence of random draws, not which exchange gets which
draw. In real time with concurrent
+consumers the threads decide that, so two runs with the same seed can give
different reports.
+
+In a `camel-test-dst` world the harness runs those executors and installs its
virtual clock in `TimeSource`: a 10 minute
+stall takes milliseconds, and the DST seed decides the order of the tasks. The
world sets `camel.chaos.seed` to that
+seed (`Dst.seed()`, from `@DstTest`, `@DstSeed` or `-Ddst.seed`), or to a
value of its own; with both fixed, a run
+gives the same report again, times included. `camel-chaos` does not depend on
`camel-test-dst`. Event timestamps stay
+on `System` in a DST run (`test-dst.adoc`), so the report does not use them. A
`stall` or `lifecycle` rule meets the
+one-stack model of `test-dst.adoc`: a suspend or stop cannot see an exchange
that is blocked in a synchronous call
+below it on the stack finish, so such a world calls its downstream with
`DstCamel.asyncStep`. Chaos does not need DST.
+
+The Javadoc of `TimeSource` says it is internal and not an API for components.
`camel-chaos` may use it: it reads the
+clock and sleeps through it, and never installs a provider (only a test
harness such as `camel-test-dst` does that).
+The Javadoc is updated to say so with the first chaos pull request.
+
+=== The report
+
+The plugin counts per route with an event notifier, as the prototype does:
completed, failed, handled by the error
+handler, redelivered, and the faults that fired. Attempts are not tied to one
message: each exchange is counted once,
+so a file that fails and completes on the next poll is one failed and one
completed exchange. Tying them by a key (the
+file name, `JMSMessageID`, the Kafka offset) is not in the first version.
+
+A failed exchange shows as "failed, kept by the consumer" when Camel knows
that the consumer keeps the message and
+reads it again, and as "failed" otherwise. (An exchange the error handler
handled is not failed; its consumer commits
+it.) The plugin reads the consumer's endpoint options through its generated
configurer (`PropertyConfigurerGetter`),
+so options set on the component count too, without a dependency on the
components:
+
+[cols="1,3,2"]
+|===
+|Consumer |Kept |Not kept
+
+|`file`, `ftp`, `sftp`, `ftps`
+|Without `moveFailed`: the rollback leaves the file (with `noop=true` it
removes it from the idempotent repository),
+and the next poll reads it again.
+|With `moveFailed`.
+
+|`jms`, `activemq`, `amqp`
+|`transacted=true`, or `acknowledgementModeName=CLIENT_ACKNOWLEDGE` with
`asyncConsumer=false` (the default): the
+listener throws, and the session rolls back or recovers.
+|Otherwise.
+
+|`kafka`
+|`breakOnFirstError=true`: the failed offset is not committed, and the
consumer reads the record again.
+|`breakOnFirstError=false` (the default).
+
+|Other consumers
+|
+|Always.
+|===
+
+Without an expectation the report never says "lost", and the verdict is PASS:
chaos makes exchanges fail on purpose.
+A rule can carry an expectation (`expect=orders:100 within=2m`: route `orders`
completes 100 more messages within
+2m), which also finds messages lost before they became an exchange. Only an
unmet expectation turns PASS into FAIL,
+with the reason, and "lost" only comes from one (100 expected, 97 completed: 3
lost). The report is plain text for the
+CLI and JSON for tools, and states the seed and the rules that were active.
+
+An expectation is decided only when its count is reached (met) or when
`within` has elapsed without it (unmet, FAIL).
+Before that it is "pending", with the count so far and the time left, and the
verdict stays PASS. It stays pending
+when the context stops first (the report of `reportOnShutdown`) or the rule is
removed first: the messages may still
+arrive, so they are not called lost. A tool must treat a pending expectation
as not yet decided, so a PASS with one
+pending is not final.
+
+=== Control
+
+- A `chaos` dev console lists the rules and shows the report, so it is
`@DevConsole(readOnly = true)`. It does not add,
+ change or remove: the flag is metadata only, and `/q/dev/*`
(`ManagementHttpServer`) takes GET and POST with the
+ options in the query string, without checking it. So over HTTP chaos only
lists and reports.
+- A `chaos` action of the CLI connector (`LocalCliConnector`) adds, changes
and removes rules through the
+ `ChaosManager`. The CLI has the flat verb-noun shape of every `camel cmd`
today (`stop-route`, `stub`), with the
+ reads under `camel get` (as `camel get circuit-breaker`): `camel cmd
chaos-add`, `chaos-change`, `chaos-remove`,
+ `chaos-clear`, and `camel get chaos` (the rules, or the report with
`--report`), the same six operations as the MCP
+ tools. Each takes the integration name or pid first. `chaos-add` and
`chaos-change` need it to match exactly one
+ integration, as `camel cmd sql` does; the others may omit it and then act on
all, as `stop-route` does.
+- The camel-jbang MCP server gets a `chaos` runtime tool group next to `sql`,
`tracing`, `resilience` and `http`,
+ offered only when the application has `camel-chaos`. The tools are in
`RuntimeTools` (`@McpSecured`) and go through
+ the CLI connector, as the other runtime tools do; the shared `ToolRegistry`
(also used by the TUI) has the same marks:
++
+[cols="2,1,1"]
+|===
+|Tool |`readOnlyHint` |`destructiveHint`
+
+|`camel_runtime_chaos_add`, `camel_runtime_chaos_change` |`false` |`true`
+|`camel_runtime_chaos_remove`, `camel_runtime_chaos_clear` |`false` |`false`
+|`camel_runtime_chaos_list`, `camel_runtime_chaos_report` |`true` |`false`
+|===
++
+With `camel.mcp.security.enabled=true` and
`camel.mcp.security.access-level=read-write`, add and change are refused,
+as `camel_runtime_route_control`, `_sql`, `_send` and `_receive` are;
`read-only` leaves list and report.
+- The TUI and `camel ask` ask the user before an AI adds or changes a fault,
with the rule shown. Their AI tools ask
+ only for the calls in `ToolConfirms` (CAMEL-25676), not by the destructive
flag, so the chaos PR adds these two
+ there and to "Developer tools" of the security model.
+- JMX is not needed for the first version.
+
+=== Order of work
+
+Each step is its own pull request after this document:
+
+. camel-core: the send hook (CAMEL-26331).
+. The module, the plugin, the options, the rule model, the report, and the
faults on sends.
+. The dev console, the `chaos` action of the CLI connector, the `chaos-*`
commands, `camel get chaos` and
+ `camel run --chaos`.
+. Faults on the messages that enter a route, and on consumer polls.
+. camel-core: the step hook in the channel (CAMEL-26338), then the faults at
steps.
+. Route lifecycle faults.
+. The MCP tool group.
+. A DST world that runs chaos rules in virtual time, as an example and a
regression test.
+
+== Alternatives considered
+
+- *Wrapping the component*, as `camel run --stub` swaps in the stub component
where the component resolver of camel
+ run creates it. It sees every send, but it changes the type of the
component: a `KafkaComponent` is then a wrapper
+ for `getComponent("kafka", KafkaComponent.class)`, the component options,
the dev consoles and the health checks.
+- *`camel run --chaos` only, no module*, as `--stub` lives in camel run. A
plain `Main`, a test and a DST world would
+ not get chaos, and neither would Spring Boot or Quarkus later. With the
module, `--chaos` only adds a dependency.
+- *An event notifier on `ExchangeSendingEvent`.* It fires for every send but
cannot stop one: an exception thrown by an
+ event notifier is logged and ignored, and the producer is called anyway.
+- *`interceptSendToEndpoint`.* It can stop a send, but it wraps the endpoints,
and it needs an intercept definition in
+ the routes or a route configuration, which the user then sees in the route
model.
+- *An `InterceptStrategy` for the steps.* It needs no camel-core change, but
it wraps every processor of every route
+ in one more processor. The channel already runs a check per step for
disabled processors.
+- *Faults from outside the JVM* (a proxy in front of the broker, killing
containers). Useful, but it cannot fail one
+ route and not another, cannot produce a typed exception for `onException`,
and needs infrastructure a developer
+ running `camel run --dev` does not have.
+- *Advice-with in tests.* It changes routes for one test; chaos is for a
running application, and does not change the
+ routes.
+
+== Compatibility
+
+Nothing changes for applications without the JAR. The two camel-core hooks
cost one check each when no hook is
+installed, like the event check that the send places do today. With the hooks
installed and no rules, the send hook, a
+route policy per route and the step hook are idle: one volatile read per send,
per exchange that enters a route and
+per step, a development-time cost only. The new options are in a new group.
+
+== Risks
+
+- The JAR left on the classpath of a production build. Mitigations: the hooks
fail closed (only with `dev` or
+ `camel.chaos.enabled=true`, never with `prod`), a WARN at startup, and rules
that expire. Rules come only from
+ `camel.chaos.rules` and the CLI connector, which the MCP tools use; over
HTTP the dev console only lists and reports.
+- `camel.chaos.rules` is an `insecure:dev` option: `@Metadata(security =
"insecure:dev")` on its field in
+ `ChaosConfigurationProperties`, as on `devConsoleEnabled` in
`DefaultConfigurationProperties`. Profile `prod` sets
+ `camel.security.policy=fail`, so it refuses to start with rules configured;
profile `dev` allows it, and with no
+ profile the default policy `warn` logs a WARN (and the hooks are not
installed). In `SecurityUtils`,
+ `SECURITY_OPTIONS` is keyed by the bare option name and the schematron
component has an endpoint option `rules`, so
+ the chaos module pull request limits the new entry to keys under
`camel.chaos.`; it also lets any non-empty value of
+ that option count, where `isInsecureValue` compares with one insecure value.
+- A verdict that is wrong. The report states what it counted and from where,
so a person can check it.
+
+== Open questions
+
+Each has a proposed answer; the maintainers decide.
+
+- Module location. Proposed: `components/camel-chaos`, next to `camel-debug`.
+- Should chaos stay off when no profile is set? Decided in review: yes (see
"Switched on by the classpath"). No
+ profile is the usual case in production too, and `camel run` already
defaults to `dev`.
+- The rule format. Proposed: the one-line text form of Rule syntax, the same
in `camel.chaos.rules`,
+ `camel cmd chaos-add` and the MCP tools. A YAML form can come later if
wanted.
+
+== Related tickets
+
+- CAMEL-26330: this feature.
+- CAMEL-26331: the send hook in camel-core.
+- CAMEL-26338: the step hook in camel-core.
+- CAMEL-25911: `TimeSource` in camel-util.
+- CAMEL-25684, CAMEL-25682: bugs the prototype reproduced on the code from
before their fixes.
+- CAMEL-26286, CAMEL-26287, CAMEL-26288, CAMEL-26302 to CAMEL-26305: the
"remove a route" bugs that a lifecycle rule
+ produces.
