This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 4e603d0932d0 CAMEL-26817: camel-chaos - every suggested rule says what
should hold, and whether its route gets messages without a feed (#29573)
4e603d0932d0 is described below
commit 4e603d0932d06ecbfb5e981e96d41ba71fbf921f
Author: Claus Ibsen <[email protected]>
AuthorDate: Tue Nov 10 20:56:31 2026 +0100
CAMEL-26817: camel-chaos - every suggested rule says what should hold, and
whether its route gets messages without a feed (#29573)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.../org/apache/camel/catalog/docs/chaos.adoc | 23 ++++--
components/camel-chaos/src/main/docs/chaos.adoc | 23 ++++--
.../apache/camel/component/chaos/ChaosAdvisor.java | 81 +++++++++++++++++++---
.../camel/component/chaos/ChaosExperiment.java | 4 ++
.../apache/camel/component/chaos/ChaosManager.java | 8 ++-
.../chaos/ChaosAdvisorDeadLetterTest.java | 2 +-
.../camel/component/chaos/ChaosAdvisorTest.java | 31 +++++++--
.../camel/component/chaos/ChaosFeedTest.java | 29 +++++++-
.../camel/component/chaos/ChaosLifecycleTest.java | 24 +++++++
.../camel/component/chaos/ChaosPairsTest.java | 10 +--
.../camel/component/chaos/ChaosStubsTest.java | 5 +-
.../modules/ROOT/pages/camel-jbang-managing.adoc | 2 +-
12 files changed, 209 insertions(+), 33 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
index 3557980aed9a..8c62dc8519ab 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
@@ -211,7 +211,7 @@ Chaos rules to try, the most worth trying first:
1. route orders: Returns an error (HTTP status) (errors)
an HTTP call to http://stock/check without a circuit breaker: what does the
route do when the service answers 503?
- camel cmd chaos-add --rule='endpoint=http://stock/check error status=503
rate=0.25 until=10m'
+ camel cmd chaos-add --rule='endpoint=http://stock/check error status=503
rate=0.25 until=10m expect=orders:failed<=0'
2. route orders: The route after a hand-off fails (lost)
route orders hands the message to route shipping over seda and does not
wait: when shipping fails, is the message lost?
@@ -261,6 +261,18 @@ channel fails too. Are messages lost? The report counts
what the dead letter cha
does an exchange take?
|===
+Every suggestion says what should hold despite its faults, as an expectation
(see
+<<_expectations_on_the_error_handler,Expectations on the error handler>>), so
it can be run as an experiment of an
+exploration as it is, and a rule added from it has a verdict that tells
something. A hand-off expects that no message is
+lost between the two routes, and a duplicate that the route receiving the call
does no work twice. Every other
+suggestion expects that no exchange fails in its route
(`expect=orders:failed<=0`): whatever the fault does, each
+exchange it hits is recovered by redelivery or handled by the error handler. A
route without error handling for the
+fault then fails the verdict, which is the finding, and a route with a dead
letter channel, redelivery or a circuit
+breaker with a fallback passes. A fault that fails or stalls a route that
another route of the integration calls (over
+`direct`, `seda` or HTTP to the integration's own server) expects of that
caller instead (`expect=orders:failed<=0` for
+a stalled `payment` that `orders` calls): the route fails where the fault is,
and what should hold is that its caller
+copes. Change the expectation when the route is meant to fail, such as a
sender that should see the error.
+
Two faults at once (kind `combined`) come after the single faults, as most
bugs that lose messages need two failures
that meet. Their rules are added together:
@@ -268,15 +280,18 @@ that meet. Their rules are added together:
----
5. route orders: An outage while the dead letter channel fails (combined)
route orders sends what fails to the dead letter channel direct:parked:
when http://payment/charge is down for a minute and the dead letter channel
fails too, are messages lost? The report counts what the dead letter channel
did not take as lost
- camel cmd chaos-add --rule='endpoint=http://payment/charge error status=503
until=1m' --rule='endpoint=direct:parked error at=dlq until=1m'
+ camel cmd chaos-add --rule='endpoint=http://payment/charge error status=503
until=1m expect=orders:failed<=0' --rule='endpoint=direct:parked error at=dlq
until=1m'
----
-Each of their rules counts as its own kind in what has been tried, so a pair
is never "not tried yet".
+Each of their rules counts as its own kind in what has been tried, so a pair
is never "not tried yet". The expectation
+is on the first rule, and the messages the dead letter channel did not take
fail the verdict too.
The rules leave the query of the endpoint out: its options do not tell which
endpoint it is, and may hold a secret.
`ChaosManager.suggest()` gives the same as JSON: `suggestions`, each with
`route`, `kind`, `recipe`, `title`, `rule`,
`why`, `tried`, `routes` for a dead letter channel that several routes share,
`feed`, and `rules` for two faults at
-once. A `feed` has its `kind` (one of `self`, `http`, `file` and `send`), a
`hint` and `real`. `rules` has all the
+once. A `feed` has its `kind` (one of `self`, `http`, `file` and `send`), a
`hint`, `real`, and `fed` when the route
+gets messages without a feed: it makes its own (a timer), or a route that does
sends to it, also with an HTTP call to
+the server of the integration. Such a suggestion can run as an experiment
without sending anything. `rules` has all the
rules to add together. The chaos screen of the Camel TUI lists the suggestions
(key `w`), where Enter types the rule
in to change or add (the rules of two faults at once separated by `;`).
diff --git a/components/camel-chaos/src/main/docs/chaos.adoc
b/components/camel-chaos/src/main/docs/chaos.adoc
index 3557980aed9a..8c62dc8519ab 100644
--- a/components/camel-chaos/src/main/docs/chaos.adoc
+++ b/components/camel-chaos/src/main/docs/chaos.adoc
@@ -211,7 +211,7 @@ Chaos rules to try, the most worth trying first:
1. route orders: Returns an error (HTTP status) (errors)
an HTTP call to http://stock/check without a circuit breaker: what does the
route do when the service answers 503?
- camel cmd chaos-add --rule='endpoint=http://stock/check error status=503
rate=0.25 until=10m'
+ camel cmd chaos-add --rule='endpoint=http://stock/check error status=503
rate=0.25 until=10m expect=orders:failed<=0'
2. route orders: The route after a hand-off fails (lost)
route orders hands the message to route shipping over seda and does not
wait: when shipping fails, is the message lost?
@@ -261,6 +261,18 @@ channel fails too. Are messages lost? The report counts
what the dead letter cha
does an exchange take?
|===
+Every suggestion says what should hold despite its faults, as an expectation
(see
+<<_expectations_on_the_error_handler,Expectations on the error handler>>), so
it can be run as an experiment of an
+exploration as it is, and a rule added from it has a verdict that tells
something. A hand-off expects that no message is
+lost between the two routes, and a duplicate that the route receiving the call
does no work twice. Every other
+suggestion expects that no exchange fails in its route
(`expect=orders:failed<=0`): whatever the fault does, each
+exchange it hits is recovered by redelivery or handled by the error handler. A
route without error handling for the
+fault then fails the verdict, which is the finding, and a route with a dead
letter channel, redelivery or a circuit
+breaker with a fallback passes. A fault that fails or stalls a route that
another route of the integration calls (over
+`direct`, `seda` or HTTP to the integration's own server) expects of that
caller instead (`expect=orders:failed<=0` for
+a stalled `payment` that `orders` calls): the route fails where the fault is,
and what should hold is that its caller
+copes. Change the expectation when the route is meant to fail, such as a
sender that should see the error.
+
Two faults at once (kind `combined`) come after the single faults, as most
bugs that lose messages need two failures
that meet. Their rules are added together:
@@ -268,15 +280,18 @@ that meet. Their rules are added together:
----
5. route orders: An outage while the dead letter channel fails (combined)
route orders sends what fails to the dead letter channel direct:parked:
when http://payment/charge is down for a minute and the dead letter channel
fails too, are messages lost? The report counts what the dead letter channel
did not take as lost
- camel cmd chaos-add --rule='endpoint=http://payment/charge error status=503
until=1m' --rule='endpoint=direct:parked error at=dlq until=1m'
+ camel cmd chaos-add --rule='endpoint=http://payment/charge error status=503
until=1m expect=orders:failed<=0' --rule='endpoint=direct:parked error at=dlq
until=1m'
----
-Each of their rules counts as its own kind in what has been tried, so a pair
is never "not tried yet".
+Each of their rules counts as its own kind in what has been tried, so a pair
is never "not tried yet". The expectation
+is on the first rule, and the messages the dead letter channel did not take
fail the verdict too.
The rules leave the query of the endpoint out: its options do not tell which
endpoint it is, and may hold a secret.
`ChaosManager.suggest()` gives the same as JSON: `suggestions`, each with
`route`, `kind`, `recipe`, `title`, `rule`,
`why`, `tried`, `routes` for a dead letter channel that several routes share,
`feed`, and `rules` for two faults at
-once. A `feed` has its `kind` (one of `self`, `http`, `file` and `send`), a
`hint` and `real`. `rules` has all the
+once. A `feed` has its `kind` (one of `self`, `http`, `file` and `send`), a
`hint`, `real`, and `fed` when the route
+gets messages without a feed: it makes its own (a timer), or a route that does
sends to it, also with an HTTP call to
+the server of the integration. Such a suggestion can run as an experiment
without sending anything. `rules` has all the
rules to add together. The chaos screen of the Camel TUI lists the suggestions
(key `w`), where Enter types the rule
in to change or add (the rules of two faults at once separated by `;`).
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java
index e93f2da975f1..074803331078 100644
---
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java
@@ -436,10 +436,60 @@ final class ChaosAdvisor {
LOG.debug("No chaos suggestions for route {}: {}",
route.getRouteId(), e.getMessage(), e);
}
}
+ Map<String, String> callers = callers(routes, consumers);
+ answer.replaceAll(s -> withExpectation(s, callers));
answer.sort(Comparator.comparingInt(Suggestion::rank).reversed());
return answer.size() > MAX ? List.copyOf(answer.subList(0, MAX)) :
answer;
}
+ /**
+ * The suggestion with what should hold despite its faults, when none of
its rules says (CAMEL-26817): every
+ * exchange the faults hit in its route is recovered by redelivery or
handled by the error handler, and none fails
+ * ({@code expect=orders:failed<=0}). An experiment needs an expectation,
and a rule added without one passes
+ * whatever happens. For two faults at once the first rule has it. A fault
that fails or stalls the route itself,
+ * when another route of the integration calls it, expects of the caller:
the route fails where the fault is, and
+ * what should hold is that the caller copes.
+ *
+ * @param callers per route, a route of the integration that calls it
+ */
+ static Suggestion withExpectation(Suggestion s, Map<String, String>
callers) {
+ if (s.rules().stream().anyMatch(r -> r.contains(" expect="))) {
+ return s;
+ }
+ String route = s.route();
+ if ((s.kind() == Kind.CONSUMER || s.kind() == Kind.LIFECYCLE) &&
s.rule().startsWith("route=")
+ && callers.containsKey(route)) {
+ route = callers.get(route);
+ }
+ return new Suggestion(
+ s.route(), s.kind(), s.recipe(), s.title(), s.rule() + "
expect=" + route + ":failed<=0", s.why(),
+ s.rank(), s.routes(), s.with());
+ }
+
+ /**
+ * Per route, the first route of the integration that calls it by a static
send: over direct or seda, or with an
+ * HTTP call to its server.
+ */
+ static Map<String, String> callers(List<Route> routes, Map<String, String>
consumers) {
+ Map<String, String> answer = new HashMap<>();
+ for (Route route : routes) {
+ try {
+ for (Processor processor : route.filter("*")) {
+ if (processor instanceof EndpointAware aware &&
aware.getEndpoint() != null) {
+ String called = receiver(uri(aware.getEndpoint()),
consumers);
+ if (called != null &&
!called.equals(route.getRouteId())) {
+ answer.putIfAbsent(called, route.getRouteId());
+ }
+ }
+ }
+ } catch (RuntimeException e) {
+ // the route changed while it was read: no callers from it now
+ LOG.debug("No callers from route {}: {}", route.getRouteId(),
e.getMessage(), e);
+ }
+ }
+ return answer;
+ }
+
private static void suggest(
Route route, Map<String, ChaosModel.Recipe> recipes, Map<String,
String> consumers,
Map<String, List<String>> deadLetters, List<Suggestion> answer) {
@@ -679,6 +729,28 @@ final class ChaosAdvisor {
return reached;
}
+ /** The routes that make their own messages: a timer, a scheduler. */
+ private static List<String> selfFeeding(CamelContext camelContext) {
+ List<String> self = new ArrayList<>();
+ for (Route route : camelContext.getRoutes()) {
+ if (route.getEndpoint() != null
+ &&
SELF_FEEDING.contains(ChaosPatterns.scheme(route.getEndpoint().getEndpointUri())))
{
+ self.add(route.getRouteId());
+ }
+ }
+ return self;
+ }
+
+ /**
+ * The routes that get messages without a feed (CAMEL-26817): the routes
that make their own (a timer, a scheduler),
+ * and the routes they send to, also over HTTP to the integration's own
server. Null when that is not known, as a
+ * route on the way has a send known only when it sends.
+ */
+ static Set<String> fedRoutes(CamelContext camelContext) {
+ Map<String, Boolean> reached = reach(camelContext,
selfFeeding(camelContext), true);
+ return reached != null ? reached.keySet() : null;
+ }
+
/**
* Why an experiment says nothing because its feed does not reach the
routes of its expectations (CAMEL-26803), or
* null when it reaches one, or that is not known. A route of an
expectation gets messages when the route the feed
@@ -689,15 +761,8 @@ final class ChaosAdvisor {
if (feedRoute == null || camelContext.getRoute(feedRoute) == null ||
expected.isEmpty()) {
return null;
}
- List<String> self = new ArrayList<>();
- for (Route route : camelContext.getRoutes()) {
- if (route.getEndpoint() != null
- &&
SELF_FEEDING.contains(ChaosPatterns.scheme(route.getEndpoint().getEndpointUri())))
{
- self.add(route.getRouteId());
- }
- }
Map<String, Boolean> fed = reach(camelContext, List.of(feedRoute),
true);
- Map<String, Boolean> others = reach(camelContext, self, true);
+ Map<String, Boolean> others = reach(camelContext,
selfFeeding(camelContext), true);
if (fed == null || others == null) {
return null;
}
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExperiment.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExperiment.java
index 36207843ee80..3989fd642479 100644
---
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExperiment.java
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExperiment.java
@@ -196,6 +196,10 @@ final class ChaosExperiment {
for (ChaosRule.Hook hook : ChaosRule.Hook.values()) {
fired += r.firedAt(hook);
}
+ if (r.lifecycle != null) {
+ // a stall or lifecycle rule fired when it took its route
down, as the report counts it
+ fired += r.lifecycle.read(now).round();
+ }
faults.put(id, fired);
firedAll += fired;
long kept = r.lostAtDlqCount();
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
index dae9e0ca2d7f..e3d50ac64061 100644
---
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
@@ -531,7 +531,8 @@ public final class ChaosManager {
*
* @return {@code suggestions}: a list of {@code route}, {@code kind},
{@code recipe}, {@code title}, {@code rule},
* {@code why}, {@code tried} when a rule of its kind faulted the
route already, and {@code routes} when it
- * is for several routes (a dead letter channel they share)
+ * is for several routes (a dead letter channel they share), and
the {@code feed} of the route, with
+ * {@code fed} when it gets messages without one
*/
public Map<String, Object> suggest() {
JsonArray list = new JsonArray();
@@ -546,6 +547,8 @@ public final class ChaosManager {
route.tried().forEach(k -> tried.add(route.route() + " " +
k.kind().key));
}
Map<String, ChaosAdvisor.Feed> feeds = new HashMap<>();
+ // the routes that get messages without a feed, so an experiment can
run without one (null when not known)
+ Set<String> fed = ChaosAdvisor.fedRoutes(camelContext);
for (ChaosAdvisor.Suggestion s : ChaosAdvisor.suggest(camelContext)) {
JsonObject jo = new JsonObject();
// a dead letter channel that routes share is tried by a rule of
its kind in any of them; two faults at once
@@ -578,6 +581,9 @@ public final class ChaosManager {
if (feed.real()) {
f.put("real", true);
}
+ if ("self".equals(feed.kind()) || fed != null &&
fed.contains(s.route())) {
+ f.put("fed", true);
+ }
jo.put("feed", f);
}
list.add(jo);
diff --git
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorDeadLetterTest.java
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorDeadLetterTest.java
index 10269380c7c2..a83ed8644689 100644
---
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorDeadLetterTest.java
+++
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorDeadLetterTest.java
@@ -59,7 +59,7 @@ class ChaosAdvisorDeadLetterTest extends ChaosTestSupport {
assertThat(s.route()).isEqualTo("orders");
// the route that consumes the dead letter channel is not one of its
routes
assertThat(s.routes()).containsExactly("orders", "billing", "audit");
- assertThat(s.rule()).isEqualTo("endpoint=seda:dlq error at=dlq");
+ assertThat(s.rule()).isEqualTo("endpoint=seda:dlq error at=dlq
expect=orders:failed<=0");
assertThat(s.why()).startsWith("routes orders, billing, audit send
what fails to the dead letter channel seda:dlq");
}
diff --git
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorTest.java
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorTest.java
index 14a6fd12526c..4df6e1cf6776 100644
---
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorTest.java
+++
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorTest.java
@@ -74,17 +74,22 @@ class ChaosAdvisorTest extends ChaosTestSupport {
Map<String, String> rules = rules(suggestions);
assertThat(rules)
- .containsEntry("orders http-error errors",
"endpoint=http://stock/check error status=503 rate=0.25 until=10m")
+ .containsEntry("orders http-error errors",
+ "endpoint=http://stock/check error status=503
rate=0.25 until=10m expect=orders:failed<=0")
.containsEntry("orders timeout slow",
- "endpoint=http://stock/check error after=5s
exception=java.net.SocketTimeoutException rate=0.1 until=10m")
+ "endpoint=http://stock/check error after=5s
exception=java.net.SocketTimeoutException rate=0.1 until=10m"
+ + "
expect=orders:failed<=0")
// a hand-off over seda: the next route fails, and nothing may
be lost between the two
.containsEntry("orders route-error lost",
"route=shipping error rate=0.2 until=2m
expect=orders->shipping:lost<=0 within=3m")
- .containsEntry("shipping dlq-unreachable dead letter channel",
"endpoint=seda:dlq error at=dlq")
+ .containsEntry("shipping dlq-unreachable dead letter channel",
+ "endpoint=seda:dlq error at=dlq
expect=shipping:failed<=0")
// a circuit breaker: the service is down for a while
- .containsEntry("pay outage errors",
"endpoint=http://pay/charge error rate=burst:10s/1m until=10m")
- .containsEntry("inbox poll-error consumer",
"endpoint=testpoll://inbox error at=poll rate=0.5 until=10m")
- .containsEntry("inbox stall lifecycle", "route=inbox stall
duration=20s");
+ .containsEntry("pay outage errors",
+ "endpoint=http://pay/charge error rate=burst:10s/1m
until=10m expect=pay:failed<=0")
+ .containsEntry("inbox poll-error consumer",
+ "endpoint=testpoll://inbox error at=poll rate=0.5
until=10m expect=inbox:failed<=0")
+ .containsEntry("inbox stall lifecycle", "route=inbox stall
duration=20s expect=inbox:failed<=0");
// no HTTP error without a breaker for the call inside the breaker,
and the query is left out
assertThat(rules).doesNotContainKey("pay http-error errors");
assertThat(rules.values()).noneMatch(r ->
r.contains("throwExceptionOnFailure"));
@@ -93,7 +98,8 @@ class ChaosAdvisorTest extends ChaosTestSupport {
// a local file has no connection to refuse, but the write can fail
assertThat(rules)
.containsEntry("archive write-error errors",
- "endpoint=file://target/chaos-archive error
exception=java.io.IOException rate=0.25 until=10m")
+ "endpoint=file://target/chaos-archive error
exception=java.io.IOException rate=0.25 until=10m"
+ + "
expect=archive:failed<=0")
.doesNotContainKeys("archive connection-refused errors",
"archive slow slow");
// the most worth trying first
@@ -112,6 +118,17 @@ class ChaosAdvisorTest extends ChaosTestSupport {
}
}
+ @Test
+ void everySuggestionSaysWhatShouldHold() {
+ for (ChaosAdvisor.Suggestion s : ChaosAdvisor.suggest(context)) {
+ // one expectation, so the suggestion can be run as an experiment
+ assertThat(s.rules().stream().filter(r -> r.contains("
expect=")).count()).as(s.title()).isEqualTo(1);
+ }
+ // a suggestion with an expectation of its own keeps it
+ assertThat(rules(ChaosAdvisor.suggest(context)).get("orders
route-error lost"))
+ .endsWith("expect=orders->shipping:lost<=0 within=3m");
+ }
+
@Test
void theManagerGivesThemAsJson() {
JsonArray list = (JsonArray) chaos().suggest().get("suggestions");
diff --git
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosFeedTest.java
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosFeedTest.java
index 3b86b093f2d9..98cc30c86992 100644
---
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosFeedTest.java
+++
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosFeedTest.java
@@ -16,6 +16,9 @@
*/
package org.apache.camel.component.chaos;
+import java.util.HashMap;
+import java.util.Map;
+
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.stub.StubComponent;
@@ -50,7 +53,8 @@ class ChaosFeedTest extends ChaosTestSupport {
return new RouteBuilder() {
@Override
public void configure() {
-
from("timer:tick?period=100000").routeId("ticker").to("mock:tick");
+
from("timer:tick?period=100000").routeId("ticker").to("mock:tick").to("direct:orders").to("remotemq:jobs");
+ from("remotemq:jobs").routeId("jobs").to("mock:jobs");
from("direct:orders").routeId("orders").to("mock:orders");
from("file:target/chaos-inbox?initialDelay=100000").routeId("inbox").to("mock:inbox");
from("remotemq:refunds?breakOnFirstError=true").routeId("refunds").to("mock:refunds");
@@ -92,6 +96,29 @@ class ChaosFeedTest extends ChaosTestSupport {
assertThat(payments.getMap("feed").get("real")).isEqualTo(true);
}
+ @Test
+ void theRoutesThatGetMessagesWithoutAFeed() {
+ // the timer makes its own, and sends them to orders and jobs
+
assertThat(ChaosAdvisor.fedRoutes(context)).containsExactlyInAnyOrder("ticker",
"orders", "jobs");
+
+ JsonArray list = (JsonArray) chaos().suggest().get("suggestions");
+ JsonObject refunds = list.stream().map(JsonObject.class::cast)
+ .filter(s ->
"refunds".equals(s.getString("route"))).findFirst().orElseThrow();
+ // refunds gets its messages from outside only
+ assertThat(refunds.getMap("feed").containsKey("fed")).isFalse();
+ }
+
+ @Test
+ void aFaultOfARouteThatAnotherRouteCallsExpectsOfTheCaller() {
+ Map<String, String> rules = new HashMap<>();
+ ChaosAdvisor.suggest(context).forEach(s -> rules.put(s.route() + " " +
s.recipe(), s.rule()));
+ // jobs fails where the fault is: what should hold is that ticker,
which sends to it, copes
+ assertThat(rules.get("jobs
route-error")).endsWith("expect=ticker:failed<=0");
+ assertThat(rules.get("jobs
stall")).endsWith("expect=ticker:failed<=0");
+ // refunds has no caller in the integration: its own failures count
+ assertThat(rules.get("refunds
route-error")).endsWith("expect=refunds:failed<=0");
+ }
+
@Test
void anHttpInputIsFedWithItsMethodAndPath() {
assertThat(ChaosAdvisor.httpMethod("platform-http:///orders?httpMethodRestrict=GET,POST";)).isEqualTo("GET");
diff --git
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosLifecycleTest.java
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosLifecycleTest.java
index 8a1c2a6ec316..811c79d84756 100644
---
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosLifecycleTest.java
+++
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosLifecycleTest.java
@@ -17,6 +17,8 @@
package org.apache.camel.component.chaos;
import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
@@ -121,6 +123,28 @@ class ChaosLifecycleTest extends ChaosTestSupport {
await().atMost(20, TimeUnit.SECONDS).until(() ->
state.equals(lifecycle(id).getString("state")));
}
+ @Test
+ void anExperimentCountsTheStallAsAFault() {
+ chaos().startExploration("10m", null);
+ Map<String, Object> begun = chaos().beginExperiment("orders cope while
route a stalls",
+ "route=a stall duration=200ms expect=orders:failed<=0
within=2s");
+ String id = String.valueOf(((List<?>) begun.get("rules")).get(0));
+ awaitLifecycle(id, "back");
+ template.sendBody("direct:orders", "order");
+ await().atMost(20, TimeUnit.SECONDS).until(() ->
!"PENDING".equals(experiment().getString("verdict")));
+
+ // the stall took the route down: the experiment was put to the test,
it did not say "never fired"
+ JsonObject experiment = experiment();
+ assertThat(((JsonObject)
experiment.get("faults")).getLong(id)).isEqualTo(1);
+ assertThat(experiment.getString("verdict")).isEqualTo("PASS");
+ }
+
+ private JsonObject experiment() {
+ JsonObject exploration = (JsonObject)
chaos().report().toJson().get("exploration");
+ List<Object> experiments = exploration.getCollection("experiments");
+ return (JsonObject) experiments.get(0);
+ }
+
@Test
void stallSuspendsTheRouteAndResumesItAfterTheDuration() {
String id = chaos().addRule("route=a stall duration=2s");
diff --git
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosPairsTest.java
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosPairsTest.java
index 537276989892..a81bef525b2b 100644
---
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosPairsTest.java
+++
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosPairsTest.java
@@ -60,8 +60,9 @@ class ChaosPairsTest extends ChaosTestSupport {
ChaosAdvisor.Suggestion s = pair("An outage while the dead letter
channel fails");
assertThat(s.route()).isEqualTo("orders");
assertThat(s.kind()).isEqualTo(ChaosAdvisor.Kind.COMBINED);
+ // what should hold is on the first rule
assertThat(s.rules()).containsExactly(
- "endpoint=http://payment/charge error status=503 until=1m",
+ "endpoint=http://payment/charge error status=503 until=1m
expect=orders:failed<=0",
"endpoint=seda:parked error at=dlq until=1m");
assertThat(s.why()).contains("dead letter channel seda:parked", "are
messages lost?");
}
@@ -70,7 +71,7 @@ class ChaosPairsTest extends ChaosTestSupport {
void aSlowServiceThatAlsoFails() {
ChaosAdvisor.Suggestion s = pair("A slow service that also fails");
assertThat(s.rules()).containsExactly(
- "endpoint=http://payment/charge delay duration=1s..3s rate=0.5
until=2m",
+ "endpoint=http://payment/charge delay duration=1s..3s rate=0.5
until=2m expect=orders:failed<=0",
"endpoint=http://payment/charge error status=503 rate=0.2
until=2m");
assertThat(s.why()).contains("expect=orders:maxTime<=30s");
}
@@ -96,10 +97,11 @@ class ChaosPairsTest extends ChaosTestSupport {
.filter(s -> "An outage while the dead letter channel
fails".equals(s.getString("title")))
.findFirst().orElseThrow();
assertThat(pair.getString("kind")).isEqualTo("combined");
-
assertThat(pair.getString("rule")).isEqualTo("endpoint=http://payment/charge
error status=503 until=1m");
+ assertThat(pair.getString("rule"))
+ .isEqualTo("endpoint=http://payment/charge error status=503
until=1m expect=orders:failed<=0");
List<Object> rules = pair.getCollection("rules");
assertThat(rules).containsExactly(
- "endpoint=http://payment/charge error status=503 until=1m",
+ "endpoint=http://payment/charge error status=503 until=1m
expect=orders:failed<=0",
"endpoint=seda:parked error at=dlq until=1m");
assertThat(pair.containsKey("tried")).isFalse();
// a single suggestion has no rules list
diff --git
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosStubsTest.java
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosStubsTest.java
index 57d8353dbeb7..ff8ac835e6ad 100644
---
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosStubsTest.java
+++
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosStubsTest.java
@@ -88,9 +88,10 @@ class ChaosStubsTest extends ChaosTestSupport {
void theSuggestionsSeeTheStubbedRemoteEndpoints() {
Map<String, String> rules = rules(ChaosAdvisor.suggest(context));
assertThat(rules)
- .containsEntry("mq route-error consumer", "route=mq error
rate=0.05 until=10m")
+ .containsEntry("mq route-error consumer", "route=mq error
rate=0.05 until=10m expect=mq:failed<=0")
.containsEntry("mq connection-refused errors",
- "endpoint=remotemq://out error
exception=java.net.ConnectException rate=0.25 until=10m");
+ "endpoint=remotemq://out error
exception=java.net.ConnectException rate=0.25 until=10m"
+ + "
expect=mq:failed<=0");
// the stub of audit stands in for nothing, so there is nothing remote
to try in route plain
assertThat(rules.keySet()).noneMatch(k -> k.startsWith("plain "));
}
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-managing.adoc
b/docs/user-manual/modules/ROOT/pages/camel-jbang-managing.adoc
index 625b11255da9..0b120ab261b6 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-managing.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-managing.adoc
@@ -907,7 +907,7 @@ and with `--dry-run` it adds nothing, and prints what the
rule would hit now: th
their route.
`camel cmd chaos-suggest` suggests rules to try per route from what the routes
do (an HTTP call without a circuit
breaker, a dead letter channel, a hand-off over seda, an input from outside),
the most worth trying first, each with
-why and the `chaos-add` command that adds it.
+why and the `chaos-add` command that adds it, with an expectation of what
should hold despite the faults.
`camel get chaos` lists the current rules with their id, version, state,
matches and the faults that fired, and
`--report` shows the report: the verdict, the seed, every version of every
rule, the counts per route, and what has
been tried per route (the kinds of fault, how it went, and what the
suggestions offer that no rule tried yet). Chaos is for
