This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new ac0a8a471ee3 CAMEL-26790: camel-chaos - an experiment that cannot tell
does not pass: a fault that never fired, a route without messages, failures
outside its expectations, a failed expectation behind a dead letter channel
(#29517)
ac0a8a471ee3 is described below
commit ac0a8a471ee3422c3f7f7680e946beb638e7f64f
Author: Claus Ibsen <[email protected]>
AuthorDate: Tue Nov 10 00:20:34 2026 +0100
CAMEL-26790: camel-chaos - an experiment that cannot tell does not pass: a
fault that never fired, a route without messages, failures outside its
expectations, a failed expectation behind a dead letter channel (#29517)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.../org/apache/camel/catalog/docs/chaos.adoc | 22 ++++--
components/camel-chaos/src/main/docs/chaos.adoc | 22 ++++--
.../apache/camel/component/chaos/ChaosAdvisor.java | 2 +-
.../camel/component/chaos/ChaosExperiment.java | 83 +++++++++++++++++++-
.../apache/camel/component/chaos/ChaosManager.java | 19 +++++
.../camel/component/chaos/ChaosExperimentTest.java | 88 ++++++++++++++++++++++
.../action/CamelChaosExperimentAction.java | 11 ++-
.../dsl/jbang/core/commands/process/ListChaos.java | 10 +++
.../core/commands/action/CamelChaosActionTest.java | 9 +++
.../jbang/core/commands/process/ListChaosTest.java | 19 ++++-
.../src/test/resources/golden/cli-help.txt | 12 +--
.../src/test/resources/golden/cli-parse.txt | 65 ++++++++++++++++
12 files changed, 336 insertions(+), 26 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
index 5f74f1a59e07..0c8b96325a9c 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
@@ -161,6 +161,10 @@ names some of the endpoints, routes or steps there are. It
is not an error, as t
creates its endpoints when it sends, and a route can be added later). The
rules of `camel.chaos.rules` are checked when
the context has started.
+A rule that expects failed exchanges in a route whose error handler is a dead
letter channel
+(`expect=parked:failed<=0`) has a warning too: the dead letter channel handles
what fails there, so nothing fails, and
+the expectation is always met. `handled` counts what the dead letter channel
handled (`expect=parked:handled<=0`).
+
To see what a rule would hit before it is added, `chaos-add --dry-run` checks
it as `chaos-add` would, and adds
nothing. It prints the rule in plain words, then each send, poll, route or
step it matches now, with its route (and the
step that sends), the sends known only when they send that may match (a
`toD`), and the warnings:
@@ -391,9 +395,16 @@ faults of each rule, the state of each expectation, and a
verdict:
The dead letter channel then handles the message without delivering it, so no
expectation of a route sees it: the
experiment counts it as lost (`lostAtDeadLetterChannel`, and per rule
`lostAtDeadLetterChannelPerRule`), as the
report does.
-* `PASS`: all expectations are met.
-* `PENDING`: it is still open and an expectation is not decided yet.
-* `INCONCLUSIVE`: it ended before an expectation was decided, so the messages
may still have come.
+* `PASS`: all expectations are met, a fault fired, and the routes of the
expectations had messages.
+* `PENDING`: it is still open, and an expectation is not decided yet or no
fault has fired yet.
+* `INCONCLUSIVE`: it ended before an expectation was decided, so the messages
may still have come; none of its
+faults fired, so its expectations say nothing about them; or an expectation
was met on a route that had no messages
+while it ran (`refunds:failed<=0` when the traffic went to another route), or
is a time (`maxTime`, `meanTime`) of
+such a route. The `reason` says which.
+
+The routes that failed while it ran, and whose failures no expectation of the
experiment counts, are listed as
+`alsoFailed`, with how many failed. An experiment that expects
`refunds:maxTime<=30s` passes while 2 refunds failed,
+and `alsoFailed` says so.
The experiment is ended with the exploration when that finishes, or when its
time is up, with what came of it so far.
@@ -406,8 +417,9 @@ The journal has them as `experiments`, each with:
* `number`, `hypothesis`, the `rules` (their ids) and `ruleTexts`
* `open`, `startedAt` and `elapsed`
-* the `verdict` and the `expectations`: per rule its `expect`, `state` (`met`,
`unmet` or `pending`), the `actual` value,
-and the `value` in words
+* the `verdict` (and the `reason` of an `INCONCLUSIVE` one) and the
`expectations`: per rule its `expect`, `state`
+(`met`, `unmet` or `pending`), the `actual` value, and the `value` in words
+* `alsoFailed`: the routes that failed outside the expectations, when there
are such
* the `faults` per rule
* the `routes` that did something while it ran, with the same counts as the
baseline
* once ended, `endedAt`, `endedBy` (`ended`, `finished` or `deadline`) and the
`note`
diff --git a/components/camel-chaos/src/main/docs/chaos.adoc
b/components/camel-chaos/src/main/docs/chaos.adoc
index 5f74f1a59e07..0c8b96325a9c 100644
--- a/components/camel-chaos/src/main/docs/chaos.adoc
+++ b/components/camel-chaos/src/main/docs/chaos.adoc
@@ -161,6 +161,10 @@ names some of the endpoints, routes or steps there are. It
is not an error, as t
creates its endpoints when it sends, and a route can be added later). The
rules of `camel.chaos.rules` are checked when
the context has started.
+A rule that expects failed exchanges in a route whose error handler is a dead
letter channel
+(`expect=parked:failed<=0`) has a warning too: the dead letter channel handles
what fails there, so nothing fails, and
+the expectation is always met. `handled` counts what the dead letter channel
handled (`expect=parked:handled<=0`).
+
To see what a rule would hit before it is added, `chaos-add --dry-run` checks
it as `chaos-add` would, and adds
nothing. It prints the rule in plain words, then each send, poll, route or
step it matches now, with its route (and the
step that sends), the sends known only when they send that may match (a
`toD`), and the warnings:
@@ -391,9 +395,16 @@ faults of each rule, the state of each expectation, and a
verdict:
The dead letter channel then handles the message without delivering it, so no
expectation of a route sees it: the
experiment counts it as lost (`lostAtDeadLetterChannel`, and per rule
`lostAtDeadLetterChannelPerRule`), as the
report does.
-* `PASS`: all expectations are met.
-* `PENDING`: it is still open and an expectation is not decided yet.
-* `INCONCLUSIVE`: it ended before an expectation was decided, so the messages
may still have come.
+* `PASS`: all expectations are met, a fault fired, and the routes of the
expectations had messages.
+* `PENDING`: it is still open, and an expectation is not decided yet or no
fault has fired yet.
+* `INCONCLUSIVE`: it ended before an expectation was decided, so the messages
may still have come; none of its
+faults fired, so its expectations say nothing about them; or an expectation
was met on a route that had no messages
+while it ran (`refunds:failed<=0` when the traffic went to another route), or
is a time (`maxTime`, `meanTime`) of
+such a route. The `reason` says which.
+
+The routes that failed while it ran, and whose failures no expectation of the
experiment counts, are listed as
+`alsoFailed`, with how many failed. An experiment that expects
`refunds:maxTime<=30s` passes while 2 refunds failed,
+and `alsoFailed` says so.
The experiment is ended with the exploration when that finishes, or when its
time is up, with what came of it so far.
@@ -406,8 +417,9 @@ The journal has them as `experiments`, each with:
* `number`, `hypothesis`, the `rules` (their ids) and `ruleTexts`
* `open`, `startedAt` and `elapsed`
-* the `verdict` and the `expectations`: per rule its `expect`, `state` (`met`,
`unmet` or `pending`), the `actual` value,
-and the `value` in words
+* the `verdict` (and the `reason` of an `INCONCLUSIVE` one) and the
`expectations`: per rule its `expect`, `state`
+(`met`, `unmet` or `pending`), the `actual` value, and the `value` in words
+* `alsoFailed`: the routes that failed outside the expectations, when there
are such
* the `faults` per rule
* the `routes` that did something while it ran, with the same counts as the
baseline
* once ended, `endedAt`, `endedBy` (`ended`, `finished` or `deadline`) and the
`note`
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java
index 5c88f23aca89..1dedff27ba84 100644
---
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java
@@ -519,7 +519,7 @@ final class ChaosAdvisor {
}
/** The dead letter channel of the route, without its options, or null
when its error handler has none. */
- private static String deadLetterUri(Route route) {
+ static String deadLetterUri(Route route) {
if (route.getErrorHandlerFactory() instanceof
DeadLetterChannelDefinition dlc && dlc.getDeadLetterUri() != null) {
String uri = dlc.getDeadLetterUri();
return uri.indexOf('?') > 0 ? uri.substring(0, uri.indexOf('?')) :
uri;
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExperiment.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExperiment.java
index 33f6d676dc53..94ef7db1fc9c 100644
---
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExperiment.java
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExperiment.java
@@ -16,10 +16,13 @@
*/
package org.apache.camel.component.chaos;
+import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
+import java.util.Set;
import java.util.StringJoiner;
+import java.util.TreeSet;
import java.util.function.Function;
import org.apache.camel.component.chaos.ChaosRule.Expectation;
@@ -36,8 +39,11 @@ import org.apache.camel.util.json.JsonObject;
* The verdict is {@code FAIL} when an expectation is unmet, or a rule of it
failed or dropped the send of a dead letter
* channel, which then handled the message without delivering it (lost, as the
report counts it, CAMEL-26789), and
* {@code PASS} when all are met. While it runs and an expectation is not
decided it is {@code PENDING}, and an
- * experiment that ended before an expectation was decided is {@code
INCONCLUSIVE}: the messages may still have come.
- * The manager guards it with its lock.
+ * experiment that ended before an expectation was decided is {@code
INCONCLUSIVE}: the messages may still have come. An
+ * experiment whose faults never fired is not a {@code PASS} either, as its
expectations say nothing about the faults
+ * ({@code INCONCLUSIVE} once it ended, CAMEL-26790). The routes that failed
while it ran, and that no expectation of it
+ * counts the failures of, are listed ({@code alsoFailed}), so a verdict on
another count does not hide them. The
+ * manager guards it with its lock.
*/
final class ChaosExperiment {
@@ -46,6 +52,17 @@ final class ChaosExperiment {
static final String PENDING = "PENDING";
static final String INCONCLUSIVE = "INCONCLUSIVE";
+ // why an experiment is INCONCLUSIVE
+ static final String NEVER_FIRED = "its faults never fired, so its
expectations say nothing about them";
+ static final String NOT_DECIDED = "it ended before an expectation was
decided, so the messages may still have come";
+
+ /** Why an experiment whose expectation was met on a route without
messages is INCONCLUSIVE. */
+ static String noMessages(Set<String> routes) {
+ return (routes.size() == 1 ? "route " : "routes ") + String.join(", ",
routes)
+ + " had no messages while it ran, so " + (routes.size() == 1 ?
"its expectation says" : "their expectations say")
+ + " nothing: feed " + (routes.size() == 1 ? "it" : "them");
+ }
+
/** How an experiment ended. */
enum End {
// endExperiment
@@ -143,6 +160,11 @@ final class ChaosExperiment {
JsonArray texts = new JsonArray();
JsonObject faults = new JsonObject();
JsonArray expectations = new JsonArray();
+ long firedAll = 0;
+ // the routes whose failures an expectation counts, and the routes of
the expectations that are met
+ Set<String> failedExpected = new HashSet<>();
+ Set<String> metRoutes = new TreeSet<>();
+ Set<String> unmetTimes = new HashSet<>();
for (String id : rules) {
ChaosRule r = rule.apply(id);
if (r == null) {
@@ -154,6 +176,7 @@ final class ChaosExperiment {
fired += r.firedAt(hook);
}
faults.put(id, fired);
+ firedAll += fired;
long kept = r.lostAtDlqCount();
if (kept > 0) {
lostAtDlq.put(id, kept);
@@ -161,11 +184,22 @@ final class ChaosExperiment {
}
Expectation e = r.expectation;
if (e != null) {
+ if (e.kind == Expectation.Kind.FAILED) {
+ failedExpected.add(e.route);
+ }
Expectation.Reading reading = e.read(now);
// an expectation the experiment did not wait for is
undecided, not cancelled
Outcome outcome = reading.outcome() == Outcome.CANCELLED ?
Outcome.PENDING : reading.outcome();
- unmet |= outcome == Outcome.UNMET;
+ if (outcome == Outcome.UNMET && e.kind.time()) {
+ // a time is unmet without an exchange too: decided once
the routes are known
+ unmetTimes.add(e.route);
+ } else {
+ unmet |= outcome == Outcome.UNMET;
+ }
undecided |= outcome == Outcome.PENDING;
+ if (outcome == Outcome.MET) {
+ metRoutes.add(e.route);
+ }
JsonObject x = new JsonObject();
x.put("rule", id);
x.put("expect", e.spec());
@@ -182,8 +216,9 @@ final class ChaosExperiment {
jo.put("lostAtDeadLetterChannel", lost);
jo.put("lostAtDeadLetterChannelPerRule", lostAtDlq);
}
- jo.put("verdict", unmet || lost > 0 ? FAIL : !undecided ? PASS : ended
? INCONCLUSIVE : PENDING);
JsonArray routes = new JsonArray();
+ JsonObject alsoFailed = new JsonObject();
+ Set<String> active = new HashSet<>();
counts.forEach((route, c) -> {
long[] now0 = read(c);
long[] then = before.getOrDefault(route, new long[5]);
@@ -194,6 +229,7 @@ final class ChaosExperiment {
any |= delta[i] > 0;
}
if (any) {
+ active.add(route);
JsonObject r = new JsonObject();
r.put("route", route);
r.put("completed", delta[0]);
@@ -202,9 +238,38 @@ final class ChaosExperiment {
r.put("handled", delta[3]);
r.put("redeliveries", delta[4]);
routes.add(r);
+ if (delta[1] > 0 && !failedExpected.contains(route)) {
+ alsoFailed.put(route, delta[1]);
+ }
}
});
jo.put("routes", routes);
+ if (!alsoFailed.isEmpty()) {
+ jo.put("alsoFailed", alsoFailed);
+ }
+ // nothing fired: the expectations were not put to the test (an unmet
one is still a FAIL, with its evidence).
+ // This is for the experiment as a whole: when one of its rules fired,
the experiment was put to the test,
+ // and the expectations still decide each rule
+ boolean neverFired = firedAll == 0;
+ // an expectation met while its route had no messages says nothing
either, such as refunds:failed<=0 when the
+ // traffic went to another route
+ for (String route : unmetTimes) {
+ if (active.contains(route)) {
+ unmet = true;
+ } else {
+ // maxTime<=10s with no exchange in its route says nothing
about the time either
+ metRoutes.add(route);
+ }
+ }
+ metRoutes.removeAll(active);
+ boolean idle = !metRoutes.isEmpty();
+ String verdict = unmet || lost > 0
+ ? FAIL : !undecided && !neverFired && !idle ? PASS : ended ?
INCONCLUSIVE : PENDING;
+ jo.put("verdict", verdict);
+ if (INCONCLUSIVE.equals(verdict)) {
+ // never fired comes first: when no fault ran, the expectations
that did not decide are a consequence of it
+ jo.put("reason", neverFired ? NEVER_FIRED : idle ?
noMessages(metRoutes) : NOT_DECIDED);
+ }
if (countsReset) {
jo.put("countsReset", true);
}
@@ -239,11 +304,21 @@ final class ChaosExperiment {
sb.append("\n expect
").append(x.getString("expect")).append(": ").append(x.getString("state"))
.append(", ").append(x.getString("value"));
}
+ String reason = jo.getString("reason");
+ if (reason != null) {
+ sb.append("\n inconclusive: ").append(reason);
+ }
long lost = jo.getLongOrDefault("lostAtDeadLetterChannel", 0);
if (lost > 0) {
sb.append("\n lost at the dead letter channel:
").append(lost)
.append(lost == 1 ? " message" : " messages").append(",
handled there without being delivered");
}
+ JsonObject alsoFailed = (JsonObject) jo.get("alsoFailed");
+ if (alsoFailed != null) {
+ StringJoiner also = new StringJoiner(", ");
+ alsoFailed.forEach((route, n) -> also.add(route + " " + n));
+ sb.append("\n also failed, in no expectation:
").append(also);
+ }
StringJoiner sj = new StringJoiner(", ");
for (Object o : jo.getCollection("routes")) {
JsonObject r = (JsonObject) o;
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
index 404e77c66d24..f711af4f1304 100644
---
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
@@ -452,9 +452,28 @@ public final class ChaosManager {
if (target != null) {
answer.add(target);
}
+ String dead = deadLetterOfFailedExpectation(rule);
+ if (dead != null) {
+ String route = rule.expectation.route;
+ answer.add("expects failed exchanges in route " + route + ", but
its error handler is the dead letter channel "
+ + dead + ", which handles what fails there, so nothing
fails: expect=" + route
+ + ":handled<=0 counts what the dead letter channel
handled");
+ }
return answer;
}
+ /**
+ * The dead letter channel of the route of an expectation of failed
exchanges ({@code expect=parked:failed<=0}), as
+ * such an expectation is always met there (CAMEL-26790), or null.
+ */
+ private String deadLetterOfFailedExpectation(ChaosRule rule) {
+ if (rule.expectation == null || rule.expectation.kind !=
ChaosRule.Expectation.Kind.FAILED) {
+ return null;
+ }
+ Route route = camelContext.getRoute(rule.expectation.route);
+ return route != null ? ChaosAdvisor.deadLetterUri(route) : null;
+ }
+
/**
* A rule in plain words, such as {@code fail half of the sends to
http:stock* with HTTP status 503, for 5 minutes},
* at the hooks where it can fire in the context now (see the {@code
hooks} of the report).
diff --git
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosExperimentTest.java
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosExperimentTest.java
index aa4ff153c363..afbf20d548cb 100644
---
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosExperimentTest.java
+++
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosExperimentTest.java
@@ -161,6 +161,94 @@ class ChaosExperimentTest extends ChaosTestSupport {
"lost at the dead letter channel: 3 messages, handled there
without being delivered");
}
+ @Test
+ void anExperimentWhoseFaultsNeverFiredDoesNotPass() {
+ chaos().startExploration("10m", null);
+ // the fault is on a step the orders do not reach (CAMEL-26790): its
expectation is met, but says nothing
+ chaos().beginExperiment("orders keep going when parking fails",
+ "endpoint=mock:parked error expect=orders:2 within=1m");
+ send(2);
+ assertThat(experiment(1).getString("verdict")).isEqualTo("PENDING");
+
+ Map<String, Object> ended = chaos().endExperiment(null);
+
+ assertThat(ended).containsEntry("verdict",
"INCONCLUSIVE").containsEntry("reason",
+ "its faults never fired, so its expectations say nothing about
them");
+ assertThat(first(new JsonObject(ended),
"expectations").getString("state")).isEqualTo("met");
+ assertThat(chaos().report().toText()).contains("1 INCONCLUSIVE orders
keep going when parking fails",
+ "inconclusive: its faults never fired");
+
+ // ended before its expectation was decided: the other reason
+ chaos().beginExperiment(null, "endpoint=mock:shipped delay
duration=10ms expect=orders:5 within=1m");
+ send(1);
+ assertThat(chaos().endExperiment(null)).containsEntry("verdict",
"INCONCLUSIVE").containsEntry("reason",
+ "it ended before an expectation was decided, so the messages
may still have come");
+ }
+
+ @Test
+ void anExpectationMetOnARouteWithoutMessagesDoesNotPass() {
+ chaos().startExploration("10m", null);
+ // the faults fire in orders, but the expectation is about billing,
which gets no message (CAMEL-26790)
+ chaos().beginExperiment("billing never fails", "endpoint=mock:shipped
error expect=billing:failed<=0 within=1m");
+ send(3);
+ clock.advance(61_000);
+
+ Map<String, Object> ended = chaos().endExperiment(null);
+
+ assertThat(ended).containsEntry("verdict",
"INCONCLUSIVE").containsEntry("reason",
+ "route billing had no messages while it ran, so its
expectation says nothing: feed it");
+ assertThat(first(new JsonObject(ended),
"expectations").getString("state")).isEqualTo("met");
+ assertThat(((JsonObject)
ended.get("faults")).getLong("r1")).isEqualTo(3);
+
+ // a time without an exchange in its route: unmet, but it says nothing
about the time either
+ chaos().beginExperiment("billing is fast", "endpoint=mock:shipped
error expect=billing:maxTime<=1s within=1m");
+ send(3);
+ clock.advance(61_000);
+ ended = chaos().endExperiment(null);
+ assertThat(ended).containsEntry("verdict",
"INCONCLUSIVE").containsEntry("reason",
+ "route billing had no messages while it ran, so its
expectation says nothing: feed it");
+ assertThat(first(new JsonObject(ended),
"expectations").getString("state")).isEqualTo("unmet");
+ }
+
+ @Test
+ void theRoutesThatFailedOutsideTheExpectationsAreListed() {
+ chaos().startExploration("10m", null);
+ // the expectation counts the completed orders, not the failed ones
(CAMEL-26790)
+ chaos().beginExperiment("orders keep coming", "endpoint=mock:shipped
error rate=every:2 expect=orders:2 within=1m");
+ send(4);
+
+ Map<String, Object> ended = chaos().endExperiment(null);
+
+ assertThat(ended).containsEntry("verdict", "PASS");
+ assertThat(((JsonObject)
ended.get("alsoFailed")).getLong("orders")).isEqualTo(2);
+ assertThat(chaos().report().toText()).contains("also failed, in no
expectation: orders 2");
+
+ // an expectation of the failures counts them: not listed again
+ chaos().beginExperiment(null, "endpoint=mock:shipped error
rate=every:2 expect=orders:failed<=5 within=1m");
+ send(2);
+ clock.advance(61_000);
+ assertThat(chaos().endExperiment(null)).containsEntry("verdict",
"PASS").doesNotContainKey("alsoFailed");
+ }
+
+ @Test
+ void anExpectationOfFailuresInARouteWithADeadLetterChannelIsWarned() {
+ // billing fails to its dead letter channel, which handles the
message: failed stays 0 there (CAMEL-26790)
+ JsonObject checked = (JsonObject) list(
+ new JsonObject(chaos().checkRules("endpoint=mock:charged error
expect=billing:failed<=0")), "rules")
+ .get(0);
+ assertThat(list(checked,
"warnings")).singleElement().asString().contains(
+ "expects failed exchanges in route billing, but its error
handler is the dead letter channel"
+
+ " direct:parked",
+ "expect=billing:handled<=0 counts what the dead letter channel
handled");
+
+ // what the dead letter channel handled, or a route without one: no
warning
+ for (String rule : List.of("endpoint=mock:charged error
expect=billing:handled<=0",
+ "endpoint=mock:shipped error expect=orders:failed<=0")) {
+ JsonObject ok = (JsonObject) list(new
JsonObject(chaos().checkRules(rule)), "rules").get(0);
+ assertThat(ok.containsKey("warnings")).as(rule).isFalse();
+ }
+ }
+
@Test
void oneAtATimeWithAnExpectationAndInAnExploration() {
String rule = "endpoint=mock:shipped error expect=orders:failed<=0
within=1m";
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosExperimentAction.java
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosExperimentAction.java
index e2b2404ec08c..3d6a42842a7f 100644
---
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosExperimentAction.java
+++
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosExperimentAction.java
@@ -47,9 +47,10 @@ import picocli.CommandLine;
"what the routes did, and the traffic. It stays
in the journal of the exploration",
"(camel get chaos --report).",
"%nThe verdict is FAIL when an expectation is
unmet (exit code 1), PASS when all are",
- "met, and INCONCLUSIVE when it ended before an
expectation was decided: wait longer",
- "than the within= of the expectation. --feed can
be the id of a route, as the feed:",
- "line of camel cmd chaos-suggest has it.",
+ "met, and INCONCLUSIVE when it cannot tell: it
ended before an expectation was decided",
+ "(wait longer than the within= of the
expectation), its faults never fired, or the",
+ "route of a met expectation had no messages.
--feed can be the id of a route, as the",
+ "feed: line of camel cmd chaos-suggest has it.",
"%nExamples:",
" camel cmd chaos-experiment --hypothesis='a
failed refund is read again' \\",
" --rule='route=refunds error rate=every:5
expect=refunds:failed<=0 within=30s' \\",
@@ -202,7 +203,9 @@ public class CamelChaosExperimentAction extends
CamelChaosAction {
if (!sb.isEmpty()) {
lines.add(" " + sb);
}
- if ("INCONCLUSIVE".equals(experiment.getString("verdict"))) {
+ // the reason of the verdict says why; waiting longer helps only when
it ended before a decision (CAMEL-26790)
+ String reason = experiment.getString("reason");
+ if ("INCONCLUSIVE".equals(experiment.getString("verdict")) && (reason
== null || reason.startsWith("it ended"))) {
lines.add(" An expectation was not decided when it ended: wait
longer than its within= (--wait)");
}
return lines;
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaos.java
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaos.java
index 4d26bf1f35c1..f52fb81f9dcd 100644
---
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaos.java
+++
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaos.java
@@ -389,11 +389,21 @@ public class ListChaos extends ProcessWatchCommand {
+ x.getString("value"));
}
}
+ String reason = experiment.getString("reason");
+ if (reason != null) {
+ lines.add(more + "inconclusive: " + reason);
+ }
long lost = experiment.getLongOrDefault("lostAtDeadLetterChannel", 0);
if (lost > 0) {
lines.add(more + "lost at the dead letter channel: " + lost +
(lost == 1 ? " message" : " messages")
+ ", handled there without being delivered");
}
+ JsonObject alsoFailed = (JsonObject) experiment.get("alsoFailed");
+ if (alsoFailed != null && !alsoFailed.isEmpty()) {
+ StringJoiner also = new StringJoiner(", ");
+ alsoFailed.forEach((route, n) -> also.add(route + " " + n));
+ lines.add(more + "also failed, in no expectation: " + also);
+ }
List<Object> routes = experiment.getCollection("routes");
if (routes != null && !routes.isEmpty()) {
lines.add(more + "routes: " + routeCounts(routes));
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosActionTest.java
b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosActionTest.java
index 6a84a4bd54ce..a4ec332abbf1 100644
---
a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosActionTest.java
+++
b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosActionTest.java
@@ -573,6 +573,15 @@ class CamelChaosActionTest extends
ActionCommandTestSupport {
" waited 2m0s for the verdict; 8m15s left of the exploration",
" An expectation was not decided when it ended: wait longer
than its within= (--wait)");
+ // another reason: waiting longer would not help (CAMEL-26790)
+ JsonObject idle = new JsonObject(
+ Map.of("number", 4, "verdict", "INCONCLUSIVE", "rules", new
JsonArray(List.of("r4")),
+ "reason", "route billing had no messages while it ran,
so its expectation says nothing: feed it"));
+ assertThat(CamelChaosExperimentAction.lines(idle))
+ .contains(
+ " inconclusive: route billing had no messages while
it ran, so its expectation says nothing: feed it")
+ .doesNotContain(" An expectation was not decided when it
ended: wait longer than its within= (--wait)");
+
JsonObject notSent = new JsonObject(
Map.of("number", 3, "verdict", "PASS", "elapsed", 1_000L,
"traffic", new JsonObject(Map.of("endpoint",
"refunds", "status", "failed", "error", "No endpoint"))));
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaosTest.java
b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaosTest.java
index 6c06f3e46c9c..f3e9a502f341 100644
---
a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaosTest.java
+++
b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaosTest.java
@@ -221,7 +221,20 @@ class ListChaosTest extends ProcessCommandTestSupport {
experiment.put("routes", new JsonArray(List.of(during)));
experiment.put("note", "orders has no error handler");
experiment.put("lostAtDeadLetterChannel", 58L);
- exploration.put("experiments", new JsonArray(List.of(experiment)));
+ JsonObject alsoFailed = new JsonObject();
+ alsoFailed.put("refunds", 2L);
+ experiment.put("alsoFailed", alsoFailed);
+ // its fault never fired
+ JsonObject second = new JsonObject();
+ second.put("number", 2);
+ second.put("open", false);
+ second.put("endedBy", "ended");
+ second.put("elapsed", 41_000L);
+ second.put("rules", new JsonArray(List.of("r3")));
+ second.put("verdict", "INCONCLUSIVE");
+ second.put("reason", "its faults never fired, so its expectations say
nothing about them");
+ second.put("expectations", new JsonArray());
+ exploration.put("experiments", new JsonArray(List.of(experiment,
second)));
((JsonObject) chaos.get("report")).put("exploration", exploration);
writeStatus(chaos);
@@ -233,8 +246,10 @@ class ListChaosTest extends ProcessCommandTestSupport {
"experiments:", "1 FAIL an order never fails (r1, r2, ended
after 1m4s)",
"expect orders:failed<=0: unmet, 9 failed",
"lost at the dead letter channel: 58 messages, handled there
without being delivered",
+ "also failed, in no expectation: refunds 2",
"routes: orders 41 completed, 9 failed",
- "note: orders has no error handler");
+ "note: orders has no error handler", "2 INCONCLUSIVE",
+ "inconclusive: its faults never fired, so its expectations say
nothing about them");
}
@Test
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/test/resources/golden/cli-help.txt
b/dsl/camel-jbang/camel-jbang-core/src/test/resources/golden/cli-help.txt
index db328bcb784c..2b32127c9287 100644
--- a/dsl/camel-jbang/camel-jbang-core/src/test/resources/golden/cli-help.txt
+++ b/dsl/camel-jbang/camel-jbang-core/src/test/resources/golden/cli-help.txt
@@ -807,11 +807,13 @@ what the routes did, and the traffic. It stays in the
journal of the exploration
The verdict is FAIL when an expectation is unmet (exit code 1), PASS when all
are
-met, and INCONCLUSIVE when it ended before an expectation was decided: wait
-longer
-than the within= of the expectation. --feed can be the id of a route, as the
-feed:
-line of camel cmd chaos-suggest has it.
+met, and INCONCLUSIVE when it cannot tell: it ended before an expectation was
+decided
+(wait longer than the within= of the expectation), its faults never fired, or
+the
+route of a met expectation had no messages. --feed can be the id of a route, as
+the
+feed: line of camel cmd chaos-suggest has it.
Examples:
camel cmd chaos-experiment --hypothesis='a failed refund is read again' \
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/test/resources/golden/cli-parse.txt
b/dsl/camel-jbang/camel-jbang-core/src/test/resources/golden/cli-parse.txt
index dbae4927d33f..e44083f6722e 100644
--- a/dsl/camel-jbang/camel-jbang-core/src/test/resources/golden/cli-parse.txt
+++ b/dsl/camel-jbang/camel-jbang-core/src/test/resources/golden/cli-parse.txt
@@ -551,6 +551,65 @@
(defaults)
> cmd chaos-clear --timeout 7
timeoutOption.timeout=7
+> cmd chaos-experiment
+ body=null
+ cmd:helpRequested=false
+ end=false
+ feed=null
+ headers=null
+ helpRequested=false
+ hypothesis=null
+ interval=null
+ name=null
+ note=null
+ repeat=0
+ rules=null
+ timeoutOption.timeout=10000
+ wait="2m"
+> cmd chaos-experiment --body v
+ body="v"
+> cmd chaos-experiment --end
+ end=true
+> cmd chaos-experiment --end=false
+ (defaults)
+> cmd chaos-experiment --feed v
+ feed="v"
+> cmd chaos-experiment --header v --header w
+ headers=["v", "w"]
+> cmd chaos-experiment --hypothesis v
+ hypothesis="v"
+> cmd chaos-experiment --interval v
+ interval="v"
+> cmd chaos-experiment --note v
+ note="v"
+> cmd chaos-experiment --repeat 7
+ repeat=7
+> cmd chaos-experiment --rule v --rule w
+ rules=["v", "w"]
+> cmd chaos-experiment --timeout 7
+ timeoutOption.timeout=7
+> cmd chaos-experiment --wait v
+ wait="v"
+> cmd chaos-explore-finish
+ cmd:helpRequested=false
+ helpRequested=false
+ name=null
+ timeoutOption.timeout=10000
+> cmd chaos-explore-finish --timeout 7
+ timeoutOption.timeout=7
+> cmd chaos-explore-start
+ budget="10m"
+ cmd:helpRequested=false
+ focus=null
+ helpRequested=false
+ name=null
+ timeoutOption.timeout=10000
+> cmd chaos-explore-start --budget v
+ budget="v"
+> cmd chaos-explore-start --focus v
+ focus="v"
+> cmd chaos-explore-start --timeout 7
+ timeoutOption.timeout=7
> cmd chaos-load v
cmd:helpRequested=false
file="v"
@@ -1090,6 +1149,7 @@
headers=null
helpRequested=false
infra=null
+ interval=null
loggingColor=true
mavenResolver.download=true
mavenResolver.fresh=false
@@ -1099,6 +1159,7 @@
pretty=false
propertiesFiles=null
property=null
+ repeat=0
reply=false
replyFile=null
showBody=true
@@ -1123,6 +1184,8 @@
headers=["v", "w"]
> cmd send --infra v
infra="v"
+> cmd send --interval v
+ interval="v"
> cmd send --logging-color
(defaults)
> cmd send --logging-color=false
@@ -1139,6 +1202,8 @@
propertiesFiles="v"
> cmd send --property v --property w
property=["v", "w"]
+> cmd send --repeat 7
+ repeat=7
> cmd send --reply
reply=true
> cmd send --reply=false
