This is an automated email from the ASF dual-hosted git repository. davsclaus pushed a commit to branch CAMEL-26732-chaos-handson-fixes in repository https://gitbox.apache.org/repos/asf/camel.git
commit ce774c23ffda0fc2f4045b119ff9f998c709b39f Author: Claus Ibsen <[email protected]> AuthorDate: Sun Nov 8 11:53:50 2026 +0100 CAMEL-26732: camel-chaos - what to try, tried and test with AI: four findings from a hands-on round Found by running an Orders demo with camel run --chaos and trying what to try, what has been tried and test with AI by hand: - A dead letter channel that the routes share (the error handler of a route builder) was suggested for every route, and for the route that consumes it. It is now suggested once, with the routes that share it (routes in the JSON), and a rule of its kind in any of them tries it. - The hand-off suggestion (route orders, kind lost: route=shipping error ... expect=orders->shipping:lost<=0) was never marked tried, as the rule fires in shipping and is of kind consumer there. A rule that expects no message lost between two routes now also tries lost for the first route. - camel cmd chaos-suggest prints "- tried" for a suggestion of a kind tried already, and the routes of a shared dead letter channel. - The scenario that t wrote did not replay the same faults: a rule draws by a key, its id (r2) when added while running and its place (r1) in a scenario. A new rule option draw=<id> draws as that rule did, and t writes the rule with draw=<its id>, and a comment line in words. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Claus Ibsen <[email protected]> --- .../org/apache/camel/catalog/docs/chaos.adoc | 24 ++++- .../org/apache/camel/chaos/chaos-model.json | 12 +++ components/camel-chaos/src/main/docs/chaos.adoc | 24 ++++- .../apache/camel/component/chaos/ChaosAdvisor.java | 109 ++++++++++++++++----- .../apache/camel/component/chaos/ChaosManager.java | 22 +++-- .../apache/camel/component/chaos/ChaosModel.java | 5 + .../apache/camel/component/chaos/ChaosRule.java | 5 +- .../camel/component/chaos/ChaosRuleParser.java | 19 +++- .../chaos/ChaosAdvisorDeadLetterTest.java | 92 +++++++++++++++++ .../camel/component/chaos/ChaosModelTest.java | 2 +- .../camel/component/chaos/ChaosScenarioTest.java | 19 ++++ .../camel/component/chaos/ChaosTriedTest.java | 35 +++++++ .../commands/action/CamelChaosSuggestAction.java | 12 ++- .../core/commands/action/CamelChaosActionTest.java | 19 ++++ .../dsl/jbang/core/commands/tui/ChaosAiPrompt.java | 19 +++- .../jbang/core/commands/tui/ChaosAiPromptTest.java | 15 +++ 16 files changed, 386 insertions(+), 47 deletions(-) diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc index 3f60d53e8671..286d04d81cc3 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc @@ -212,7 +212,9 @@ does the fallback answer? |A call over another protocol |The connection is refused, and the call is slow |An input that can deliver a message again (Kafka, JMS, AMQP, SQS and more, or a consumer that reads a failed message again, such as a file without `moveFailed`) |The call out is made twice (`duplicate`): is it safe to repeat? -|A dead letter channel |It fails too (`at=dlq`): is the message then lost? Add it with a rule that fails the route. +|A dead letter channel |It fails too (`at=dlq`): is the message then lost? Add it with a rule that fails a route. A +dead letter channel that the routes share (the error handler of a route builder) is suggested once, for all of them, +and not for the route that consumes it. |A hand-off to another route over `seda`, `vm` or `disruptor` |The next route fails, with an expectation that no message is lost between the two routes (see <<_expectations_on_time_and_loss,Expectations on time and loss>>) |An input from outside (not `direct` or `seda`) |The route fails at its entry (what does the sender see: a rollback, a @@ -220,7 +222,8 @@ retry, an error?), or the polls fail for a polling consumer; and the route stall |=== The rules leave the query of the endpoint out: its options do not tell which endpoint it is, and may hold a secret. -`ChaosManager.suggest()` gives the same as JSON (`suggestions`: `route`, `kind`, `recipe`, `title`, `rule`, `why`), and +`ChaosManager.suggest()` gives the same as JSON (`suggestions`: `route`, `kind`, `recipe`, `title`, `rule`, `why`, +`tried`, and `routes` for a dead letter channel that several routes share), and the chaos screen of the Camel TUI lists them (key `w`), where Enter types the rule in to change or add. === What has been tried @@ -243,7 +246,9 @@ $ camel get chaos --report `consumer` (the polls, or the entry of the route), `lifecycle` (stall and lifecycle) and `dead letter channel` (`at=dlq`). A rule has one kind, from its fault and where it acts. * A kind is tried in a route when a rule of that kind fired in it: a rule counts its faults per route, so a rule that -reaches several routes is tried in each. The rules removed or replaced count too, so the report says what the +reaches several routes is tried in each. A rule that expects no message lost between two routes +(`expect=orders->shipping:lost<=0`) also tries `lost` for the first route, as the hand-off suggestion fails the next +route. A dead letter channel that the routes share is tried by a rule of its kind in any of them. The rules removed or replaced count too, so the report says what the experiment tried, not only what runs now. * The outcome is how the exchanges the rules faulted came out (recovered by a redelivery, handled by the error handler, failed), counted per rule, and the outcomes of their expectations. @@ -252,7 +257,8 @@ left, the route has been tried for what chaos can say about it. In JSON the report has `tried`: per `route` its `kinds` (`kind`, `rules`, `fired`, `recovered`, `handled`, `failed`, `expectations`) and `notTried`. `ChaosManager.suggest()` marks a suggestion whose kind was tried in its route already -(`tried`), and the what to try list of the Camel TUI (key `w`) shows both. +(`tried`): `camel cmd chaos-suggest` ends its line with `- tried`, and the what to try list of the Camel TUI (key `w`) +shows both. === From an AI agent (MCP) @@ -335,7 +341,8 @@ way again (the same exception and message at the same step, or the same HTTP sta fix the bug, then prove the fix against the same failure, and keep the rule in a scenario. And `t` on a rule asks its AI panel to make a test of it: a JUnit test with `ChaosAssertions` for a Maven project, or a scenario with an expectation and the `camel run --chaos=<file> --max-seconds=<seconds>` command, whose exit code is the verdict, for a -folder that runs with `camel run`. +folder that runs with `camel run`. The test has the seed, and the rule with `draw=<its id>`, so it fails the same +messages as the rule did (see <<_time_and_the_seed,Time and the seed>>). == Rules @@ -364,6 +371,7 @@ expect: expect=<route id>:<count> (at least that many completed exchanges), or expect=<route id>:meanTime|maxTime<op><time> (such as expect=orders:meanTime<=2s), or the messages lost between two routes expect=<route id>-><route id>:lost<op><count> (such as expect=orders->shipping:lost<=0); with an optional within=<time> +draw: draw=<rule id> draws the random choices as the rule with that id did (draw=r2), see Time and the seed ---- The patterns of the targets are matched as text, with `*` for any characters; they are not regular expressions. The @@ -991,6 +999,12 @@ exchange gets which draw: with several threads (concurrent consumers, parallel p so two runs with the same seed can give different reports. With one thread and the same exchanges in the same order, the same seed gives the same faults. +Each rule draws by a key, with the seed: its id when it is added (`r2`), and its place in a scenario (the second rule +draws as `r2`) or in `camel.chaos.rules`. So a rule added while the application runs, written alone in a scenario or a +test, draws other choices there (it is the first rule, `r1`). `draw=<id>` names the key: `route=shipping error rate=0.2 +draw=r2` with the same seed fails the same messages as rule `r2` did. The test that the Camel TUI makes of a rule (key +`t`) writes it. + == Chaos in a test A finding of a chaos session becomes a regression test: the same rules, the same seed, and assertions on the report. diff --git a/components/camel-chaos/src/generated/resources/META-INF/org/apache/camel/chaos/chaos-model.json b/components/camel-chaos/src/generated/resources/META-INF/org/apache/camel/chaos/chaos-model.json index 1e7ad670ecd5..e99ffa8d907f 100644 --- a/components/camel-chaos/src/generated/resources/META-INF/org/apache/camel/chaos/chaos-model.json +++ b/components/camel-chaos/src/generated/resources/META-INF/org/apache/camel/chaos/chaos-model.json @@ -190,6 +190,7 @@ ], "options": [ "at", + "draw", "expect", "from", "in", @@ -227,6 +228,7 @@ ], "options": [ "at", + "draw", "expect", "from", "in", @@ -253,6 +255,7 @@ ], "options": [ "at", + "draw", "expect", "from", "in", @@ -301,6 +304,7 @@ ], "options": [ "at", + "draw", "expect", "from", "in", @@ -325,6 +329,7 @@ ], "options": [ "at", + "draw", "expect", "from", "in", @@ -444,6 +449,13 @@ "step" ] }, + { + "name": "draw", + "type": "ruleId", + "required": false, + "description": "Draws the random choices of rate= as the rule with this id did with the same seed, so a scenario or a test fails the same messages as a rule added while the application ran", + "example": "r2" + }, { "name": "expect", "type": "expectation", diff --git a/components/camel-chaos/src/main/docs/chaos.adoc b/components/camel-chaos/src/main/docs/chaos.adoc index 3f60d53e8671..286d04d81cc3 100644 --- a/components/camel-chaos/src/main/docs/chaos.adoc +++ b/components/camel-chaos/src/main/docs/chaos.adoc @@ -212,7 +212,9 @@ does the fallback answer? |A call over another protocol |The connection is refused, and the call is slow |An input that can deliver a message again (Kafka, JMS, AMQP, SQS and more, or a consumer that reads a failed message again, such as a file without `moveFailed`) |The call out is made twice (`duplicate`): is it safe to repeat? -|A dead letter channel |It fails too (`at=dlq`): is the message then lost? Add it with a rule that fails the route. +|A dead letter channel |It fails too (`at=dlq`): is the message then lost? Add it with a rule that fails a route. A +dead letter channel that the routes share (the error handler of a route builder) is suggested once, for all of them, +and not for the route that consumes it. |A hand-off to another route over `seda`, `vm` or `disruptor` |The next route fails, with an expectation that no message is lost between the two routes (see <<_expectations_on_time_and_loss,Expectations on time and loss>>) |An input from outside (not `direct` or `seda`) |The route fails at its entry (what does the sender see: a rollback, a @@ -220,7 +222,8 @@ retry, an error?), or the polls fail for a polling consumer; and the route stall |=== The rules leave the query of the endpoint out: its options do not tell which endpoint it is, and may hold a secret. -`ChaosManager.suggest()` gives the same as JSON (`suggestions`: `route`, `kind`, `recipe`, `title`, `rule`, `why`), and +`ChaosManager.suggest()` gives the same as JSON (`suggestions`: `route`, `kind`, `recipe`, `title`, `rule`, `why`, +`tried`, and `routes` for a dead letter channel that several routes share), and the chaos screen of the Camel TUI lists them (key `w`), where Enter types the rule in to change or add. === What has been tried @@ -243,7 +246,9 @@ $ camel get chaos --report `consumer` (the polls, or the entry of the route), `lifecycle` (stall and lifecycle) and `dead letter channel` (`at=dlq`). A rule has one kind, from its fault and where it acts. * A kind is tried in a route when a rule of that kind fired in it: a rule counts its faults per route, so a rule that -reaches several routes is tried in each. The rules removed or replaced count too, so the report says what the +reaches several routes is tried in each. A rule that expects no message lost between two routes +(`expect=orders->shipping:lost<=0`) also tries `lost` for the first route, as the hand-off suggestion fails the next +route. A dead letter channel that the routes share is tried by a rule of its kind in any of them. The rules removed or replaced count too, so the report says what the experiment tried, not only what runs now. * The outcome is how the exchanges the rules faulted came out (recovered by a redelivery, handled by the error handler, failed), counted per rule, and the outcomes of their expectations. @@ -252,7 +257,8 @@ left, the route has been tried for what chaos can say about it. In JSON the report has `tried`: per `route` its `kinds` (`kind`, `rules`, `fired`, `recovered`, `handled`, `failed`, `expectations`) and `notTried`. `ChaosManager.suggest()` marks a suggestion whose kind was tried in its route already -(`tried`), and the what to try list of the Camel TUI (key `w`) shows both. +(`tried`): `camel cmd chaos-suggest` ends its line with `- tried`, and the what to try list of the Camel TUI (key `w`) +shows both. === From an AI agent (MCP) @@ -335,7 +341,8 @@ way again (the same exception and message at the same step, or the same HTTP sta fix the bug, then prove the fix against the same failure, and keep the rule in a scenario. And `t` on a rule asks its AI panel to make a test of it: a JUnit test with `ChaosAssertions` for a Maven project, or a scenario with an expectation and the `camel run --chaos=<file> --max-seconds=<seconds>` command, whose exit code is the verdict, for a -folder that runs with `camel run`. +folder that runs with `camel run`. The test has the seed, and the rule with `draw=<its id>`, so it fails the same +messages as the rule did (see <<_time_and_the_seed,Time and the seed>>). == Rules @@ -364,6 +371,7 @@ expect: expect=<route id>:<count> (at least that many completed exchanges), or expect=<route id>:meanTime|maxTime<op><time> (such as expect=orders:meanTime<=2s), or the messages lost between two routes expect=<route id>-><route id>:lost<op><count> (such as expect=orders->shipping:lost<=0); with an optional within=<time> +draw: draw=<rule id> draws the random choices as the rule with that id did (draw=r2), see Time and the seed ---- The patterns of the targets are matched as text, with `*` for any characters; they are not regular expressions. The @@ -991,6 +999,12 @@ exchange gets which draw: with several threads (concurrent consumers, parallel p so two runs with the same seed can give different reports. With one thread and the same exchanges in the same order, the same seed gives the same faults. +Each rule draws by a key, with the seed: its id when it is added (`r2`), and its place in a scenario (the second rule +draws as `r2`) or in `camel.chaos.rules`. So a rule added while the application runs, written alone in a scenario or a +test, draws other choices there (it is the first rule, `r1`). `draw=<id>` names the key: `route=shipping error rate=0.2 +draw=r2` with the same seed fails the same messages as rule `r2` did. The test that the Camel TUI makes of a rule (key +`t`) writes it. + == Chaos in a test A finding of a chaos session becomes a regression test: the same rules, the same seed, and assertions on the report. diff --git a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java index 66029be3e45a..28f8ab5c1a79 100644 --- a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java +++ b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosAdvisor.java @@ -34,6 +34,7 @@ import org.apache.camel.model.ProcessorDefinitionHelper; import org.apache.camel.model.RouteDefinition; import org.apache.camel.model.errorhandler.DeadLetterChannelDefinition; import org.apache.camel.support.ScheduledPollConsumer; +import org.apache.camel.util.URISupport; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -85,8 +86,15 @@ final class ChaosAdvisor { * @param rule the rule, ready to add * @param why why it is worth trying for this route * @param rank higher is tried first + * @param routes the routes a rule of its kind tries it in: the route, and the other routes of a dead letter channel + * they share, which is suggested once */ - record Suggestion(String route, Kind kind, String recipe, String title, String rule, String why, int rank) { + record Suggestion(String route, Kind kind, String recipe, String title, String rule, String why, int rank, + List<String> routes) { + + Suggestion(String route, Kind kind, String recipe, String title, String rule, String why, int rank) { + this(route, kind, recipe, title, rule, why, rank, List.of(route)); + } } /** @@ -124,17 +132,28 @@ final class ChaosAdvisor { * What has been tried per route: the kinds of fault of the rules (current and removed) that fired in it, and the * kinds the suggestions offer for it that none tried. The routes in the order of the suggestions, then the others. */ - static List<RouteTried> tried(List<ChaosRule> rules, Map<String, Set<Kind>> suggested, long now) { + static List<RouteTried> tried(List<ChaosRule> rules, Map<String, Map<Kind, List<String>>> suggested, long now) { // route -> kind -> the rules of that kind that fired in the route Map<String, Map<Kind, List<ChaosRule>>> byRoute = new LinkedHashMap<>(); suggested.keySet().forEach(route -> byRoute.put(route, new LinkedHashMap<>())); for (ChaosRule rule : rules) { + boolean fired = false; for (Map.Entry<String, ChaosRule.InRoute> e : rule.inRoute.entrySet()) { if (e.getValue().fired.sum() > 0) { + fired = true; byRoute.computeIfAbsent(e.getKey(), k -> new LinkedHashMap<>()) .computeIfAbsent(kindOf(rule), k -> new ArrayList<>()).add(rule); } } + // a rule that expects no message lost between two routes (orders->shipping:lost<=0) tries what a hand-off + // loses for the first route, wherever its faults are: the hand-off suggestion fails the next route + if (fired && rule.expectation != null && rule.expectation.toRoute != null) { + List<ChaosRule> lost = byRoute.computeIfAbsent(rule.expectation.route, k -> new LinkedHashMap<>()) + .computeIfAbsent(Kind.LOST, k -> new ArrayList<>()); + if (!lost.contains(rule)) { + lost.add(rule); + } + } } List<RouteTried> answer = new ArrayList<>(); for (Map.Entry<String, Map<Kind, List<ChaosRule>>> entry : byRoute.entrySet()) { @@ -152,12 +171,15 @@ final class ChaosAdvisor { if (!ids.contains(id)) { ids.add(id); } - // the outcomes of the exchanges the rule faulted in this route, not in all of them + // the outcomes of the exchanges the rule faulted in this route, not in all of them; a rule that + // tries a hand-off for this route (by its expectation) faulted the next route: all of them ChaosRule.InRoute in = rule.inRoute.get(route); - fired += in.fired.sum(); - recovered += in.effect.count(ChaosRule.Effect.Outcome.RECOVERED); - handled += in.effect.count(ChaosRule.Effect.Outcome.HANDLED); - failed += in.effect.count(ChaosRule.Effect.Outcome.FAILED); + for (ChaosRule.InRoute r : in != null ? List.of(in) : List.copyOf(rule.inRoute.values())) { + fired += r.fired.sum(); + recovered += r.effect.count(ChaosRule.Effect.Outcome.RECOVERED); + handled += r.effect.count(ChaosRule.Effect.Outcome.HANDLED); + failed += r.effect.count(ChaosRule.Effect.Outcome.FAILED); + } if (rule.expectation != null) { expectations.add(rule.expectation.spec() + " " + rule.expectation.read(now).outcome().key); } @@ -165,8 +187,11 @@ final class ChaosAdvisor { tried.add(new KindTried(k.getKey(), ids, fired, recovered, handled, failed, expectations)); } List<Kind> notTried = new ArrayList<>(); + Map<Kind, List<String>> kinds = suggested.getOrDefault(route, Map.of()); for (Kind kind : Kind.values()) { - if (suggested.getOrDefault(route, Set.of()).contains(kind) && !entry.getValue().containsKey(kind)) { + // a kind suggested for several routes (a dead letter channel they share) is tried in any of them + if (kinds.containsKey(kind) && kinds.get(kind).stream() + .noneMatch(r -> byRoute.getOrDefault(r, Map.of()).containsKey(kind))) { notTried.add(kind); } } @@ -177,11 +202,14 @@ final class ChaosAdvisor { return answer; } - /** The kinds of fault the suggestions offer per route, in the order of the routes. */ - static Map<String, Set<Kind>> suggestedKinds(List<Suggestion> suggestions) { - Map<String, Set<Kind>> answer = new LinkedHashMap<>(); + /** + * The kinds of fault the suggestions offer per route, in the order of the routes, each with the routes a rule of + * that kind tries it in. + */ + static Map<String, Map<Kind, List<String>>> suggestedKinds(List<Suggestion> suggestions) { + Map<String, Map<Kind, List<String>>> answer = new LinkedHashMap<>(); for (Suggestion s : suggestions) { - answer.computeIfAbsent(s.route(), k -> new LinkedHashSet<>()).add(s.kind()); + answer.computeIfAbsent(s.route(), k -> new LinkedHashMap<>()).putIfAbsent(s.kind(), s.routes()); } return answer; } @@ -203,10 +231,22 @@ final class ChaosAdvisor { consumers.put(uri(route.getEndpoint()), route.getRouteId()); } } + // the routes of each dead letter channel, which is suggested once, not for the route that consumes it + Map<String, List<String>> deadLetters = new LinkedHashMap<>(); + for (Route route : routes) { + try { + String dead = deadLetterUri(route); + if (dead != null && !route.getRouteId().equals(consumers.get(URISupport.normalizeUri(dead)))) { + deadLetters.computeIfAbsent(dead, k -> new ArrayList<>()).add(route.getRouteId()); + } + } catch (Exception e) { + LOG.debug("No dead letter channel for route {}: {}", route.getRouteId(), e.getMessage(), e); + } + } List<Suggestion> answer = new ArrayList<>(); for (Route route : routes) { try { - suggest(route, recipes, consumers, answer); + suggest(route, recipes, consumers, deadLetters, answer); } catch (RuntimeException e) { // the route changed while it was read (such as a reload): nothing to say for it now LOG.debug("No chaos suggestions for route {}: {}", route.getRouteId(), e.getMessage(), e); @@ -217,7 +257,8 @@ final class ChaosAdvisor { } private static void suggest( - Route route, Map<String, ChaosModel.Recipe> recipes, Map<String, String> consumers, List<Suggestion> answer) { + Route route, Map<String, ChaosModel.Recipe> recipes, Map<String, String> consumers, + Map<String, List<String>> deadLetters, List<Suggestion> answer) { String id = route.getRouteId(); Endpoint from = route.getEndpoint(); boolean breaker = route.getRoute() instanceof RouteDefinition definition @@ -281,14 +322,15 @@ final class ChaosAdvisor { } } - // the dead letter channel of the route - if (route.getErrorHandlerFactory() instanceof DeadLetterChannelDefinition dlc && dlc.getDeadLetterUri() != null) { - String dead = dlc.getDeadLetterUri().indexOf('?') > 0 - ? dlc.getDeadLetterUri().substring(0, dlc.getDeadLetterUri().indexOf('?')) : dlc.getDeadLetterUri(); + // the dead letter channel of the route: once for the routes that share it, with the first of them + String dead = deadLetterUri(route); + List<String> shared = dead != null ? deadLetters.getOrDefault(dead, List.of()) : List.of(); + if (!shared.isEmpty() && id.equals(shared.get(0))) { + String who = shared.size() == 1 ? "route " + id + " sends" : "routes " + names(shared) + " send"; add(answer, recipes, "dlq-unreachable", id, Kind.DEAD_LETTER, Map.of("endpoint", dead), - "route " + id + " sends what fails to the dead letter channel " + dead - + ": when that fails too, is the message lost? Add it with a rule that fails the route", - 78); + who + " what fails to the dead letter channel " + dead + + ": when that fails too, is the message lost? Add it with a rule that fails a route", + 78, shared); } // the input of the route @@ -320,10 +362,33 @@ final class ChaosAdvisor { return query > 0 ? uri.substring(0, query) : uri; } + /** The dead letter channel of the route, without its options, or null when its error handler has none. */ + private static String deadLetterUri(Route route) { + if (route.getErrorHandlerFactory() instanceof DeadLetterChannelDefinition dlc && dlc.getDeadLetterUri() != null) { + String uri = dlc.getDeadLetterUri(); + return uri.indexOf('?') > 0 ? uri.substring(0, uri.indexOf('?')) : uri; + } + return null; + } + + /** The first three routes and how many more: orders, inventory, stock and 2 more. */ + private static String names(List<String> routes) { + if (routes.size() <= 3) { + return String.join(routes.size() == 2 ? " and " : ", ", routes); + } + return String.join(", ", routes.subList(0, 3)) + " and " + (routes.size() - 3) + " more"; + } + /** A suggestion from a known rule, with its placeholders filled in: the given values, else their defaults. */ private static void add( List<Suggestion> answer, Map<String, ChaosModel.Recipe> recipes, String recipeId, String route, Kind kind, Map<String, String> values, String why, int rank) { + add(answer, recipes, recipeId, route, kind, values, why, rank, List.of(route)); + } + + private static void add( + List<Suggestion> answer, Map<String, ChaosModel.Recipe> recipes, String recipeId, String route, Kind kind, + Map<String, String> values, String why, int rank, List<String> routes) { ChaosModel.Recipe recipe = recipes.get(recipeId); if (recipe == null) { return; @@ -337,6 +402,6 @@ final class ChaosAdvisor { } rule = rule.replace("${" + p.name() + "}", value); } - answer.add(new Suggestion(route, kind, recipe.id(), recipe.title(), rule, why, rank)); + answer.add(new Suggestion(route, kind, recipe.id(), recipe.title(), rule, why, rank, routes)); } } diff --git a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java index 5d73cb512e5d..970b2b4621e4 100644 --- a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java +++ b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java @@ -229,8 +229,11 @@ public final class ChaosManager { } b.scenario = scenario.getName(); b.seed = scenario.getSeed(); - // the id the rule gets when the scenario is the first thing loaded, so it draws the same wherever it is - b.drawKey = "r" + (i + 1); + // the id the rule gets when the scenario is the first thing loaded, so it draws the same wherever it is; + // unless draw= names the rule it draws as + if (b.drawKey == null) { + b.drawKey = "r" + (i + 1); + } definitions.add(b); } return definitions; @@ -467,8 +470,9 @@ public final class ChaosManager { * with the known rule it comes from and why, such as an HTTP call without a circuit breaker, a dead letter channel, * or a hand-off to another route over seda. * - * @return {@code suggestions}: a list of {@code route}, {@code kind}, {@code recipe}, {@code title}, {@code rule} - * and {@code why} + * @return {@code suggestions}: a list of {@code route}, {@code kind}, {@code recipe}, {@code title}, {@code rule}, + * {@code why}, {@code tried} when a rule of its kind faulted the route already, and {@code routes} when it + * is for several routes (a dead letter channel they share) */ public Map<String, Object> suggest() { JsonArray list = new JsonArray(); @@ -484,10 +488,14 @@ public final class ChaosManager { } for (ChaosAdvisor.Suggestion s : ChaosAdvisor.suggest(camelContext)) { JsonObject jo = new JsonObject(); - if (tried.contains(s.route() + " " + s.kind().key)) { + // a dead letter channel that routes share is tried by a rule of its kind in any of them + if (s.routes().stream().anyMatch(r -> tried.contains(r + " " + s.kind().key))) { jo.put("tried", true); } jo.put("route", s.route()); + if (s.routes().size() > 1) { + jo.put("routes", new JsonArray(s.routes())); + } jo.put("kind", s.kind().key); if (s.recipe() != null) { jo.put("recipe", s.recipe()); @@ -603,10 +611,10 @@ public final class ChaosManager { private volatile Suggested suggested; /** The kinds the suggestions offer, with when they were read and for how many routes: one value, read as one. */ - private record Suggested(Map<String, Set<ChaosAdvisor.Kind>> kinds, long at, int routes) { + private record Suggested(Map<String, Map<ChaosAdvisor.Kind, List<String>>> kinds, long at, int routes) { } - private Map<String, Set<ChaosAdvisor.Kind>> suggestedKinds(long now) { + private Map<String, Map<ChaosAdvisor.Kind, List<String>>> suggestedKinds(long now) { Suggested answer = suggested; int routeCount = camelContext.getRoutes().size(); if (answer == null || now - answer.at() > SUGGESTED_FOR_MILLIS || routeCount != answer.routes()) { diff --git a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosModel.java b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosModel.java index 576d2287d61b..06785c4907ac 100644 --- a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosModel.java +++ b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosModel.java @@ -345,6 +345,11 @@ final class ChaosModel { "orders:handled<=0"); case "within" -> param("within", "duration", false, "The time of the expectation (by default until the rule ends)", "2m"); + case "draw" -> param("draw", "ruleId", false, + "Draws the random choices of rate= as the rule with this id did with the same seed, so a" + + " scenario or a test fails the same messages as a rule" + + " added while the application ran", + "r2"); default -> throw new IllegalStateException("The chaos model has no description of the option " + key); }); } diff --git a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosRule.java b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosRule.java index 576daa27acbd..14127dd59479 100644 --- a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosRule.java +++ b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosRule.java @@ -306,7 +306,7 @@ final class ChaosRule { fired[i] = new LongAdder(); } // a rule of a scenario draws from the seed of the scenario, by its place in it, so the scenario fails the same - // messages whenever it is loaded, whatever the ids of its rules are then + // messages whenever it is loaded, whatever the ids of its rules are then; draw=<id> draws as that rule did long base = b.seed != null ? b.seed : seed; String key = b.drawKey != null ? b.drawKey : id; this.random = new AtomicLong(mix(base + GOLDEN_GAMMA * mix(key.hashCode()))); @@ -1221,7 +1221,8 @@ final class ChaosRule { /** The definition of a rule, as the parser builds it. */ static final class Builder { String text; - // set by a scenario: its name, its seed (null when it has none), and the key of the draws of the rule + // set by a scenario: its name, its seed (null when it has none), and the key of the draws of the rule (also + // set by draw=) String scenario; Long seed; String drawKey; diff --git a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosRuleParser.java b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosRuleParser.java index 73f83bedf216..728cbad2c7ec 100644 --- a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosRuleParser.java +++ b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosRuleParser.java @@ -57,7 +57,8 @@ final class ChaosRuleParser { // the options of a rule besides the parameters of its fault: all of them for the faults at a hook, and those of a // stall or lifecycle rule, which acts on its route at no hook - static final Set<String> GENERAL_KEYS = Set.of("at", "in", "when", "rate", "until", "from", "expect", "within"); + static final Set<String> GENERAL_KEYS + = Set.of("at", "in", "when", "rate", "until", "from", "expect", "within", "draw"); static final Set<String> LIFECYCLE_KEYS = Set.of("from", "expect", "within"); // the prefix of the response headers of error status=: header.<name>=<value> static final String HEADER_PREFIX = "header."; @@ -229,6 +230,7 @@ final class ChaosRuleParser { + " to match the endpoint"); } from(b, params.get("from"), maxDuration); + draw(b, params.get("draw")); in(b, params.get("in")); if (b.fault == Fault.LIFECYCLE && params.containsKey("when")) { throw new IllegalArgumentException( @@ -892,6 +894,21 @@ final class ChaosRuleParser { b.probability = probability; } + /** + * draw=r2: the random choices (rate=) are drawn as the rule with that id drew them with the same seed, so a + * scenario or a test can fail the same messages as a rule that was added while the application ran. + */ + private static void draw(ChaosRule.Builder b, String draw) { + if (draw == null) { + return; + } + if (!draw.matches("r[1-9][0-9]{0,8}")) { + throw new IllegalArgumentException( + "draw=" + draw + " is not the id of a rule: draw=<id>, such as draw=r2, draws as rule r2 did"); + } + b.drawKey = draw; + } + private static void until(ChaosRule.Builder b, String until, long maxDuration) { if (until == null) { return; diff --git a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorDeadLetterTest.java b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorDeadLetterTest.java new file mode 100644 index 000000000000..10269380c7c2 --- /dev/null +++ b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosAdvisorDeadLetterTest.java @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.component.chaos; + +import java.util.List; + +import org.apache.camel.builder.RouteBuilder; +import org.apache.camel.util.json.JsonArray; +import org.apache.camel.util.json.JsonObject; +import org.junit.jupiter.api.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * A dead letter channel that the routes share (the error handler of the route builder) is one thing to try: it is + * suggested once, not for each route, and never for the route that consumes it; a rule of its kind in any of its routes + * tries it (CAMEL-26732). + */ +class ChaosAdvisorDeadLetterTest extends ChaosTestSupport { + + @Override + protected RouteBuilder createRouteBuilder() { + return new RouteBuilder() { + @Override + public void configure() { + errorHandler(deadLetterChannel("seda:dlq?size=100")); + + from("direct:orders").routeId("orders").to("mock:stock"); + from("direct:billing").routeId("billing").throwException(new IllegalStateException("no money")); + from("direct:audit").routeId("audit").to("mock:audit"); + from("seda:dlq?pollTimeout=10").routeId("dlq").to("mock:dead"); + } + }; + } + + private List<ChaosAdvisor.Suggestion> deadLetter() { + return ChaosAdvisor.suggest(context).stream().filter(s -> s.kind() == ChaosAdvisor.Kind.DEAD_LETTER).toList(); + } + + @Test + void theSharedDeadLetterChannelIsSuggestedOnce() { + List<ChaosAdvisor.Suggestion> dead = deadLetter(); + assertThat(dead).hasSize(1); + ChaosAdvisor.Suggestion s = dead.get(0); + assertThat(s.route()).isEqualTo("orders"); + // the route that consumes the dead letter channel is not one of its routes + assertThat(s.routes()).containsExactly("orders", "billing", "audit"); + assertThat(s.rule()).isEqualTo("endpoint=seda:dlq error at=dlq"); + assertThat(s.why()).startsWith("routes orders, billing, audit send what fails to the dead letter channel seda:dlq"); + } + + @Test + void aRuleOfItsKindInAnyOfItsRoutesTriesIt() { + chaos().addRule("endpoint=seda:dlq error at=dlq until=10m"); + // billing fails, and the send to the dead letter channel fails too + template.send("direct:billing", e -> e.getMessage().setBody("invoice")); + + JsonObject suggestion = null; + for (Object o : (JsonArray) chaos().suggest().get("suggestions")) { + JsonObject jo = (JsonObject) o; + if ("dead letter channel".equals(jo.getString("kind"))) { + suggestion = jo; + } + } + assertThat(suggestion).isNotNull(); + assertThat(suggestion.getString("route")).isEqualTo("orders"); + assertThat((JsonArray) suggestion.get("routes")).containsExactly("orders", "billing", "audit"); + assertThat(suggestion.getBooleanOrDefault("tried", false)).isTrue(); + + // what was tried: in billing, and orders has nothing of that kind left to try + for (Object o : (JsonArray) chaos().report().toJson().get("tried")) { + JsonObject route = (JsonObject) o; + if ("orders".equals(route.getString("route"))) { + assertThat((JsonArray) route.get("notTried")).doesNotContain("dead letter channel"); + } + } + } +} diff --git a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosModelTest.java b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosModelTest.java index 142d027fc02d..ed553b29af1c 100644 --- a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosModelTest.java +++ b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosModelTest.java @@ -100,7 +100,7 @@ class ChaosModelTest { assertThat(strings(byName("faults", "duplicate").get("hooks"))).containsExactly("send"); assertThat(strings(byName("faults", "duplicate").get("targets"))).containsExactly("component", "endpoint"); assertThat(paramNames(byName("faults", "duplicate"))).isEmpty(); - assertThat(names("options")).containsExactly("at", "expect", "from", "in", "rate", "until", "when", + assertThat(names("options")).containsExactly("at", "draw", "expect", "from", "in", "rate", "until", "when", "within"); // the TUI form picks the route of in= assertThat(byName("options", "in").getString("pick")).isEqualTo("routes"); diff --git a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosScenarioTest.java b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosScenarioTest.java index 4fa4cedee041..8e4d98490579 100644 --- a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosScenarioTest.java +++ b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosScenarioTest.java @@ -124,6 +124,25 @@ class ChaosScenarioTest extends ChaosTestSupport { assertThat(failures()).isEqualTo(first); } + @Test + void drawReplaysARuleThatWasAddedWhileRunning() { + // rule r2 was added while the application ran (seed 42, camel.chaos.seed) + assertThat(chaos().addRule("endpoint=mock:nowhere error")).isEqualTo("r1"); + assertThat(chaos().addRule("endpoint=mock:stock error rate=0.5")).isEqualTo("r2"); + List<Boolean> live = failures(); + assertThat(live).contains(true, false); + + // a scenario with the rule alone: it is r1 there, so it draws other choices, unless draw= says it is r2 + chaos().loadScenario("other.chaos", "seed=42\nendpoint=mock:stock error rate=0.5\n"); + assertThat(failures()).isNotEqualTo(live); + chaos().loadScenario("r2-test.chaos", "# rule r2 again\nseed=42\nendpoint=mock:stock error rate=0.5 draw=r2\n"); + assertThat(failures()).isEqualTo(live); + + assertThatThrownBy(() -> chaos().addRule("endpoint=mock:stock error rate=0.5 draw=orders")) + .isInstanceOf(IllegalArgumentException.class) + .hasMessageContaining("draw=orders is not the id of a rule"); + } + @Test void withoutASeedTheRulesDrawFromTheSeedOfTheApplication() { chaos().loadScenario("outage.chaos", OUTAGE); diff --git a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosTriedTest.java b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosTriedTest.java index 54918e49cc1a..92bd4826ff56 100644 --- a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosTriedTest.java +++ b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosTriedTest.java @@ -16,6 +16,8 @@ */ package org.apache.camel.component.chaos; +import java.util.concurrent.TimeUnit; + import org.apache.camel.BindToRegistry; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.util.json.JsonArray; @@ -23,6 +25,7 @@ import org.apache.camel.util.json.JsonObject; import org.junit.jupiter.api.Test; import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; /** * CAMEL-26728: what has been tried per route, how it went, and what the suggestions offer that no rule tried yet. @@ -129,6 +132,38 @@ class ChaosTriedTest extends ChaosTestSupport { assertThat(errorsTried).isFalse(); } + @Test + void theHandOffSuggestionIsTriedByARuleThatExpectsNothingLost() { + // the suggestion for route orders fails the next route, and expects nothing lost between them + chaos().addRule("route=shipping error until=1 expect=orders->shipping:lost<=0 within=10s"); + template.sendBody("direct:orders", "order"); + // shipping takes the order from seda on its own thread + await().atMost(10, TimeUnit.SECONDS).until(this::firedInShipping); + + JsonObject orders = route("orders"); + assertThat(((JsonArray) orders.get("kinds")).stream().map(k -> ((JsonObject) k).getString("kind"))) + .contains("lost"); + assertThat((JsonArray) orders.get("notTried")).doesNotContain("lost"); + boolean lostTried = false; + for (Object o : (JsonArray) chaos().suggest().get("suggestions")) { + JsonObject s = (JsonObject) o; + if ("orders".equals(s.getString("route")) && "lost".equals(s.getString("kind"))) { + lostTried = s.getBooleanOrDefault("tried", false); + } + } + assertThat(lostTried).isTrue(); + } + + private boolean firedInShipping() { + for (Object o : (JsonArray) chaos().report().toJson().get("tried")) { + if ("shipping".equals(((JsonObject) o).getString("route")) + && !((JsonArray) ((JsonObject) o).get("kinds")).isEmpty()) { + return true; + } + } + return false; + } + @Test void theKindOfARule() { assertThat(kind("endpoint=x error")).isEqualTo(ChaosAdvisor.Kind.ERRORS); diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosSuggestAction.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosSuggestAction.java index f8378239071a..22174ed7c2cf 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosSuggestAction.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosSuggestAction.java @@ -17,6 +17,7 @@ package org.apache.camel.dsl.jbang.core.commands.action; import java.util.List; +import java.util.stream.Collectors; import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain; import org.apache.camel.dsl.jbang.core.common.ProcessFinder.CamelProcess; @@ -69,8 +70,17 @@ public class CamelChaosSuggestAction extends CamelChaosAction { int n = 0; for (Object o : suggestions) { JsonObject s = (JsonObject) o; - sb.append("\n\n").append(++n).append(". route ").append(s.getString("route")).append(": ") + // a dead letter channel that routes share is suggested once, for all of them + List<Object> routes = s.getCollection("routes"); + String where = routes != null && routes.size() > 1 + ? "routes " + routes.stream().map(String::valueOf).collect(Collectors.joining(", ")) + : "route " + s.getString("route"); + sb.append("\n\n").append(++n).append(". ").append(where).append(": ") .append(s.getString("title")).append(" (").append(s.getString("kind")).append(')'); + if (s.getBooleanOrDefault("tried", false)) { + // a rule of this kind faulted the route already (camel get chaos --report shows how it went) + sb.append(" - tried"); + } sb.append("\n ").append(s.getString("why")); sb.append("\n camel cmd chaos-add --rule='").append(s.getString("rule")).append('\''); } diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosActionTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosActionTest.java index 612dbeadb814..5d1554d1e4df 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosActionTest.java +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/action/CamelChaosActionTest.java @@ -118,6 +118,25 @@ class CamelChaosActionTest extends ActionCommandTestSupport { "camel cmd chaos-add --rule='endpoint=http://stock/check error status=503 rate=0.25 until=10m'"); } + @Test + void suggestSaysWhatWasTriedAndTheRoutesOfASharedDeadLetterChannel() throws Exception { + writeStatusFile(TEST_PID, "orders"); + JsonObject tried = new JsonObject( + Map.of("route", "orders", "kind", "errors", "title", "Returns an error (HTTP status)", "tried", true, + "rule", "endpoint=http://stock/check error status=503", "why", "an HTTP call")); + JsonObject shared = new JsonObject( + Map.of("route", "orders", "routes", new JsonArray(List.of("orders", "billing")), "kind", + "dead letter channel", "title", "The dead letter channel is unreachable", + "rule", "endpoint=seda:dlq error at=dlq", "why", "routes orders and billing send what fails")); + JsonObject answer = new JsonObject(Map.of("suggestions", new JsonArray(List.of(tried, shared)))); + + assertThat(callWithResponse(command(new CamelChaosSuggestAction(main()), "orders"), answer)).isZero(); + + assertThat(printer.getLines()).contains( + "1. route orders: Returns an error (HTTP status) (errors) - tried", + "2. routes orders, billing: The dead letter channel is unreachable (dead letter channel)"); + } + @Test void suggestSaysWhenThereIsNothingToTry() throws Exception { writeStatusFile(TEST_PID, "orders"); diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/ChaosAiPrompt.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/ChaosAiPrompt.java index 0c434d1fe15a..c1981efc810e 100644 --- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/ChaosAiPrompt.java +++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/ChaosAiPrompt.java @@ -58,6 +58,9 @@ final class ChaosAiPrompt { */ static String ofTest(IntegrationInfo info, ChaosInfo.Rule rule) { long seed = info.chaos != null ? info.chaos.seed : 0; + // a rule draws its random choices by a key: its id here, its place in a scenario or camel.chaos.rules there, so + // the test names it with draw= to fail the same messages (CAMEL-26732) + String replay = replayRule(rule); StringBuilder sb = new StringBuilder(); sb.append("Make a test of chaos rule ").append(rule.id).append(" of the running integration ").append(name(info)) .append(", so this finding runs again with the same faults. The rule (camel-chaos, injected on purpose):\n") @@ -73,7 +76,9 @@ final class ChaosAiPrompt { sb.append("The integration is a Maven project. Read its routes first, then write a JUnit test in src/test/java") .append(" that runs them with chaos switched on: add org.apache.camel:camel-chaos as a test dependency") .append(" when the pom.xml does not have it, and set the properties camel.chaos.enabled=true,") - .append(" camel.chaos.seed=").append(seed).append(" and camel.chaos.rules=<the rule> (CamelTestSupport:") + .append(" camel.chaos.seed=").append(seed).append(" and camel.chaos.rules=").append(replay) + .append(" (draw=").append(rule.id).append(" makes it fail the same messages as rule ").append(rule.id) + .append(" did; CamelTestSupport:") .append(" override configureContext and call config.withUseOverridePropertiesWithPropertiesComponent(props);") .append(" a camel-test-main-junit6 test sets the same properties in its configuration). ") .append(expectation) @@ -85,8 +90,10 @@ final class ChaosAiPrompt { } else { sb.append("The integration runs with camel run, without a Maven project, so the test is a scenario with an") .append(" expectation, and a run whose exit code is the verdict. Write the scenario chaos/") - .append(rule.id).append("-test.chaos: a comment line that says what it tests, seed=").append(seed) - .append(", and the rule. ").append(expectation) + .append(rule.id).append("-test.chaos: a comment line that says in words what it tests (not the rule") + .append(" again), seed=").append(seed).append(", and the rule as the line ").append(replay) + .append(" (draw=").append(rule.id).append(" makes it fail the same messages as rule ").append(rule.id) + .append(" did). ").append(expectation) .append(" Then give the command that runs it, with --max-seconds longer than the within= of the") .append(" expectation: camel run * --chaos=chaos/").append(rule.id) .append("-test.chaos --max-seconds=<seconds> --chaos-report=target/chaos-report.json. It ends with exit") @@ -96,6 +103,12 @@ final class ChaosAiPrompt { return sb.toString(); } + /** The rule as a test writes it: with draw=<its id>, unless it has a draw= already. */ + static String replayRule(ChaosInfo.Rule rule) { + String text = rule.text.strip(); + return text.contains(" draw=") ? text : text + " draw=" + rule.id; + } + /** Whether the folder of the integration is a Maven project, which can have a JUnit test. */ static boolean isMavenProject(IntegrationInfo info) { return info.directory != null && Files.isRegularFile(Path.of(info.directory, "pom.xml")); diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/ChaosAiPromptTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/ChaosAiPromptTest.java index 9e5fc0e497a3..824dd02fa8d1 100644 --- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/ChaosAiPromptTest.java +++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/ChaosAiPromptTest.java @@ -274,6 +274,9 @@ class ChaosAiPromptTest { .contains("The seed of its random draws: 42.") .contains("Expectation: orders:handled<=0, unmet") .contains("a JUnit test in src/test/java").contains("camel.chaos.seed=42") + // the rule draws as it did here, as r1 + .contains("camel.chaos.rules=endpoint=http://localhost:8765/stock* error status=503 rate=0.2" + + " expect=orders:handled<=0 draw=r1 (draw=r1 makes it fail the same messages as rule r1 did") .contains("assertExpectationMet(context, \"r1\", Duration.ofSeconds(10))") .contains("The rule has an expectation: keep it.") .endsWith("Write the file with camel_write_file, which asks me first, and change no route."); @@ -288,11 +291,23 @@ class ChaosAiPromptTest { String question = ChaosAiPrompt.ofTest(info, rule); assertThat(question).contains("runs with camel run, without a Maven project") .contains("Write the scenario chaos/r1-test.chaos").contains("seed=42") + .contains("says in words what it tests (not the rule again)") + .contains("the rule as the line endpoint=http://localhost:8765/stock* error status=503 rate=0.2" + + " expect=orders:handled<=0 draw=r1") .contains("The rule has no expectation (expect=): add one") .contains("camel run * --chaos=chaos/r1-test.chaos --max-seconds=<seconds>") .doesNotContain("JUnit"); } + @Test + void aRuleThatHasADrawAlreadyKeepsIt() { + ChaosInfo.Rule rule = info.chaos.rules.get(0); + rule.text = "endpoint=mock:stock error rate=0.5 draw=r7"; + assertThat(ChaosAiPrompt.replayRule(rule)).isEqualTo("endpoint=mock:stock error rate=0.5 draw=r7"); + rule.text = "endpoint=mock:stock error rate=0.5 "; + assertThat(ChaosAiPrompt.replayRule(rule)).isEqualTo("endpoint=mock:stock error rate=0.5 draw=r1"); + } + @Test void tOnTheChaosScreenAsksTheAiToMakeATest() { ChaosTab tab = new ChaosTab(ctx);
