winglam commented on a change in pull request #309:
URL: https://github.com/apache/maven-surefire/pull/309#discussion_r465161332
##########
File path:
maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java
##########
@@ -310,6 +310,22 @@
@Parameter( property = "failsafe.runOrder", defaultValue = "filesystem" )
private String runOrder;
+ /**
+ * Sets the random seed that will be used to order the tests if {@code
failsafe.runOrder} is set to {@code random}.
+ * <br>
+ * <br>
+ * If no seeds are set and {@code failsafe.runOrder} is set to {@code
random}, then the seed used will be
+ * outputted (search for "To reproduce ordering use flag -Dfailsafe.seed").
+ * <br>
+ * <br>
+ * To deterministically reproduce any random test order that was run
before, simply set the seed to
+ * be the same value.
+ *
+ * @since 3.0.0-M6
+ */
+ @Parameter( property = "failsafe.seed" )
+ private Long randomSeed;
Review comment:
> capture the test order and replay it explicitly (a runOrder=fromFile
and runOrderFile=/path would be more relevant there)
I actually have some changes (in a separate local branch) that would
accomplish this as well and plan to submit it after this PR is merged. I could
still use some help/advice on how to capture the test order (with or without
running tests). The goal of my changes would be to empower runOrder to be more
customizable and deterministic with seeds, passing in a file, controlling not
just test class order but also test method order, etc.
I have the changes for merging `failsafe.seed` and `surefire.seed` with the
`runOrder` parameter ready, but will leave those unpushed for now as I am fine
with merging them or keep them separate (as they are now or moving it, etc.).
##########
File path:
maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java
##########
@@ -310,6 +310,22 @@
@Parameter( property = "failsafe.runOrder", defaultValue = "filesystem" )
private String runOrder;
+ /**
+ * Sets the random seed that will be used to order the tests if {@code
failsafe.runOrder} is set to {@code random}.
+ * <br>
+ * <br>
+ * If no seeds are set and {@code failsafe.runOrder} is set to {@code
random}, then the seed used will be
+ * outputted (search for "To reproduce ordering use flag -Dfailsafe.seed").
+ * <br>
+ * <br>
+ * To deterministically reproduce any random test order that was run
before, simply set the seed to
+ * be the same value.
+ *
+ * @since 3.0.0-M6
+ */
+ @Parameter( property = "failsafe.seed" )
+ private Long randomSeed;
Review comment:
> capture the test order and replay it explicitly (a runOrder=fromFile
and runOrderFile=/path would be more relevant there)
I actually have some changes (in a separate local branch) that would
accomplish this as well and plan to submit it after this PR is merged. I could
still use some help/advice on how to capture the test order (with or without
running tests). The goal of my changes would be to empower runOrder to be more
customizable and deterministic with seeds, passing in a file, controlling not
just test class order but also test method order, etc.
I have the changes for merging `failsafe.seed` and `surefire.seed` with the
`runOrder` parameter ready, but will leave those unpushed for now as I am fine
with merging them or keep them separate (as they are now or moving it, etc.).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
