Owolabi Legunsen created SUREFIRE-1399:
------------------------------------------
Summary: More stable sort when using `-Dsurefire.runOrder`
Key: SUREFIRE-1399
URL: https://issues.apache.org/jira/browse/SUREFIRE-1399
Project: Maven Surefire
Issue Type: New Feature
Components: Maven Surefire Plugin
Affects Versions: 2.20
Reporter: Owolabi Legunsen
Attachments: order.patch
The intent here is to check if a proposed feature that I need would
also be useful for others. The proposed feature (1) maintains a stable
sort when running tests with {{-Dsurefire.runOrder=FAILEDFIRST}}, by
using the position in a test-order file to break ties, and (2) allows
users to provide their own test-order file.
Would this feature be useful for others? Using the attached *draft*
patch, users can optionally specify a test-order file via
{{-Dsurefire.runOrderFile}} (which should eventually be properly added
as {{@Parameter}}), and the comparators that determine test order have
been changed to break ties during sorting by using the position of
tests in the test-order file (new comparators may need to be
added). Would it be worthwhile to further improve this draft patch?
To test this feature, clone Surefire, checkout SHA {{f1f4718e}}, apply the
patch, then run tests in the order specified in a file, e.g.:
{quote}
$ git clone https://github.com/apache/maven.git
$ cd maven
$ sed -i
's;maven-surefire-plugin</artifactId>;maven-surefire-plugin</artifactId>\n<version>2.21-SNAPSHOT</version>;'
pom.xml
$ cd maven-core
$ find -name "*Test.java" | grep -v AppTest | sed
"sX^./src/test/java/\(.*\).java\$X0,0,dummyMethod\(\1\)X" | tr / . | shuf
>.test-order
$ mvn test -Dsurefire.runOrder=FAILEDFIRST -Dsurefire.runOrderFile=.test-order
-Drat.skip
{quote}
Would it be better to keep changing the code as shown in this draft
patch or to add a new prioritization order which selects from a file?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
