[
https://jira.codehaus.org/browse/SUREFIRE-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=352133#comment-352133
]
Tibor Digana edited comment on SUREFIRE-1043 at 9/1/14 6:55 PM:
----------------------------------------------------------------
@Tomasz Halama
The group of tests sounds to me like Maven <execution> section in plugin
configuration. Each plugin execution may have same phase but different
configuration. Executions run in a sequence.
Parallel executions are possible if you assign tests in a suite. Thus every
Suite may have own tests and their order is guaranted.
If you trigger surefire plugin with parallel methods, such of hierarchy of
suites and classes will guarantee ordering of classes, and the methods of
particular test class will be executed in parallel.
You can execute suites and methods in parallel in plugin configuration if you
set parallel=suitesAndMethods. This means the groups of test classes (=Suites)
are run in parallel, the tests classes in sequence and methods in parallel.
Would this help?
was (Author: tibor17):
@Tomasz Halama
The group of tests sounds to me like Maven <execution> section in plugin
configuration. Each plugin execution may have same phase but different
configuration. Executions run in a sequence.
Parallel executions are possible if you assign tests in a suite. Thus every
Suite may have own tests and they order is guaranted.
If you trigger surefire plugin with parallel methods, such of hierarchy of
suites and classes will guarantee ordering of classes, and the methods of
particular test class will be executed in parallel.
You can execute suites and methods in parallel in plugin configuration if you
set parallel=suitesAndMethods. This means the groups of test classes (=Suites)
are run in parallel, the tests classes in sequence and methods in parallel.
Would this help?
> Please add possibility to specify tests groups order for sequential run and
> tests groups content for parallel run.
> ------------------------------------------------------------------------------------------------------------------
>
> Key: SUREFIRE-1043
> URL: https://jira.codehaus.org/browse/SUREFIRE-1043
> Project: Maven Surefire
> Issue Type: Improvement
> Affects Versions: 2.16
> Reporter: Tomasz Halama
>
> What we need is as following:
> For sequential run:
> We would like to have some means to specify run order of whole groups of
> tests, during one invocation of plugin. Content of the group should be
> determined by ant style expressions. Inside each such group, tests should be
> run in order, specified by 'runOrder' parameter.
> Example:
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-failsafe-plugin</artifactId>
> <configuration>
> <groupsOrder>
> <group>**/*ST</group>
> <group>com/something/SomeTest,com/something/SomeTest2</group>
> <group>**/*IT,**/IT*</group>
> <group>com/something/SomeTest3</group>
> </groupsOrder>
> <runOrder>alphabetical</runOrder>
> </configuration>
> </plugin>
> In above example run order of tests should be as following:
> 1. all tests, which have "ST" suffix, should be run in alphabetical order
> 2. com/something/SomeTest and com/something/SomeTest2 should be run in
> alphabetical order
> 3. all tests, which have "IT" suffix or prefix, should be run in alphabetical
> order
> 4. com/something/SomeTest3 should be run
> Instead of adding some new parameter (like above 'groupsOrder' section), I
> think 'includes' can be used for this purpose (each entry would determine a
> group), with some additional boolean value, i.e. 'groupsByIncludes'
> (indicating, that each 'include' entry should be considered as a separate
> group):
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-failsafe-plugin</artifactId>
> <configuration>
> <includes>
> <include>**/*ST.java</include>
>
> <include>com/something/SomeTest.java,com/something/SomeTest2.java</include>
> <include>**/*IT.java,**/IT*.java</include>
> <include>com/something/SomeTest3.java</include>
> </includes>
> <runOrder>alphabetical</runOrder>
> <groupsByIncludes>true<groupsByIncludes>
> </configuration>
> </plugin>
> In our case we cannot simply divide one invocation of plugin into several
> ones (which would solve this problem), because setuping tests context takes
> too much time.
> This feature will also give possibility to run tests by some specific order
> (when each 'group'/'include' would consist of only one entry). Possibility to
> set specific order could be very handy, when i.e. you detected, that some of
> your tests are not independent and you want to reproduce the problem.
> For parallel run:
> We would like to have some means to specify tests groups for parallel run. By
> this I mean: tests in each group will be run in parallel, but the groups
> itself will be run sequentially (so only tests from the same group can be
> executed at the same time)
> There should be also some annotation, which can be used to marked tests,
> which cannot be run in parallel at all.
> If 'includes' contains such tests - they should be run sequentially at the
> end.
> To achieve all of this, new (i.e. groupsOrder) or 'includes' parameter can be
> used here, in the same way as for sequential run, except 'parallel' attribute
> should be set to new value: i.e. parallelGroup.
> I think, that logging information, when each test (each test method) starts
> and stops, would be also a good idea (we would know exact tests, which are
> run at the same time).
--
This message was sent by Atlassian JIRA
(v6.1.6#6162)
