Tom Eyckmans wrote:
Hi all,
just started a testing page
http://docs.codehaus.org/display/GRADLE/Testing with my first thought
on how to provide control over multiple VM forks to execute tests.
Your proposal looks like it is trying to solve 3 problems:
1. Defining a group of tests which need to run as a batch, with some
settings about how to run those tests
2. Defining a workflow for these test groups, so that some are executed
sequentially and some are executed in parallel
3. Defining composite test groups, each made up of other test groups
I think problem 1 is the only thing specific to testing. Problems 2 and
3 really apply to anything that Gradle does, and so would be better done
as general-purpose facilities, rather than something fundamental to a
test suite.
For example, given that a task ends up being defined for each test
suite, problem 2 could be solved by some facility to mark up which tasks
should run in parallel. Or, a better option would be to use a
constraint-based model and let Gradle figure out which tasks to run in
parallel or sequentially, similar to what we do with task dependencies.
We already have examples of problem 3: the 'libs' task builds all the
JARs and WARs in the project, 'buildArtifacts' task builds the artifacts
published by the project. We have solved these so far using task
dependencies. We could do the same with grouping the execution of test
suites. Or we could introduce some kind of domain object collection
concept. I'm not so sure we need anything particularly comprehensive
here for testing. In my experience, different test suites execute at
very different points in a project's lifecycle (and usually in different
builds), so there's little practical benefit in actually grouping them
together.
So, I think you could trim the test suite proposal down to:
main(TestSuite) {
exclude '**/integtest/**'
useJUnit()
options.forkOptions.forkMode = ForkMode.Once
}
integTests(TestSuite) {
include '**/integtest/**'
useJUnit()
}
On the syntax: We're defining a thing with a type and a name here, just
like tasks, or repositories (or source dirs etc). We have a syntax for
defining these things, and we should use the same syntax for defining
test suites as well. This could mean changing how we define tasks or
repositories, or it could mean using the existing syntax for defining
test suites.
Using the existing syntax, this means declaring test suites like
testsuites {
main {
... some options ...
}
}
or
testsuite main {
... some options ...
}
I think the test framework should be fundamental to the type of the test
suite:
testsuites {
main(type: JUnit) { ... }
another(type: TestNg) { ... }
}
Where JUnit would be the default. Amongst other things, this typing
means we can flatten the various options objects into the appropriate
TestSuite subclass.
Adam
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
