2009/6/3 Adam Murdoch <[email protected]>
>
>
> Tom Eyckmans wrote:
>
>> Hi all,
>>
>> just started a testing page
>> http://docs.codehaus.org/display/GRADLE/Testing with my first thought on
>> how to provide control over multiple VM forks to execute tests.
>>
>>
> Your proposal looks like it is trying to solve 3 problems:
>
> 1. Defining a group of tests which need to run as a batch, with some
> settings about how to run those tests
>
> 2. Defining a workflow for these test groups, so that some are executed
> sequentially and some are executed in parallel
>
> 3. Defining composite test groups, each made up of other test groups
>
> I think problem 1 is the only thing specific to testing. Problems 2 and 3
> really apply to anything that Gradle does, and so would be better done as
> general-purpose facilities, rather than something fundamental to a test
> suite.
I agree I was also thinking in that direction.
>
>
> For example, given that a task ends up being defined for each test suite,
> problem 2 could be solved by some facility to mark up which tasks should run
> in parallel. Or, a better option would be to use a constraint-based model
> and let Gradle figure out which tasks to run in parallel or sequentially,
> similar to what we do with task dependencies.
That would be cool but with the test task this really depends on whether the
tests are written in a way that they can be executed in parallel. For
example our integration tests, you could run these in parallel per
integration test project, there is no way for Gradle to detect in some way
that these can be executed in parallel. I think a constraint based model is
really nice idea I'm not sure the serial / parallel blocks are sufficient
for a general purpose facility. Also when a series of batch tests are
executed, multiple test tasks are executed in serially or in parallel the
test result xml files would be generated in the Gradle VM, how would we
model this with serial / parallel in a generic way without troubling users
with the inner workings of how test execution works? We might also allow
naming the batch test groups so specific batch test groups are easily found
in the test report.
>
>
> We already have examples of problem 3: the 'libs' task builds all the JARs
> and WARs in the project, 'buildArtifacts' task builds the artifacts
> published by the project. We have solved these so far using task
> dependencies. We could do the same with grouping the execution of test
> suites. Or we could introduce some kind of domain object collection concept.
> I'm not so sure we need anything particularly comprehensive here for
> testing. In my experience, different test suites execute at very different
> points in a project's lifecycle (and usually in different builds), so
> there's little practical benefit in actually grouping them together.
>
> So, I think you could trim the test suite proposal down to:
>
> main(TestSuite) {
> exclude '**/integtest/**'
> useJUnit()
> options.forkOptions.forkMode = ForkMode.Once
> }
>
> integTests(TestSuite) {
> include '**/integtest/**'
> useJUnit()
> }
>
> On the syntax: We're defining a thing with a type and a name here, just
> like tasks, or repositories (or source dirs etc). We have a syntax for
> defining these things, and we should use the same syntax for defining test
> suites as well.
I agree.
> This could mean changing how we define tasks or repositories, or it could
> mean using the existing syntax for defining test suites.
I'll use the existing syntax.
>
>
> Using the existing syntax, this means declaring test suites like
>
> testsuites {
> main {
> ... some options ...
> }
> }
>
> or
>
> testsuite main {
> ... some options ...
> }
>
>
> I think the test framework should be fundamental to the type of the test
> suite:
I agree
>
>
> testsuites {
> main(type: JUnit) { ... }
> another(type: TestNg) { ... }
> }
>
> Where JUnit would be the default. Amongst other things, this typing means
> we can flatten the various options objects into the appropriate TestSuite
> subclass.
That is a really nice :)
>
>
>
> Adam
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
