On 11/06/10 4:23 AM, Peter Niederwieser wrote:
Adam Murdoch-3 wrote:
Would this allow me to freely reorder test classes without losing any
Gradle-provided testing features like running tests in parallel?
Yes.
I don't see how this could be achieved with the current TestClassProcessor.
Maybe you can explain.
They would be chained, so that you're providing a decorator over the top
of the build-in implementation. The hook might look something like:
test.withTestProcessor { TestClassProcessor builtIn -> new
MyProcessorImpl(builtIn) }
Adam Murdoch-3 wrote:
How about just letting me provide an ordered list of classes to the test
task? Or a comparator?
This is a possibility. How would the list of classes work? Would you
take over the test detection?
List of classes: yes (Spock already has test detection, so it would be easy
for me to do)
Comparator: no (that's why I suggested it)
Adam Murdoch-3 wrote:
I wanted something general purpose, which could do whatever filtering
and reordering (or duplication or failure thresholds or whatever) it
wants. And then, later, we would add convenience methods which take,
say, a comparator.
In terms of the public extension point, I had thought the other way around:
First provide a hook which is simple to design, simple to use, and easy to
commit yourself to (my hope was that I could get this one "immediately"). At
a later point add a more sophisticated hook. But I'm open to other
approaches.
My assumption is that this hook is something that only a handful of
plugins are going to need to use. And so, in this case, I want minimal
impact on the Test task's API to keep the noise down. I don't want a
bunch of different ways of achieving the same thing.
I will add something soon.
I've written a Spock extension called OptimizeRunOrderExtension. It collects
metrics such as the number of times a test has succeeded in a row and the
test's duration. Collecting the metrics works in any environment and thus I
don't need help from Gradle's side for this.
The metrics are then used for reordering classes and their methods: elements
that have failed more recently or have run faster come first. However, the
JUnit @RunWith SPI only lets me reorder the methods within a class - it
doesn't give me control over the order in which classes are run. So for
class reordering I need a hook provided by the IDE or build system.
Now that I think of it, optimizing test run order (with the aim of providing
faster feedback to the user) might also make an attractive feature for
Gradle's test framework.
Absolutely. For me personally, I think I would get most value out of
this feature if I could use it with our functional tests, which are
plain JUnit tests and don't use Spock (though maybe they should). Our
unit tests are fast enough, particularly with parallel testing and
incremental build, that I don't really care about whether they are
reordered or not.
--
Adam Murdoch
Gradle Developer
http://www.gradle.org
CTO, Gradle Inc. - Gradle Training, Support, Consulting
http://www.gradle.biz
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
