Hi

I made a start on a couple of design docs.

This one is specifically about cucumber
https://github.com/3ll3d00d/gradle/blob/cuke-jvm/design-docs/support-cucumber-jvm.md

This one is about opening up the Test task so that you don't have to have
someone commit to gradle core tasks when this sort of requirement comes up
https://github.com/3ll3d00d/gradle/blob/cuke-jvm/design-docs/extensible-test-frameworks.md

In lieu of having an open api for adding test frameworks to the gradle test
task (as per the 2nd doc), I am inclined to suggest that the vast majority
of the benefit will come from improving gradle's test reporting rather than
baking cucumber-jvm into gradle core.

My rationale is that the current gradle test reporting does not appear to
adequately describe what one can do with junit let alone cucumber. For
example, a suite is represented as the sum of the tests found within the
classes in the suite as opposed to a 1-n from suite to class to test.

One option is to extend the reporting to allow an arbitrary depth to the
tree (though probably 3-4 is as deep as it gets) as opposed to the current
2 level structure & to include some other, ideally test centric, naming
convention to replace package/class/method.

I made an example here - https://github.com/3ll3d00d/cuke-example

(though the cucumber junit integration is still a bit flaky, on my machine
it just started NPEing for, as yet, unknown reasons)

Compare the output for the following runs

1) Run gradlew junitTest
2) Run gradlew cukeTest
3) Run MySuite in intellij
4) Run CukeRunner in intellij

Notice how 3 & 4 faithfully produce the appropriate tree structure whereas
1 & 2 do not.

Another use for such an extension would be TestNG DataProviders, i.e.
instead of printing out the same test name n times then print the test as a
node in the tree and the n times as the children.

NB: I said earlier "the vast majority of the benefit", the remaining
advantages [to making cucumber a known test framework] come from the static
nature of junit runners which make it impossible to fiddle what you run on
the fly. For example, when you need to make a quick change in a CI env or
in your build, instead of a cli param you have to change the junit runner
class and rebuild. In time, I suspect this would lead to a big bloat in the
no of runner classes.

Cheers
Matt

Reply via email to