Classification: Public >> Are you interested in helping with some of this stuff? Yes.
>> It might also be worthwhile putting together a plugin that connects up >> Gradle's TestClassProcessor directly to cucumber, assuming it has some kind >> of native execution API. cucumber-jvm has a Reporter api which provide before/result/after callbacks. The provided public api for test execution is either a main class or using the junit integration. This suggests it is feasible to provide closer integration to cucumber-jvm from gradle. >> This way, we can get proper descriptions for these things in the reports >> (e.g. 'Feature', 'Scenario', 'Step' rather than generic 'Test'). All of this info is also available via the junit xml report. The mapping is; testcase.classname = Feature name testcase.name = Scenario name testcase.systemout = a CDATA block that contains a list of strings where each string is Step.keyword + Step.name + (enough dots to make the line 140 chars) + Step.result this produces content like <testsuite failures="0" tests="1"> <testcase classname="Addition" name="Add Two Number" time="38.735708"> <system-out> <![CDATA[ Given I have entered 50 into the calculator.........passed And I have entered 70 into the calculator...........passed When I press add....................................passed Then the result should be 120 on the screen.........passed </system-out> </testcase> </testsuite> There is 1 junit xml file per invocation of the cucumber-jvm cli runner. Cheers Matt From: Adam Murdoch [mailto:[email protected]] Sent: 30 December 2012 00:46 To: [email protected] Subject: Re: [gradle-dev] Gradle and cucumber-jvm On 19/12/2012, at 6:18 AM, Matt Khan wrote: Classification: Public Hello We have a gradle based build/release/deployment system and we use cucumber-jvm (https://github.com/cucumber/cucumber-jvm) for certain types of testing. ***Some preamble about cucumber first which could be skipped if you're familiar with cucumber*** cucumber (or rather gherkin) is set of keywords organised into a tree like structure like (taking the example from their page) Feature: Addition Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen cucumber-jvm gives you 2 ways to run tests; via cli interface or via a junit runner. The junit approach exposes each level in this hierarchy as a junit Description where each description is simply it's displayed name, in this example we'd have a Description with name "Feature: Addition" with 1 child, which is a Description with name "Scenario: Add two numbers" with 4 children, which are a Description for each step with names "Given I have entered 50 into the calculator" etc There are other variations (scenario outlines, data tables etc) but they all fall into this sort of hierarchical structure. *** The meat of the problem *** As of today (gradle 1.3), the only option to run cucumber-jvm from gradle is to create a custom javaexec task that runs the cucumber cli Main with build specific kludges to make it agree with how gradle runs testng/junit (like checking that tests have run). This is because cucumber's use of junit does appear to agree with gradle's interpretation. There are, at least, 2 problems; 1) a blocking bug (an NPE in gradle code triggered by the fact that cucumber generated Descriptions produce a null "methodName") that prevents it from working at all (described in this thread -http://forums.gradle.org/gradle/topics/beforetest_descriptor_is_null_and_the_test_is_not_actually_run_trying_to_run_groovy_and_java_cucumber_step_definition) 2) (if one avoids the NPE by just adding a null check) gradle renders the results as a list of scenarios only The 2nd point means that key information is missing (on which step did the test fail?). My Q is pretty simple... what's the "right" way to get this working in gradle such that the full hierarchy of information available is preserved? Alternatively should we not expect to see the gradle test reporting encompass this sort of content and instead focus on getting the execution side working (without the need for custom javaexec tasks) but using cucumber to generate test reports? I'd like to see Gradle be able to deal with this. To fix this, we'd need to change the test reporting so that it does not assume that a test == a method in a class. It might also be worthwhile putting together a plugin that connects up Gradle's TestClassProcessor directly to cucumber, assuming it has some kind of native execution API. This way, we can get proper descriptions for these things in the reports (e.g. 'Feature', 'Scenario', 'Step' rather than generic 'Test'). Are you interested in helping with some of this stuff? -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com --- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures.
