[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[
https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15639146#comment-15639146
]
Tibor Digana commented on SUREFIRE-1230:
[~giacomotaylor]
[~mpalki]
I have lately found this issue:
https://github.com/cucumber/cucumber-jvm/issues/966
As I understood the problem in Cucumber the RunListener and run of test method
is not totally synchronized.
Is it so that the Cucumber annotation on the test method does not match "step
definitions" and should be a subject to ignored test?
If it is this issue, then Cucumber can fix it by overriding
{{ParentRunner#runChild(final FrameworkMethod method, RunNotifier notifier)}}
to something like this:
{code}
if (child.getAnnotation(SomeCucumberAnnotation.class) != matched step
definition) notifier.fireTestIgnored(description);
super.runChild(method, notifier);
{code}
I can provide a workaround but it will be only workaround which is in your case
still worse than regular consistent report.
> Issue with config parameter 'redirectTestOutputToFile' when running cucumber
> parallel tests
> ---
>
> Key: SUREFIRE-1230
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1230
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
>Affects Versions: 2.18.1
>Reporter: Manoj Palki
>Assignee: Tibor Digana
> Fix For: 2.19.2
>
> Attachments: cucumber_java_skeleton.tar.gz
>
>
> The config parameter redirectTestOutputToFile behaves differently based on
> how parallelism is configured in the surefire plugin.
> If the config is
> 2
> false
> true
> then the output from the tests is directed correctly to 'xxxtest-output.txt'
> file.
> However, if the config is
> 2
> classes
> true
> then the output from the tests is partially directed to the
> 'xxxtest-output.txt file and partially to stdout.
> Also the output files have different names in two cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629551#comment-15629551 ] James Taylor commented on SUREFIRE-1230: Any ETA on getting this patch committed to Surefire and getting a 2.19.2 patch release, [~tibor17]? > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki >Assignee: Tibor Digana > Fix For: 2.19.2 > > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514287#comment-15514287 ] James Taylor commented on SUREFIRE-1230: Got it, thanks. I'm happy to test out your patch once it's ready. > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki >Assignee: Tibor Digana > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[
https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513630#comment-15513630
]
Tibor Digana commented on SUREFIRE-1230:
If the parameter {{redirectTestOutputToFile}} is set to {{true}} the logs
should be dropped from console but appear in *.txt. The logic is a little bit
more difficult because this depends on {{useFile}} as well.
If some Thread started by your test is logging to a console, we are not able to
associate these logs with particular test class and therefore you won't be able
to see the logs in {{TEST-MyTest.txt}} but you should be able to see it in
{{TEST-null.txt}} together with logs from all tests having this problem.
> Issue with config parameter 'redirectTestOutputToFile' when running cucumber
> parallel tests
> ---
>
> Key: SUREFIRE-1230
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1230
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
>Affects Versions: 2.18.1
>Reporter: Manoj Palki
>Assignee: Tibor Digana
> Attachments: cucumber_java_skeleton.tar.gz
>
>
> The config parameter redirectTestOutputToFile behaves differently based on
> how parallelism is configured in the surefire plugin.
> If the config is
> 2
> false
> true
> then the output from the tests is directed correctly to 'xxxtest-output.txt'
> file.
> However, if the config is
> 2
> classes
> true
> then the output from the tests is partially directed to the
> 'xxxtest-output.txt file and partially to stdout.
> Also the output files have different names in two cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[
https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513452#comment-15513452
]
James Taylor commented on SUREFIRE-1230:
bq. I plan to redirect the logs to {{target/surefire-reports/TEST-null.txt}} in
these circumstances.
Do you mean it'll redirect to the log file for the particular test that's
running or that the output will simply be dropped? Is the output that is
currently going to the console also getting logged in the tests log file?
BTW, this parallel feature is really awesome - our tests are able to run 10x
faster than before.
> Issue with config parameter 'redirectTestOutputToFile' when running cucumber
> parallel tests
> ---
>
> Key: SUREFIRE-1230
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1230
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
>Affects Versions: 2.18.1
>Reporter: Manoj Palki
>Assignee: Tibor Digana
> Attachments: cucumber_java_skeleton.tar.gz
>
>
> The config parameter redirectTestOutputToFile behaves differently based on
> how parallelism is configured in the surefire plugin.
> If the config is
> 2
> false
> true
> then the output from the tests is directed correctly to 'xxxtest-output.txt'
> file.
> However, if the config is
> 2
> classes
> true
> then the output from the tests is partially directed to the
> 'xxxtest-output.txt file and partially to stdout.
> Also the output files have different names in two cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[
https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513433#comment-15513433
]
Tibor Digana commented on SUREFIRE-1230:
I am currently working on several fixes and this is one of them.
I plan to redirect the logs to {{target/surefire-reports/TEST-null.txt}} in
these circumstances.
> Issue with config parameter 'redirectTestOutputToFile' when running cucumber
> parallel tests
> ---
>
> Key: SUREFIRE-1230
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1230
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
>Affects Versions: 2.18.1
>Reporter: Manoj Palki
>Assignee: Tibor Digana
> Attachments: cucumber_java_skeleton.tar.gz
>
>
> The config parameter redirectTestOutputToFile behaves differently based on
> how parallelism is configured in the surefire plugin.
> If the config is
> 2
> false
> true
> then the output from the tests is directed correctly to 'xxxtest-output.txt'
> file.
> However, if the config is
> 2
> classes
> true
> then the output from the tests is partially directed to the
> 'xxxtest-output.txt file and partially to stdout.
> Also the output files have different names in two cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512114#comment-15512114 ] James Taylor commented on SUREFIRE-1230: I'm seeing this issue and I'm not using Cucumber. > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki >Assignee: Tibor Digana > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512100#comment-15512100 ] Manoj Palki commented on SUREFIRE-1230: --- Hi Tibor, When I posted this bug, I was not sure whether it was caused by Surefire or Cucumber. It seems like this is a Cucumber issue. They have fixed it. See https://github.com/cucumber/cucumber-jvm/issues/966 > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki >Assignee: Tibor Digana > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512059#comment-15512059 ] James Taylor commented on SUREFIRE-1230: Good find, [~tibor17]. Is there a workaround? Would be awesome to get the fix into a point release. > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki >Assignee: Tibor Digana > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[
https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504926#comment-15504926
]
Tibor Digana commented on SUREFIRE-1230:
[~mpalki]
{{ConcurrentRunListener.java}} in the method {{void writeTestOutput( byte[]
buf, int off, int len, boolean stdout )}}.
If the {{TestMethod}} is not associated with current Thread, the logs go to the
console.
> Issue with config parameter 'redirectTestOutputToFile' when running cucumber
> parallel tests
> ---
>
> Key: SUREFIRE-1230
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1230
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
>Affects Versions: 2.18.1
>Reporter: Manoj Palki
>Assignee: Tibor Digana
> Attachments: cucumber_java_skeleton.tar.gz
>
>
> The config parameter redirectTestOutputToFile behaves differently based on
> how parallelism is configured in the surefire plugin.
> If the config is
> 2
> false
> true
> then the output from the tests is directed correctly to 'xxxtest-output.txt'
> file.
> However, if the config is
> 2
> classes
> true
> then the output from the tests is partially directed to the
> 'xxxtest-output.txt file and partially to stdout.
> Also the output files have different names in two cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187846#comment-15187846 ] Tibor Digana commented on SUREFIRE-1230: [~mpalki] Sorry for late reply. Can you check it out with no forks (forkCount=0, forkMode=never)? If it happens only with forking, debug the events received by ForkCLient.java. If it happens with in-process run, then the bug has to do with Surefire's RunListener.java and subtypes. > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15163455#comment-15163455 ] Manoj Palki commented on SUREFIRE-1230: --- When I add more threads, I notice that the problem exists even in version 2.15 . Can you tell me where in the code I should start looking to debug this ? I can try attaching a debugger and see where the problem lies. > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159817#comment-15159817 ] Tibor Digana commented on SUREFIRE-1230: This issue was first time introduced in version 2.16. I reproduced it with your test but I have been quite busy to fix it. Somebody else should have a look because I will concentrate on other very urgent issues. > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159382#comment-15159382 ] Manoj Palki commented on SUREFIRE-1230: --- Any updates or workarounds for this bug ? > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149426#comment-15149426 ] Manoj Palki commented on SUREFIRE-1230: --- Same issue with 2.19.1 too. > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SUREFIRE-1230) Issue with config parameter 'redirectTestOutputToFile' when running cucumber parallel tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149298#comment-15149298 ] Tibor Digana commented on SUREFIRE-1230: [~mpalki] Check it out with the latest version 2.19.1. > Issue with config parameter 'redirectTestOutputToFile' when running cucumber > parallel tests > --- > > Key: SUREFIRE-1230 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1230 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Surefire Plugin >Affects Versions: 2.18.1 >Reporter: Manoj Palki > Attachments: cucumber_java_skeleton.tar.gz > > > The config parameter redirectTestOutputToFile behaves differently based on > how parallelism is configured in the surefire plugin. > If the config is > 2 > false > true > then the output from the tests is directed correctly to 'xxxtest-output.txt' > file. > However, if the config is > 2 > classes > true > then the output from the tests is partially directed to the > 'xxxtest-output.txt file and partially to stdout. > Also the output files have different names in two cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
