[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519521#comment-17519521
]
Alexander Kriegisch commented on SUREFIRE-2004:
---
[~tibordigana], I did *not* say the issue has anything to do with long release
cycles. I said that due to those long cycles, chances are that a release fixing
this issue is still far away, because it did not get in before the M6 cut-off
date. My statement does not say anything about your work load - I see you fixed
111 issues for M6, which is impressive - or how high a priority this issue has
for you. I simply expressed my sympathy and understanding for another person
who tried to get it into M6, because I was in the same situation of having to
wait long for a release with some of my own Surefire issues.
Of course, you do not want to make a long-awaited milestone release unstable at
the last minute by introducing an unstable fix. We all appreciate your
diligence and quality focus as a Surefire maintainer. Having said that, smaller
feedback cycles and more frequent releases would still be good. If a milestone
takes almost two years to finish, even if (and also because) you are only one
person, it is simply too big. You work as fast and high-quality as you can, no
doubt about that. Therefore, you can do at least two things (maybe even both at
the same time):
* Make your milestones smaller. "Release early, release often", the good old
Linux principle. You can benefit from more frequent user feedback, amplifying
your learning process for the next release and simultaneously providing
business value to users in smaller increments. Win-win.
* Involve more collaborators, e.g. by managing contributions in a different,
more collaborative and less tiresome way, maximising the work *not* done by
yourself and accepting contributions, even if it means that you might have to
do some more polishing. That would still be quicker than micro-managing
contributors until they changed every detail the way you would have implemented
it yourself. You would get more work done per time unit like that. If PRs would
be less time-consuming and bureaucratic, the danger of disheartening
contibutors and making them stop contributing after the first few tries would
also be smaller. Not everyone can afford to focus so much on this project as
you can, i.e. if PR reviews require many iterations, you only get one-time
contributors. That does not scale well. People who contribute more often also
tend to learn and improve the quality of contributions over time. For yourself,
many iterations of reviewing, discussing and re-reviewing is also wasteful,
because each iteration requires a context switch from what you did before and
what you want to do next. You lose focus. It would be better to get a PR off
the table quickly, actively helping to finish it. When it is merged, it is off
the table, does not dangle around for weeks or months, having to be rebased
often or ending with an ugly merge. You can forget about it and focus on your
next piece of work. The ration of touch time vs. cycle time for each given
piece of work should be as small as possible, everything else is waste. Can you
afford waste, given your limited resources?
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
> Fix For: waiting-for-apache-feedback
>
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file a
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519089#comment-17519089
]
Tibor Digana commented on SUREFIRE-2004:
[~kriegaex]
This issue has nothing to do with long development releases.
If the fix not clear, then it is better not to touch the code, otherwise I
break it, and if I break it then the problem is mine and very hard to argue the
next fix. So this issue required some clarification. The problem with M6 was
that we concentrated only on issues reported by the users, then came covid and
I became more less idle in OSS and more active in legal affairs in advocacy of
some people. Today the situation is different because we have got Slawomir, so
we are two active developers. The highest priority is to catch up the plan of
milestones where our tasks would lead to break backwards compatibility (first
inside, then outside in config params).
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
> Fix For: waiting-for-apache-feedback
>
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516425#comment-17516425
]
Alexander Kriegisch commented on SUREFIRE-2004:
---
[~tibordigana], I understand [~gzm55]'s question, because Surefire release
cycles are quite long. I quickly checked on Maven Central: The first Surefire 3
milestone was released in November 2018, i.e. ~3.5 years ago. The current
milestone 5 was released in June 2020, almost 2 years ago. I myself have been
using self-built releases since M5, because bugs were either not fixed or after
being fixed there was not non-snapshot release.
I fully understand that everyone involved in Surefire is doing their best,
working for free and in their sparetime. But from a user perspective, these
"milestones" take longer than major or at least minor releases in some other
projects. Even a giant like Java has a 6 month release cadence. Would you mind
taking this topic into your next meeting or written exchange with your
co-developers and explore options to release more often? Even small bugfix or
feature releases would take some pressure off of both the dev team and the user
community.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
> Fix For: waiting-for-apache-feedback
>
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516289#comment-17516289
]
Tibor Digana commented on SUREFIRE-2004:
[~gzm55]
I am sorry but we are already on the mailing list with the release Vote.
I analysed the code but I was not totally sure about this business feature, so
I dedicated my time to the plan with milestones.
Now we know what lines of code should be touched, there are two potentional
fixes but this fix would require the developer to debug the code because there
are some open questions about the code. It would be worth to debug the code,
compare the flow in both one-pom, multiple-poms are make the decision about the
fix.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
> Fix For: waiting-for-apache-feedback
>
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516286#comment-17516286
]
James Z.M. Gao commented on SUREFIRE-2004:
--
hi [~tibordigana], thx for your response.
this should be not a complex issue, could we have time to resolve this in
3.0.0-M6?
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
> Fix For: waiting-for-feedback
>
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516282#comment-17516282
]
Tibor Digana commented on SUREFIRE-2004:
[~kriegaex]
I do not expect your feedback. It is opposite :-)
We will give you our feedback.
Last days I have assigned several issues to the backlog as they suddenly
appeared during the release Vote 3.0.0-M6 but they are not related to this
release Vote. We only do not want to forget these issues, and we will have a
chance to have a look at them during the next development iteration.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
> Fix For: waiting-for-feedback
>
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516208#comment-17516208
]
Alexander Kriegisch commented on SUREFIRE-2004:
---
[~tibordigana], I noticed the status change to "waiting for feedback", which
occurred a couple of hours *after* you got feedback and and the reproducer you
requested already. Maybe you did not notice, so I am reminding you. I think the
ball is in your court.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
> Fix For: waiting-for-feedback
>
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515832#comment-17515832
]
James Z.M. Gao commented on SUREFIRE-2004:
--
save the following script as gen-bug.sh, the you can compare the two command
result:
- ./gen-bug.sh # ok, output 100%
- ./gen-bug.sh -Daggregate # error, cannot find 100%
{code:bash}
#!/bin/sh
set -e
bug=$(mktemp -d)
trap 'rm -rf "$bug"' EXIT
mkdir -p "$bug/src/main/java" "$bug/src/test/java"
>"$bug"/src/main/java/A.java echo 'class A {}'
>"$bug"/src/test/java/ATest.java echo 'public class ATest { @org.junit.Test
>public void testFuncA() { org.junit.Assert.assertTrue(null != new A()); } }'
>"$bug"/pom.xml cat <<'END'
4.0.0
test
test
1-SNAPSHOT
junit
junit
4.13.2
test
maven-site-plugin
3.11.0
org.codehaus.mojo
exec-maven-plugin
3.0.0
exec
post-site
grep
100%
${project.reporting.outputDirectory}/surefire-report.html
maven-surefire-report-plugin
3.0.0-M5
report
END
mvn -q -f "$bug/pom.xml" post-site "$@"
{code}
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515778#comment-17515778
]
Alexander Kriegisch commented on SUREFIRE-2004:
---
[~tibordigana], I am not sure if you were asking me or James for a reproducible
project. But actually, it is super trivial: Just use any single-module JAR
project and activate the aggregation option.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515736#comment-17515736
]
Tibor Digana commented on SUREFIRE-2004:
[~gzm55]
As it seems, [~kriegaex] has a simple project with single POM which does not
work but multi-module project works. It would be worth to debug a reproducible
project because there we can better decide on the fix.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515724#comment-17515724
]
James Z.M. Gao commented on SUREFIRE-2004:
--
i met the same issue, and it seems the cause in the code here
https://github.com/apache/maven-surefire/blob/b9b2381a3dba6574bb69bd91d45fe0edea29c779/maven-surefire-report-plugin/src/main/java/org/apache/maven/plugins/surefire/report/AbstractSurefireReportMojo.java#L203
when setting `aggregate=true`, the reporter plugin will *skip* the report dir
for execution root project. It seems that we should also
`resolvedReportsDirectories.add( getSurefireReportsDirectory( project ) );` in
the aggregate branch.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488654#comment-17488654
]
Alexander Kriegisch commented on SUREFIRE-2004:
---
{quote}It does not say and does not mean that it aggregates reports accross
Maven modules.{quote}
But that is exactly what it does. It aggregates reports from multiple modules
into one, also merging packages and test classes into a single list. So far, so
good. So it is just about aggregating Surefire + Failsafe into one report. I
did not even try if it does. I know 100% that it does what I described, because
I am using it this way.
So far, so good. Like I said, the main problem is that the special case for
{{n}} modules with {{n ==1}} is what is not working, probably because the
plugin cannot handle the case that the root is not just a place to aggregate
something from other modules, but also can contain a report itself (if it is
jar-packaged), which in this case simply means that the plugin also has to
search for test reports to aggregate in its own build directory. That is
actually quite simple.
Please let us not talk about JaCoCo or other plugins, I am talking about
Surefire Reports. Never having tested those plugins for this scenario, I cannot
say anything intelligent about how they behave when told to aggregate something
for a single-module project. But I would assume that they know how to handle
this special case, because it makes sense.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488039#comment-17488039
]
Tibor Digana commented on SUREFIRE-2004:
Another plugins have a separate goal "aggregate" in order to do only one thing
on the level of parent POM - aggregating reports from reactor project.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487992#comment-17487992
]
Tibor Digana commented on SUREFIRE-2004:
[~kriegaex]
As an act of as-is status analysis of the code, th XML reports are aggregated
in one statistics, the Javadoc in the parameter "aggregate" only aggregates XML
reports from one or given directories.
{{"Whether to build an aggregated report at the root, or build individual
reports."}}
It does not say and does not mean that it aggregates reports accross Maven
modules.
IMHO, this feature might be useful for aggregating Unit tests and Integration
tests, see
[https://www.prient.co.uk/2017/03/aggregating-reports-in-multi-module.html|https://www.prient.co.uk/2017/03/aggregating-reports-in-multi-module.html].
The aggregations are not always so clear. For instance, the JaCoCo plugin has
confised me with its aggregation capabilities.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487848#comment-17487848
]
Alexander Kriegisch commented on SUREFIRE-2004:
---
[~tibordigana], I can report that there is no dedicated unit test for the
aggregate report feature and a [single
IT|https://github.com/apache/maven-surefire/blob/463fb64e4cc651fd3ca5bba54df84e8b74073f65/surefire-its/src/test/java/org/apache/maven/surefire/its/AggregateReportIT.java]
which only tests a multi-module setup with a pom-packaged root and two child
modules. See [test
resources|https://github.com/apache/maven-surefire/tree/master/surefire-its/src/test/resources/aggregate-report].
The if-clause dealing with the aggregate switch in the actual code is
[here|https://github.com/apache/maven-surefire/blob/5dcd98482c4ac599d0a053ddbe35ed18a70e62b4/maven-surefire-report-plugin/src/main/java/org/apache/maven/plugins/surefire/report/AbstractSurefireReportMojo.java#L195-L232].
I hope that this information helps you.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487672#comment-17487672
]
Tibor Digana commented on SUREFIRE-2004:
[~kriegaex]
Read my comment again. You should help us, it's the minumum work I described
above.
I am currently working on very impotant issues: wrong claspath on custom
projects, wrong classpath on ShadeFire, new xml repor impl, randomly printed
stacktrace.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487654#comment-17487654
]
Alexander Kriegisch commented on SUREFIRE-2004:
---
I did not say anything to versions prior 2.4, but about versions 2.4+.
Why would you want me to check the integration tests? I am reporting something
which is not working as expected for single-module projects, namely that in the
case of a requested aggregate report the plugin should be smart enough to
figure out that the reporter actually does not need to aggregate anything,
because the aggregate for a single-module project is simply the same as a
non-aggregate report for the same module. The reporter works for a pom-packaged
root POM with a single child module, so I am expecting it to work for a single,
jar-packaged module, too.
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (SUREFIRE-2004) Empty report for single-module project with 'aggregate=true'
[
https://issues.apache.org/jira/browse/SUREFIRE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487651#comment-17487651
]
Tibor Digana commented on SUREFIRE-2004:
So it means that the {{aggregate}} was never working in no version prior to 2.4?
Can you pls check our integration tests and see whether we are testing this
feature and how? What's the difference between our IT and your case?
> Empty report for single-module project with 'aggregate=true'
>
>
> Key: SUREFIRE-2004
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2004
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Report Plugin
>Affects Versions: 2.4, 3.0.0-M5
>Reporter: Alexander Kriegisch
>Priority: Major
>
> Using either {{-Daggregate=true}} on CLI or {{true}}
> in the plugin configuration leads to an empty report (i.e. zero tests
> reported) when e.g. executing
> {code:none}
> mvn -Dmaven.test.failure.ignore=true -Daggregate=true clean verify
> surefire-report:report-only
> {code}
> in the context of a single-module project. As soon as I make the root module
> pom-packaged and move the tests to into a child module, the aggregate report
> works.
> FYI, if I do not define the plugin and its version in my POM at all, the
> default version 2.4 used by Maven on my workstation has the same problem, so
> this does not seem to be a 3.0.0-M5 issue only.
>
> Background info about how and why I actually stumbled across this problem: I
> have an OSS multi-module project with lots of expensive UI tests. The full
> build can take 2.5 hours. I wanted to test a few CLI settings before creating
> an additional GitHub CI build workflow which can be run on demand and always
> runs all tests in all modules (ignoring errors and failures), no matter what.
> In the end, it is supposed to create a single-file aggregate HTML report
> which can easily be attached to the build and later is available for
> download, if the user so chooses in order to analyse failing tests
> comfortable and without having to scroll through build logs. You get the
> picture, I guess. In the original project, there is a pom-packaged root POM,
> so the problem described in this issue does not occur there. I simply created
> a single-module dummy project in order to verify the effect of certain build
> options quickly and not having to wait for the slow original build to finish.
> Eventually, I noticed the issue described above.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
