No I think more about
if (severity == "WARN") {
log.warn("There is something wrong");
} else if (severity == "ERROR") {
throw new MojoFailureException("...") {
} else {
throw new MojoExecutionException("...") {
}
That way the plugin can handle any error/failure/... in a way that the
user can "downgrade" a certain problem if desired.
Am 18.03.22 um 00:06 schrieb Tibor Digana:
@Christoph
FATAL , WARN , ERROR
They are log levels?
The plugin does not control the log level after caught exception in Maven
core. The Maven Core does!
I think it's time to make a demo.
On Thu, Mar 17, 2022 at 6:21 AM Christoph Läubrich <m...@laeubi-soft.de>
wrote:
Hi Tibor,
it shouldn't be to hard to guess another property like
maven.test.jvmcrash=FATAL
maven.test.failure=WARN
maven.test.error=ERROR
:-)
Am 16.03.22 um 12:25 schrieb Tibor Digana:
Hi Christoph,
Such a granularity with error/failure might be also an additional
requirement but still you miss the third option to JVM error which is
different from test error/failure - they don;t have the same meaning.
T
On Mon, Mar 14, 2022 at 10:57 AM Christoph Läubrich <m...@laeubi-soft.de
wrote:
Just wondering but maybe it would be better to configure the severity
instead of a true/false, something like:
maven.test.failure=WARN
maven.test.error=ERROR
would only warn about failing tests but thrw exception if starting the
test fails?
Am 14.03.22 um 10:52 schrieb Tibor Digana:
Romain, it is not a bug.
Don't consider this as a bug. It was a feature request for change by
Olivier, and not a bug.
I closed both issues years ago but not because of ignorance but because
the
appearance of the exceptional behavior is a wrong compromise and which
does
not help anyone and even it makes the situation worse because typically
other group of users would fire a new Jira tickets. You would not be
able
to satisfy two contradictory parties which have completely different
opinions, and so we use to introduce new params and this way we satisfy
both parties, they may combine the params as they wish, and everybody
would
be happy and nobody would claim. Many technical solutions might exist,
e.g.
param=boolean|string or new param=boolean, whatever.
The truth is that this problem with (java --add-reads ...) happened in
our
ASF environment on Java 8 which in good configuration would not happen
and
should not.
It is not right way that we abuse "maven.test.failure.ignore" which
would
tell us "Hey, you have illegal configuration in your build system and
it
would not work by JDK design", it is not the goal of the plugin to tell
you
that you have configured the build wrong because it won't and the same
configuration of the plugin (not the build) says "ignore the error".
Yesterday I discussed this problem with Herve and we independently
observed
equal opinions and that's not everything because we understood that the
purpose of the config parameter is to not throw MOJO exception which is
right, but the exceptional behavior should have an exact new param for
the
exact use case.
SPI for this parameter is too much because no user would implement it
for a
trivial parameter;; the SPI is used to be implemented by frameworks or
systems or application servers but this is not our case.
On Mon, Mar 14, 2022 at 9:11 AM Romain Manni-Bucau <
rmannibu...@gmail.com>
wrote:
+1
if it is to investigate a CI issue, it is generally easy to add debug
insights (by code or agent) so a SPI sounds like the sanest for the
plugin
to me.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <
https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<
https://www.packtpub.com/application-development/java-ee-8-high-performance
Le lun. 14 mars 2022 à 09:08, Guillaume Nodet <gno...@apache.org> a
écrit
:
If that's not currently possible, maybe a SPI should be provided so
that
people can use plug in extensions to analyze the test result and
override
it if necessary (transforming an error into a warning, storing
results
in a
way which is easier to use by other tools later...) ?
Guillaume
Le lun. 14 mars 2022 à 07:43, Christoph Läubrich <
m...@laeubi-soft.de>
a
écrit :
I also agree that the test at least should run, we use this property
to
run the test and produce result and later on have a buildstep that
analyze the results (and probably fail the build job).
As it is not recommend, I wonder what is the recommended way to
archive
something similar?
Am 14.03.22 um 06:29 schrieb Olivier Lamy:
On Mon, 14 Mar 2022 at 11:55, Tibor Digana <tibordig...@apache.org
wrote:
In case of the user property *maven.test.failure.ignore* the MOJO
must
not
throw any exception which is interpreted by the Maven Core as
BUILD
SUCCESS.
This is a very simple reduction of the problem description.
The documentation here
https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#testFailureIgnore
says
"Set this to "true" to ignore a failure during testing. Its use is
NOT
RECOMMENDED, but quite convenient on occasion"
Personally, I understand this to ignore failures in junit results
BUT
at
least I want the tests to run.
I guess this is how our users use this feature (feature we do not
recommend
by the way...)
And this is perfectly explained here
https://issues.apache.org/jira/browse/SUREFIRE-1426?focusedCommentId=16188077&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16188077
there is a difference between ignoring some junit failures and
ignoring a
configuration error because some jvm args has been misconfigured
for
many
reasons and surefire cannot start.
If I follow your reasoning ("MOJO must not throw any exception ")
we
should
ignore every surefire configuration error and keep running the
build
until
the end and says BUILD SUCCESS
what about
mvn test -Dsurefire.rerunFailingTestsCount=notanumber
-Dmaven.test.failure.ignore=true
we should not throw any exceptions as you said below and keep
saying
BUILD
SUCCESS?
argLine is a configuration element as any others so it should fail
directly
and not be ignored especially when the surefire plugin cannot run.
We have received an internal requirement to change the behavior of
the
user
property *maven.test.failure.ignore* so that the behavior will
have
one
exception.
Suppose that you have JDK 1.8 but you use:
/jre/java --add-reads ...
The outcome is JVM exit with an error message.
I agree with Herve who also says that *maven.test.failure.ignore*
should
not allow the MOJO to throw the exception. It is not a typical JVM
segmentation fault or another JVM crash where we cannot do
anything
about
it, and where the entire build would crash in the command line
which
of course means that the build cannot normally be interpreted as
BUILD
SUCCESS. So we are still on the same level of failures related to
the
test
purposes.
On the other hand, Olivier has reopened the issues SUREFIRE-1426
and
SUREFIRE-1681 where the exceptional behavior of the feature is
expected.
This is exactly the reason why I closed these tickets several
years
ago
and
my proposal was to not to have any exceptions in the feature
behavior
and
the proposal was to introduce a new user property for exact use
cases.
The next idea, which comes from two developers, would provide us
with
the
same non exceptional and exact behavior of the user property
*maven.test.failure.ignore* but it would also provide us with new
user
property in the case with fine grade control of the build errors,
e.g.
*maven.test.jvm.error.ignore*.
with a default of?
honestly I just see this new parameter as introducing more
complexity
in
an
already very complex code and not worth it.
again read both issue's comments and my comments.
Please try to have a user POV and think about making our users'
experience
more simple.
This should be very simple:
If surefire forked jvm cannot start it's build error and cannot run
any
tests, it's a problem users want to know immediately because it can
be
for
a lot of reasons: wrong argLine, not enough memory on the system
etc...
AND AGAIN it is very different from ignoring junit result failures.
Try to look at a user point of view and think about what is the use
case
of
the property maven.test.failure.ignore=true, I guess 99% of the
time,
users
wants to run all their tests (even on a CI with different matrix)
so
they
can collect all the tests results which has runned to see if there
is
any
issue for some combination of their matrix and eventually turn the
result
as unstable (this is a very typical use case in Jenkins and was
even
a
built in feature of the previous Jenkins Maven plugin)
BUT if for any reasons one of the module do not start his tests
because
the
jvm cannot be start the users will not see that and will be totally
blind
until maybe someone look inside a very very large log file (which
means
probably never)
Long story short as my experience as a user facing this
problem/bug:
I had the case on a very large multi modules (~250 modules) build
of
a
very
used open source project.
I was using this maven.test.failure.ignore property but one of the
modules
had a bad jpms configuration for a jdk17 profile on the CI.
The build has a matrix of 2 os and 4 jdks and different maven run
which
means around ~60k tests and a Jenkins log file about 40M
So because of this property the build was not failing and kept
saying
BUILD
SUCCESS for weeks/months and basically not testing one module with
jdk
17...
And frankly you do not dig into a log file of 32M after each run
especially
when it says success...
3 days after the first release claiming supporting jdk 17 we
received a
bug
report about a something not working with jdk17....
and guess what? Where was this feature supposed to be tested?
so I frankly believe we do not need a complex new property, in this
case
this should fail directly because this will improve user
experience.
I would like to open the discussion on this topic. You're welcome!
Cheers
Tibor
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
--
------------------------
Guillaume Nodet
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
