[
https://issues.apache.org/jira/browse/SUREFIRE-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Jones updated SUREFIRE-1851:
---------------------------------
Description:
If throwing an exception within a test where the stack trace is null, a core
utility (SmartStackTraceParser in comon-java5) will throw an NPE. This will
cause the test suite to fail.
What is especially scary about this is that it will not log that the test has
failed, and in some configurations will even declare the build is successful
and all tests passed. Additionally, the surefire report plugin will declare all
tests passed too.
An exception with a null stacktrace sounds odd, but is easy to do by mocking an
exception with frameworks like Mockito. While people probably shouldn't be
mocking exceptions, it definitely [can and does
happen|https://github.com/search?q=mock%28DataIntegrityViolationException.class%29+language%3AJava+fork%3Afalse&type=code].
Not sure what versions exactly are affected. But it's affected the
`common-java5` package since 2012. At least both JUnit 4 and 5 are affected,
with surefire 2.22.1 or 3.0.0-M5. Logs attached.
Can be reproduced with https://github.com/domdomegg/surefire-1851-demo
I have a patch ready that I believe fixes this.
was:
If throwing an exception within a test where the stack trace is null, a core
utility (SmartStackTraceParser in comon-java5) will throw an NPE. This will
cause the test suite to fail.
What is especially scary about this is that it will not log that the test has
failed, and in some configurations will even declare the build is successful
and all tests passed. Additionally, the surefire report plugin will declare all
tests passed too.
An exception with a null stacktrace sounds odd, but is easy to do by mocking an
exception with frameworks like Mockito. While people probably shouldn't be
mocking exceptions, it definitely [can and does
happen|https://github.com/search?q=mock%28DataIntegrityViolationException.class%29+language%3AJava+fork%3Afalse&type=code].
Not sure what versions exactly are affected. But it's affected the
`common-java5` package since 2012. At least both JUnit 4 and 5 are affected,
with surefire 2.22.1 or 3.0.0-M5. Logs attached.
I have a patch ready that I believe fixes this.
> NPE in SmartStackTraceParser causes false positive test results
> ---------------------------------------------------------------
>
> Key: SUREFIRE-1851
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1851
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.x support, JUnit 5.x support, Maven Surefire
> Plugin
> Reporter: Adam Jones
> Priority: Major
> Attachments: junit4.11andsurefire2.22.2.txt,
> junit5.7.0andsurefire3.0.0M5.txt
>
>
> If throwing an exception within a test where the stack trace is null, a core
> utility (SmartStackTraceParser in comon-java5) will throw an NPE. This will
> cause the test suite to fail.
> What is especially scary about this is that it will not log that the test has
> failed, and in some configurations will even declare the build is successful
> and all tests passed. Additionally, the surefire report plugin will declare
> all tests passed too.
> An exception with a null stacktrace sounds odd, but is easy to do by mocking
> an exception with frameworks like Mockito. While people probably shouldn't be
> mocking exceptions, it definitely [can and does
> happen|https://github.com/search?q=mock%28DataIntegrityViolationException.class%29+language%3AJava+fork%3Afalse&type=code].
> Not sure what versions exactly are affected. But it's affected the
> `common-java5` package since 2012. At least both JUnit 4 and 5 are affected,
> with surefire 2.22.1 or 3.0.0-M5. Logs attached.
> Can be reproduced with https://github.com/domdomegg/surefire-1851-demo
> I have a patch ready that I believe fixes this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
