[
https://issues.apache.org/jira/browse/KAFKA-14682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751178#comment-17751178
]
Chris Egerton commented on KAFKA-14682:
---------------------------------------
I've taken a stab at an upstream Mockito fix. If it's accepted, we can either
bump to a new version with that fix, or look into tweaking our Jenkinsfile to
run the {{test}} task instead of {{{}unitTest integrationTest{}}}.
> Unused stubbings are not reported by Mockito during CI builds
> -------------------------------------------------------------
>
> Key: KAFKA-14682
> URL: https://issues.apache.org/jira/browse/KAFKA-14682
> Project: Kafka
> Issue Type: Test
> Components: unit tests
> Reporter: Chris Egerton
> Assignee: Chris Egerton
> Priority: Major
>
> We've started using [strict
> stubbing|https://javadoc.io/static/org.mockito/mockito-core/4.6.1/org/mockito/junit/MockitoJUnitRunner.StrictStubs.html]
> for unit tests written with Mockito, which is supposed to automatically fail
> tests when they set up mock expectations that go unused.
> However, these failures are not reported during Jenkins builds, even if they
> are reported when building/testing locally.
> In at least one case, this difference appears to be because our [Jenkins
> build|https://github.com/apache/kafka/blob/6d11261d5deaca300e273bebe309f9e4f814f815/Jenkinsfile#L32-L35]
> uses the custom {{unitTest}} and {{integrationTest}} tasks defined in the
> project's [Gradle build
> file|https://github.com/apache/kafka/blob/6d11261d5deaca300e273bebe309f9e4f814f815/build.gradle#L452-L543],
> instead of the {{test}} task. Some IDEs (such as IntelliJ) may use the
> latter instead of the former, which can cause tests to fail due to
> unnecessary stubbings when being run in that IDE but not when being built on
> Jenkins.
> It's possible that, because the custom test tasks filter out some tests from
> running, Mockito does not check for unnecessary stubbings in order to avoid
> incorrectly failing tests that set up mocks in, e.g., a {{@BeforeEach}}
> method.
>
> This exact behavior has been reported elsewhere as a [Gradle
> issue|https://github.com/gradle/gradle/issues/10694]; based on [discussion on
> that
> thread|https://github.com/gradle/gradle/issues/10694#issuecomment-1374911274],
> it appears this is a known and somewhat-intentional limitation of Mockito:
> {quote}I spent some time trying to solve this and eventually I stumbled upon
> this piece in Mockito's JUnit runner:
> [https://github.com/mockito/mockito/blob/main/src/main/java/org/mockito/internal/runners/StrictRunner.java#L47-L53]
> {{ }}
> {{}}
> {{ // only report when:
> // 1. if all tests from given test have ran (filter requested is
> false)
> // Otherwise we would report unnecessary stubs even if the user
> runs just single test
> // from the class
> // 2. tests are successful (we don't want to add an extra failure
> on top of any existing
> // failure, to avoid confusion)}}
>
> (1) suggests that skipping unused stub validation is the intended behavior
> when the user filters a single test from the class. However, this behavior
> applies to any type of filter.
> And Gradle indeed applies a {{CategoryFilter}} if categories are configured:
> [https://github.com/rieske/gradle/blob/e82029abb559d620fdfecb4708a95c6313af625c/subprojects/testing-jvm-infrastructure/src/main/java/org/gradle/api/internal/tasks/testing/junit/JUnitTestClassExecutor.java#L70-L96]
> Which then causes Mockito to not validate unused stubs.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
