It was the execution data. Feel free to answer over on
https://stackoverflow.com/questions/60438452/gradle-jacoco-report-coverage-percent-doesnt-match-verification-percent
and
collect the bounty.
As for the name classDirectories, I looked at the Gradle source code and it
really does look like a poor choice of name, because it's just a file
collection.
On Sunday, March 1, 2020 at 11:22:16 AM UTC-8, Jim Showalter wrote:
>
> We have a Gradle project with JaCoCo coverage.
>
> We have centralized our excludes in a function, so we can use it in both
> the reporting and verification tasks:
>
> def coverageExcludes() {
> return [
> "com/ourcompany/*Config*",
> "com/ourcompany/ServiceApplication*",
> "com/ourcompany/Shutdown*",
> "com/ourcompany/ShutdownConnector*",
> "com/ourcompany/Tomcat*",
> "com/ourcompany/TomcatCustomConnectorCustomizer*",
> "com/ourcompany/endpoint/exception/**",
> "com/ourcompany/endpoint/util/ObjectMapperBuilder*",
> "com/ourcompany/framework/**",
> "com/ourcompany/helper/ApiHelper*",
> "com/ourcompany/helper/OffsetDateTimeDeserializer*",
> "com/ourcompany/persistence/entity/**",
> "com/ourcompany/persistence/exception/**",
> "com/ourcompany/service/exception/ServiceException*",
> "com/ourcompany/service/model/**",
> "com/ourcompany/v2/**",
> "com/ourcompany/v3/**"
> ]
> }
>
> The reporting task is configured like this:
>
> jacocoTestReport {
> reports {
> xml {
> enabled true
> }
> html {
> enabled true
> }
> }
> getExecutionData().setFrom(fileTree(buildDir).include("/jacoco/*.exec"));
> afterEvaluate {
> classDirectories.setFrom(files(classDirectories.files.collect {
> fileTree(dir: it,
> exclude: coverageExcludes()
> )
> }))
> for (Object c : classDirectories) { // Temporary loop, using it for
> debugging this problem.
> System.out.println(c);
> }
> }
> }
>
> The build generates coverage reports without a problem, and the numbers
> look correct:
>
> [image: wuJDM.png]
> But when we try to set coverage-percentage thresholds to fail the build if
> coverage dips too low, we have to set it to a much lower number than the
> report displays.
>
> For example, instead of the report's 87% coverage, we have to set the
> failure threshold at 70%. Any higher results in:
>
> Execution failed for task ':settings:jacocoTestCoverageVerification'. Rule
>> violated for bundle settings: instructions covered ratio is 0.70, but
>> expected minimum is 0.87
>>
>
> We are using the same excludes in both Gradle tasks:
>
> jacocoTestCoverageVerification {
> afterEvaluate {
> classDirectories.setFrom(files(classDirectories.files.collect {
> fileTree(dir: it,
> exclude: coverageExcludes()
> )
> }))
> for (Object c : classDirectories) { // Temporary loop, using it for
> debugging this problem.
> System.out.println(c);
> }
> }
> violationRules {
> rule {
> element = 'BUNDLE'
> limit {
> counter = 'INSTRUCTION'
> value = 'COVEREDRATIO'
> minimum = 0.87
> }
> }
> }
> } check.dependsOn jacocoTestCoverageVerification
>
> The loops that dump the contents of classDirectories output the same exact
> lists in both tasks.
>
> Comparing the un-excluded classes to the exclusions, we see that the
> excludes are working as expected.
>
> It's just that the two tasks are seeing different coverage numbers.
>
> We are pretty sure we configured the jacocoTestCoverageVerification
> violationRules with the same settings that the report task defaults to:
> BUNDLE, INSTRUCTION, COVEREDRATIO.
>
> We're like the guy with two watches: we're not sure what time it is. Are
> the numbers in the report correct, or do we in fact only have 70% coverage,
> not 87%?
>
> If we could get past this problem, we could retire an in-house custom
> test-coverage plugin that isn't multi-module aware and isn't compatible
> with Gradle past 5.5.1, and that would free us to upgrade to latest Gradle,
> so there is pent-up demand for a solution!
>
> A possible related question: why is the variable called
> "classDirectories", instead of something like "classes"? Is there some
> other variable we should be using? We got the above task configurations
> from multiple online examples that said that's the way to configure
> excludes.
>
> P.S. I reported this on stackoverflow in
> https://stackoverflow.com/q/60438452/1461450 and nobody responded. It now
> has a bounty. If someone here knows the answer, you could answer there and
> here, and pick up 50 reputation points.
>
--
You received this message because you are subscribed to the Google Groups
"JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jacoco/5ecba71d-3f4f-4a4f-8877-1bca10004625%40googlegroups.com.
