We have a Gradle project with JaCoCo coverage.
We have centralized our excludes in a function, so we can use it in both
the reporting and verification tasks:
def coverageExcludes() {
return [
"com/ourcompany/*Config*",
"com/ourcompany/ServiceApplication*",
"com/ourcompany/Shutdown*",
"com/ourcompany/ShutdownConnector*",
"com/ourcompany/Tomcat*",
"com/ourcompany/TomcatCustomConnectorCustomizer*",
"com/ourcompany/endpoint/exception/**",
"com/ourcompany/endpoint/util/ObjectMapperBuilder*",
"com/ourcompany/framework/**",
"com/ourcompany/helper/ApiHelper*",
"com/ourcompany/helper/OffsetDateTimeDeserializer*",
"com/ourcompany/persistence/entity/**",
"com/ourcompany/persistence/exception/**",
"com/ourcompany/service/exception/ServiceException*",
"com/ourcompany/service/model/**",
"com/ourcompany/v2/**",
"com/ourcompany/v3/**"
]
}
The reporting task is configured like this:
jacocoTestReport {
reports {
xml {
enabled true
}
html {
enabled true
}
}
getExecutionData().setFrom(fileTree(buildDir).include("/jacoco/*.exec"));
afterEvaluate {
classDirectories.setFrom(files(classDirectories.files.collect {
fileTree(dir: it,
exclude: coverageExcludes()
)
}))
for (Object c : classDirectories) { // Temporary loop, using it for
debugging this problem.
System.out.println(c);
}
}
}
The build generates coverage reports without a problem, and the numbers
look correct:
[image: wuJDM.png]
But when we try to set coverage-percentage thresholds to fail the build if
coverage dips too low, we have to set it to a much lower number than the
report displays.
For example, instead of the report's 87% coverage, we have to set the
failure threshold at 70%. Any higher results in:
Execution failed for task ':settings:jacocoTestCoverageVerification'. Rule
> violated for bundle settings: instructions covered ratio is 0.70, but
> expected minimum is 0.87
>
We are using the same excludes in both Gradle tasks:
jacocoTestCoverageVerification {
afterEvaluate {
classDirectories.setFrom(files(classDirectories.files.collect {
fileTree(dir: it,
exclude: coverageExcludes()
)
}))
for (Object c : classDirectories) { // Temporary loop, using it for
debugging this problem.
System.out.println(c);
}
}
violationRules {
rule {
element = 'BUNDLE'
limit {
counter = 'INSTRUCTION'
value = 'COVEREDRATIO'
minimum = 0.87
}
}
}
} check.dependsOn jacocoTestCoverageVerification
The loops that dump the contents of classDirectories output the same exact
lists in both tasks.
Comparing the un-excluded classes to the exclusions, we see that the
excludes are working as expected.
It's just that the two tasks are seeing different coverage numbers.
We are pretty sure we configured the jacocoTestCoverageVerification
violationRules with the same settings that the report task defaults to:
BUNDLE, INSTRUCTION, COVEREDRATIO.
We're like the guy with two watches: we're not sure what time it is. Are
the numbers in the report correct, or do we in fact only have 70% coverage,
not 87%?
If we could get past this problem, we could retire an in-house custom
test-coverage plugin that isn't multi-module aware and isn't compatible
with Gradle past 5.5.1, and that would free us to upgrade to latest Gradle,
so there is pent-up demand for a solution!
A possible related question: why is the variable called "classDirectories",
instead of something like "classes"? Is there some other variable we should
be using? We got the above task configurations from multiple online
examples that said that's the way to configure excludes.
P.S. I reported this on stackoverflow
in https://stackoverflow.com/q/60438452/1461450 and nobody responded. It
now has a bounty. If someone here knows the answer, you could answer there
and here, and pick up 50 reputation points.
--
You received this message because you are subscribed to the Google Groups
"JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jacoco/99e10edd-4f09-45df-a899-cc83b2b06723%40googlegroups.com.
