Hi All,
I love the coverage metrics that I get out of TestNG/jacoco when I get them,
but I'm having a problem that's been unsolvable so far.
We're using ant as our build platform, TestNG in "mixed" mode to run mostly
JUnit, but also some TestNG tests, with Jenkins for CI. With EXACTLY THE SAME
CONFIGURATION sometimes a build will generate a 1.3M jacoco.exec (~500k lines
of code in the project) and other times it generates an empty file (0 bytes).
I *suspect* that something is causing the jvm to exit with a non-zero return
code since the documentation for jacoco says stats are only generated if the
jvm exits normally. Here's my setup...
java version "1.7.0_76"
Apache Ant(TM) version 1.9.3
junit 4.11
testng 6.8
A trigger kicks off the jenkins build. The trigger is the throttle (only 1 job
can run at a time) and the build runs by calling other jobs sequentially and
blocking until all steps complete.
Step 1 is the trigger, it deletes artifacts using a shell script...
if [ -f ./sonar/reports/jacoc.exec ] ; then echo "Removing
./sonar/report/jacoco.exec" ; rm -rf ./sonar/report/jacoco.exec ; else echo "No
./sonar/report/jacoco.exec." ; fi ;
if [ -d ./sonar/report/debug ] ; then echo "Removing ./sonar/report/debug" ; rm
-rf ./sonar/report/debug ; else echo "No ./sonar/report/debug" ; fi ;
if [ -d ./sonar/report/junit ] ; then echo "Removing ./sonar/report/junit" ; rm
-rf ./sonar/report/junit ; else echo "No ./sonar/report/junit" ; fi ;
Step 2 is a compile job that compiles the source and test code into a working
directory.
(script not included)
Step 3 is calling a shell script to eliminate any possibility of the jvm being
shared with anything else, even though testng theoretically always forks, just
to be really, really, really sure...
#!/bin/bash
echo invoking ant build...
cd sonar
/opt/local/ant/bin/ant -file build.xml run-tests
cd ..
The code has already been compiled into a working directory, there's a throttle
on the upstream job that prevents other jobs from running - this is the only
thing happening in the working directory at this time, I swear! The build step
looks like...
<property name="build.dir" value="${basedir}/build" />
<property name="app.name" value="XXXXXXXXXXXX" />
<property name="build.app.dir" value="${build.dir}/${app.name}" />
<property name="build.outputDir"
value="${build.app.dir}/WEB-INF/classes"/>
<property name="test.src.dir" value="test"/>
<property name="test.resource.dir" value="test/resources"/>
<property name="jacoco.exec" value="../sonar/report/jacoco.exec"/>
<property name="report.dir" value="../sonar/report/"/>
<property name="junit.report"
value="../sonar/report/junit/junitreports/"/>
<taskdef uri="antlib:org.jacoco.ant"
resource="org/jacoco/ant/antlib.xml" classpath="../sonar/lib/jacocoant.jar">
</taskdef>
<taskdef name="testng" classname="org.testng.TestNGAntTask">
<classpath>
<pathelement location="../sonar/lib/testng-6.8.jar" />
</classpath>
</taskdef>
<target name="run-tests" description="Run unit tests to get code
coverage metrics">
<jacoco:coverage
destfile="${jacoco.exec}"
append="true"
classdumpdir="../sonar/report/debug/"
enabled="true"
includes="com/XXXXXXX/**.*:com/XXXXXXXXX/**.*"
sessionId="1"
output="file"
dumpOnExit="true">
<testng
mode="mixed"
workingDir="${basedir}"
classfilesetref="unit.tests.fileset"
outputdir="../sonar/report/junit"
suitename="junitreports"
verbose="2"
haltonfailure="false">
<classpath>
<path refid="test.classpath" />
<pathelement
location="${build.dir}/test/classes" />
<pathelement
location="${build.outputDir}" />
<pathelement
location="${test.resource.dir}" />
</classpath>
<jvmarg line="-Xmx4096m -XX:PermSize=512m
-XX:MaxPermSize=2048m -Duser.timezone=UTC -XX:-UseGCOverheadLimit"/>
<jvmarg value="-Dfips.provider=SunRsaSign" />
</testng>
</jacoco:coverage>
</target>
The next step to try to get this to work, I guess, is to run the whole thing
from the command line as a direct invocation of TestNG outside of ant with the
jacoco command line args passed. That seems a very, very brute force way to
go, but there doesn't seem to be any practical way to debug the exit code from
testNG and I'm getting these empty files...
Verifying jacoco.exec...
/var/lib/jenkins/workspace/XXXXXXXXX-sonar-bugfix
Found ./sonar/report/jacoco.exec
-rw-r--r-- 1 jenkins jenkins 0 Apr 11 21:12 ./sonar/report/jacoco.exec
...so that's where I'm headed now, but I'm not full of hope and enthusiasm for
this solution. If I do find I'm getting a non-zero return code I'm going to
have to go diving through the TestNG code. If I don't then I'm going to be
really stuck. Surely this must be some kind of race condition...???
If anyone has any insights about where I may have stumbled or knows what the
problem is it would surely make me smile!
--
You received this message because you are subscribed to the Google Groups
"JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jacoco/e3d66467-9713-44fc-b96c-61108e18722b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
