Hi,
if exec files are not generated at all, or clipped somewhere in the
middle (as you stack trace suggests) this is typically due to a
non-graceful shutdown of the JVM running the tests.
Just this week we identified a JVM bug which causes corrupted exec files
under a specific condition, see:
https://github.com/jacoco/jacoco/issues/394
You might check whether the main thread or the thread finally calling
System.exit() is in the interrupted state.
Regards,
-marc
On 2016-04-11 23:51, [email protected] wrote:
Hi All,
I love the coverage metrics that I get out of TestNG/jacoco when I get
them, but I'm having a problem that's been unsolvable so far.
We're using ant as our build platform, TestNG in "mixed" mode to run
mostly JUnit, but also some TestNG tests, with Jenkins for CI. With
EXACTLY THE SAME CONFIGURATION sometimes a build will generate a 1.3M
jacoco.exec (~500k lines of code in the project) and other times it
generates an empty file (0 bytes).
I *suspect* that something is causing the jvm to exit with a non-zero
return code since the documentation for jacoco says stats are only
generated if the jvm exits normally. Here's my setup...
java version "1.7.0_76"
Apache Ant(TM) version 1.9.3
junit 4.11
testng 6.8
A trigger kicks off the jenkins build. The trigger is the throttle
(only 1 job can run at a time) and the build runs by calling other
jobs sequentially and blocking until all steps complete.
Step 1 is the trigger, it deletes artifacts using a shell script...
if [ -f ./sonar/reports/jacoc.exec ] ; then echo "Removing
./sonar/report/jacoco.exec" ; rm -rf ./sonar/report/jacoco.exec ; else
echo "No ./sonar/report/jacoco.exec." ; fi ;
if [ -d ./sonar/report/debug ] ; then echo "Removing
./sonar/report/debug" ; rm -rf ./sonar/report/debug ; else echo "No
./sonar/report/debug" ; fi ;
if [ -d ./sonar/report/junit ] ; then echo "Removing
./sonar/report/junit" ; rm -rf ./sonar/report/junit ; else echo "No
./sonar/report/junit" ; fi ;
Step 2 is a compile job that compiles the source and test code into a
working directory.
(script not included)
Step 3 is calling a shell script to eliminate any possibility of the
jvm being shared with anything else, even though testng theoretically
always forks, just to be really, really, really sure...
#!/bin/bash
echo invoking ant build...
cd sonar
/opt/local/ant/bin/ant -file build.xml run-tests
cd ..
The code has already been compiled into a working directory, there's a
throttle on the upstream job that prevents other jobs from running -
this is the only thing happening in the working directory at this
time, I swear! The build step looks like...
<property name="build.dir" value="${basedir}/build" />
<property name="app.name" value="XXXXXXXXXXXX" />
<property name="build.app.dir" value="${build.dir}/${app.name}" />
<property name="build.outputDir"
value="${build.app.dir}/WEB-INF/classes"/>
<property name="test.src.dir" value="test"/>
<property name="test.resource.dir" value="test/resources"/>
<property name="jacoco.exec" value="../sonar/report/jacoco.exec"/>
<property name="report.dir" value="../sonar/report/"/>
<property name="junit.report"
value="../sonar/report/junit/junitreports/"/>
<taskdef uri="antlib:org.jacoco.ant"
resource="org/jacoco/ant/antlib.xml"
classpath="../sonar/lib/jacocoant.jar">
</taskdef>
<taskdef name="testng" classname="org.testng.TestNGAntTask">
<classpath>
<pathelement location="../sonar/lib/testng-6.8.jar" />
</classpath>
</taskdef>
<target name="run-tests" description="Run unit tests to get code
coverage metrics">
<jacoco:coverage
destfile="${jacoco.exec}"
append="true"
classdumpdir="../sonar/report/debug/"
enabled="true"
includes="com/XXXXXXX/**.*:com/XXXXXXXXX/**.*"
sessionId="1"
output="file"
dumpOnExit="true">
<testng
mode="mixed"
workingDir="${basedir}"
classfilesetref="unit.tests.fileset"
outputdir="../sonar/report/junit"
suitename="junitreports"
verbose="2"
haltonfailure="false">
<classpath>
<path refid="test.classpath" />
<pathelement
location="${build.dir}/test/classes" />
<pathelement location="${build.outputDir}"
/>
<pathelement
location="${test.resource.dir}" />
</classpath>
<jvmarg line="-Xmx4096m -XX:PermSize=512m
-XX:MaxPermSize=2048m
-Duser.timezone=UTC -XX:-UseGCOverheadLimit"/>
<jvmarg value="-Dfips.provider=SunRsaSign" />
</testng>
</jacoco:coverage>
</target>
The next step to try to get this to work, I guess, is to run the whole
thing from the command line as a direct invocation of TestNG outside
of ant with the jacoco command line args passed. That seems a very,
very brute force way to go, but there doesn't seem to be any practical
way to debug the exit code from testNG and I'm getting these empty
files...
Verifying jacoco.exec...
/var/lib/jenkins/workspace/XXXXXXXXX-sonar-bugfix
Found ./sonar/report/jacoco.exec
-rw-r--r-- 1 jenkins jenkins 0 Apr 11 21:12 ./sonar/report/jacoco.exec
...so that's where I'm headed now, but I'm not full of hope and
enthusiasm for this solution. If I do find I'm getting a non-zero
return code I'm going to have to go diving through the TestNG code.
If I don't then I'm going to be really stuck. Surely this must be
some kind of race condition...???
If anyone has any insights about where I may have stumbled or knows
what the problem is it would surely make me smile!
--
You received this message because you are subscribed to the Google Groups "JaCoCo
and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jacoco/f0e0ff7b6142736dd47c97329d32f52c%40mountainminds.com.
For more options, visit https://groups.google.com/d/optout.
