Richard, all,
Yes, I agree it would be nice if we could get code-coverage reports
published.
But that being said, I have a couple of comments, which I will say up
front appear contradictory, even to me, as I write them.
1. The Java SQE org does do what you're suggesting: they run coverage
reports and look to plug the holes.
2. With respect, I think you're looking for needles in haystacks, and
missing the giant low hanging fruit all around us. In other words,
you're looking at the wrong granularity.
When I was writing my recent blog entry last week, I went looking for
examples of bad shell scripts. I started looking at the first 10 shell
scripts that find showed me, and found problems. Within an hour, I had
written scripts to identify more instances of those problems and filed
bugs in the system. Separately, as you may know if you follow the
various aliases, there is an ongoing effort to improve the reliability
of many areas of the test suite, and/or to "fix" tests that appear on
the Problem list". In other words, it's not so much that we need /more/
tests; we need /better/ tests.
If you do want to write more, new tests, I think that is as easy as
deciding what area you're interested in, and then asking on the relevant
aliases. For example, even though the code coverage numbers for javac
are pretty good, we do not have many tests that verify the debug
attributes generated by javac, such LineNumberTable, LocalVariableTable,
etc. Mostly, class files generated by javac are tested by running them.
We do not have any/many tests that check javac's behavior with respect
to reading invalid class files. We went *15 years* before anyone
noticed and/or got concerned that javadoc generated invalid HTML. (Ooops.)
So, bottom line, even without code coverage tools, there's still easy
pickings ;-)
-- Jon
On 05/18/2013 10:54 AM, Richard Warburton wrote:
Hi,
Thanks for the replies everyone.
In Oracle, the tool that we use is called "jcov", it's proprietary
so I can't point you at anything. Maybe someday it could be
proposed to the Code Tool Projects but I have no knowledge of
specific plans on that.
If jcov produces HTML reports - would it be possible to have the
reports from your internal CI builds published? This would at least
improve the situation even in the absence of jcov being made available.
I wonder if anyone has tried using EMMA or any of the other code
coverage tools out there? At least for the JDK releases in
development then one challenge would be keeping the tool up to the
date with the class file format, another would be taking care when
instrumenting very core classes that are used early in VM startup.
It's the same challenge that profiles and other tools that do
bytecode instrumentation have used on a JDK version that is still
in development.
I imagine with sufficient effort Jacoco could be made to work - though
I was hoping to avoid debugging Jacoco with Java 8. I'll have a look
into this. At work we've got a patch against Jacoco in order to merge
together the results from different sub module builds in maven, so
I'll see if its possible to use this this same approach in order to
make Jacoco work with merging separate files from different openjdk
VMs as Johnathan has suggested.
regards,
Richard Warburton
http://insightfullogic.com
@RichardWarburto <http://twitter.com/richardwarburto>
