On 01/09/2013 02:03 PM, Jonathan Gibbons wrote:
Rory,
It is good to see that we are now able to publish Early Access Build
Test Results.
What is being done to address the test failures that you report?
Ideally, test failures should have corresponding bugs filed on
JBS/bugs.sun.com.
It would also be good to see the complete list of tests that did not
pass for a build. Right now, the numbers under Failed and Error do not
match your list of "known issues". How about automatically publishing
tests listed in JTreport/text/summary.txt that are reported as
"Failed." or "Error."?
-- Jon
Hello,
I agree it is very welcome to see the regression test results for builds.
I also agree with Jon that it would be very helpful to see the full
summary.txt output files for the test runs. Such files would allow
developers to compare the test results of their own builds to the recent
promoted builds. As a point for comparison, when I was release manager
of OpenJDK 6, I published the summary.txt files as well as the jtdiff
output; for a few examples see:
https://blogs.oracle.com/darcy/entry/openjdk_6_b22_regression_test
https://blogs.oracle.com/darcy/entry/openjdk_6_b21_regression_test
https://blogs.oracle.com/darcy/entry/openjdk_6_b20_regression_test
https://blogs.oracle.com/darcy/entry/openjdk_6_b19_regression_test
It would would be useful to have persistent per-build pages to serve as
an archive to test results over time.
Finally, how do the jtreg options used to generate the reported results
compare to the jtreg options used in the "make test" target?
Thanks,
-Joe
