Hi Vincent,
Thanks for the update, plenty of info for us to review. We may comeback
with follow on questions.
Rgds,Rory
On 03/11/2017 15:33, Vincent Privat wrote:
Hello Rory,
Thank you :)
Here is how we test various OpenJDK builds:
1.1: we use only EA builds from http://jdk.java.net/
2.1.1: no
2.1.2/2.1.3: we run all unit and integration tests of JOSM
2.1.4: yes, using jacoco
2.2: all tests are automated
2.3: it depends. During OpenJDK9 cycle we did not run a test series
for each weekly build, but rather once every two or three builds,
unless we were waiting for a specific correction from OpenJDK or
another tool. At the end of development, we were not waiting for any
correction from OpenJDK but were eagerly waiting for the updates of
ASM and everything that as blocked by the non-availability of ASM (in
particular various Maven plugins)
2.4: Yes, they are visible here:
- JOSM core:
https://josm.openstreetmap.de/jenkins/job/Java-EarlyAccess-JOSM/
- JOSM community plugins:
https://josm.openstreetmap.de/jenkins/job/Java-EarlyAccess-JOSM-Plugins/
I just have initiated JDK10 tests today, and we have 53 failures
caused by Byte Buddy not recognizing Java version 10:
java.lang.IllegalArgumentException: Unknown Java version: 10
at
nl.jqno.equalsverifier.internal.lib.bytebuddy.ClassFileVersion.ofJavaVersion(ClassFileVersion.java:135)
I'm sure Rafael is aware of it. I will see how to update our tools
(equalsverifier here).
Cheers,
Vincent
2017-11-03 14:05 GMT+01:00 Rory O'Donnell <rory.odonn...@oracle.com
<mailto:rory.odonn...@oracle.com>>:
Hi Vincent,
I noticed that JOSM have been commended in the last two Quality
reports [1] [2] for the number and
indeed the quality of bugs you have found and logged against
various early access releases.
Thanks again for your contribution to the quality of our releases!
The big question is how do you do it ?
We would love to hear more details about your test environment,
what tests you run etc. Below is a
series of questions that come to mind, but there may be other
details that you would like to share.
1. Builds
1. do you build your own OpenJDK builds, use Oracle's or
other EA builds ?
2. Tests
1. What tests do you run
1. do you run the JCK ?
2. open regression tests ?
3. other tests ?
4. do you measure code coverage ?
2. Are the tests Automated or Manual ?
3. How frequently to do execute the tests
4. Are the results visible via an external link
Rgds, Rory
[1]
https://wiki.openjdk.java.net/display/quality/Quality+Outreach+report+September+2017
<https://wiki.openjdk.java.net/display/quality/Quality+Outreach+report+September+2017>
[2]
https://wiki.openjdk.java.net/display/quality/Quality+Outreach+report+March+2017
<https://wiki.openjdk.java.net/display/quality/Quality+Outreach+report+March+2017>
--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland
--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland
