Hi Vitaly,
A little bit of background, the purpose of the posting of EA build
results is to provide consistent, reproducible
results that people can generate and compare against in their own
environment.
To this end, we do not run the awt, swing and 2d tests, as there are so
many variables and the results
can be inconsistent.
On 04/10/2016 03:35, Vitaly Provodin wrote:
Hello,
My question relates to failures of the Regression tests (jdk/test) on
OpenJDK8 and how to identify whether a failure is expected or not.
Running java/awt and javax/swing tests against OpenJDK8 on various
platforms I got some failures. Trying to find any explanations for
these failures I looked through the JDK Bug System and could not find
any bugs explaining them, for example, for the following two tests
As I mentioned above, we don't run these tests as part of the regular
testing see NOTE 2
http://download.java.net/openjdk/testresults/9/testresults.html
2) ** Results do not includes test results for awt, swing and 2d.
/-java/awt/Frame/UnfocusableMaximizedFrameResizablity/UnfocusableMaximizedFrameResizablity.java/
Failed.
Program
`/Users/builduser/buildAgent/work/13ab0d4c51b0de0b/jbsdk/jdk/Contents/Home/bin/java'
timed out (timeout set to 240000ms, elapsed time was 240786ms).
/- javax/swing/JPopupMenu/6800513/bug6800513.java/
Failed.
Execution failed: `main' threw exception: java.lang.Exception:
popup class is: javax.swing.PopupFactory$HeavyWeightPopup,
expected: javax.swing.PopupFactory$LightWeightPopup
I found out that these tests were excluded from runs against JDK 8
Update Releases Early Access Build (EAB) - see Test Results page at
http://download.java.net/openjdk/testresults/8. Please note, I do not
know the status of these results, whether they are actual for OpenJDK8
final release or not. Also, I paid attention to that most of the
failures I am talking about are not included into
jdk/test/ProblemList.txt as for now.
As I mentioned above, we don't run these tests as part of the regular
testing see NOTE 2
http://download.java.net/openjdk/testresults/9/testresults.html
2) ** Results do not includes test results for awt, swing and 2d.
Supposing once these tests were excluded from runs, there were some
causes for that e.g. some known product or test issues which were not
intended to be fixed for this particular release. My question is how
can I get to know these causes in order to differentiate expected and
unexpected failures - any of these tests, which were excluded from EAB
runs because of one issue, may fail because of another.
With the exception of awt, swing and 2d, the test results or any
differences from one build to another are highlighted in the diff column.
I tried to follow the link at Summary column placed on the page
http://download.java.net/openjdk/testresults/9 and could not get
information about statuses of the tests (similar to OpenJDK8).
This link works for me, can you retry ?
So another question, how can I get the list of regression tests (if
any) which expectedly fail for some particular OpenJDK release?
In the test results page - for example
http://download.java.net/openjdk/testresults/9/testresults.html
The summary link provides a link to the tests that ran successfully. The
diff column provides a link to the failures that are different to the
previous build by component - jdk, hotspot & langtools
I hope that helps.
Rgds,Rory
Thanks,
Vitaly
--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA, Dublin,Ireland
