Hi Rory, Thanks a lot for the detailed answer - now it is clear for EAB runs. Sorry I overlooked the note 2. Actually, the reason of my mail was to understand that the failures I got (including those two) are expected and can be explained by existing bugs. Totally there were about 300 test failures of java/awt and javax/swing tests in my environment. Most of them are explained by existing bugs. For others (about 70) no bugs were found - most probably they are somehow mentioned in Bug System but i could find no entries.
Some of these 70 failures can be caused by specifics in my environment. But others like javax/swing/JPopupMenu/6800513/bug6800513.java may be considered as an issue in Java. The main question here how can I report these issues (if I have no write-access to the Bug System)? It looks like time-consuming job doing it via emails... What can you recommend me? Thanks in advance, Vitaly P.S. now I am able to open number-links via http://download.java.net/openjdk/testresults/9/testresults.html <http://download.java.net/openjdk/testresults/9/testresults.html> But they can not be opened if starting page is http://download.java.net/openjdk/testresults/9 <http://download.java.net/openjdk/testresults/9> > On 04 Oct 2016, at 17:02, Rory O'Donnell <rory.odonn...@oracle.com> wrote: > > Hi Vitaly, > > A little bit of background, the purpose of the posting of EA build results is > to provide consistent, reproducible > results that people can generate and compare against in their own environment. > > To this end, we do not run the awt, swing and 2d tests, as there are so many > variables and the results > can be inconsistent. > > On 04/10/2016 03:35, Vitaly Provodin wrote: >> Hello, >> >> My question relates to failures of the Regression tests (jdk/test) on >> OpenJDK8 and how to identify whether a failure is expected or not. >> >> Running java/awt and javax/swing tests against OpenJDK8 on various platforms >> I got some failures. Trying to find any explanations for these failures I >> looked through the JDK Bug System and could not find any bugs explaining >> them, for example, for the following two tests >> > As I mentioned above, we don't run these tests as part of the regular testing > see NOTE 2 > http://download.java.net/openjdk/testresults/9/testresults.html > <http://download.java.net/openjdk/testresults/9/testresults.html> > 2) ** Results do not includes test results for awt, swing and 2d. >> >> -java/awt/Frame/UnfocusableMaximizedFrameResizablity/UnfocusableMaximizedFrameResizablity.java >> >> Failed. >> Program >> `/Users/builduser/buildAgent/work/13ab0d4c51b0de0b/jbsdk/jdk/Contents/Home/bin/java' >> timed out (timeout set to 240000ms, elapsed time was 240786ms). >> >> - javax/swing/JPopupMenu/6800513/bug6800513.java >> >> >> Failed. >> Execution failed: `main' threw exception: java.lang.Exception: popup class >> is: javax.swing.PopupFactory$HeavyWeightPopup, expected: >> javax.swing.PopupFactory$LightWeightPopup >> >> I found out that these tests were excluded from runs against JDK 8 Update >> Releases Early Access Build (EAB) - see Test Results page at >> http://download.java.net/openjdk/testresults/8 >> <http://download.java.net/openjdk/testresults/8>. Please note, I do not know >> the status of these results, whether they are actual for OpenJDK8 final >> release or not. Also, I paid attention to that most of the failures I am >> talking about are not included into jdk/test/ProblemList.txt as for now. >> > As I mentioned above, we don't run these tests as part of the regular testing > see NOTE 2 > http://download.java.net/openjdk/testresults/9/testresults.html > <http://download.java.net/openjdk/testresults/9/testresults.html> > 2) ** Results do not includes test results for awt, swing and 2d. > >> Supposing once these tests were excluded from runs, there were some causes >> for that e.g. some known product or test issues which were not intended to >> be fixed for this particular release. My question is how can I get to know >> these causes in order to differentiate expected and unexpected failures - >> any of these tests, which were excluded from EAB runs because of one issue, >> may fail because of another. >> > With the exception of awt, swing and 2d, the test results or any differences > from one build to another are highlighted in the diff column. > >> I tried to follow the link at Summary column placed on the page >> http://download.java.net/openjdk/testresults/9 >> <http://download.java.net/openjdk/testresults/9> and could not get >> information about statuses of the tests (similar to OpenJDK8). >> > This link works for me, can you retry ? >> >> So another question, how can I get the list of regression tests (if any) >> which expectedly fail for some particular OpenJDK release? >> > In the test results page - for example > http://download.java.net/openjdk/testresults/9/testresults.html > <http://download.java.net/openjdk/testresults/9/testresults.html> > > The summary link provides a link to the tests that ran successfully. The diff > column provides a link to the failures that are different to the previous > build by component - jdk, hotspot & langtools > > I hope that helps. > > Rgds,Rory >> Thanks, >> >> Vitaly >> > > -- > Rgds,Rory O'Donnell > Quality Engineering Manager > Oracle EMEA, Dublin,Ireland
