Hi Rory, Got it.
Thanks, Vitaly > On 04 Oct 2016, at 19:32, Rory O'Donnell <rory.odonn...@oracle.com> wrote: > > Hi Vitaly, > > Comments inline. > > > On 04/10/2016 13:00, Vitaly Provodin wrote: >> Hi Rory, >> >> Thanks a lot for the detailed answer - now it is clear for EAB runs. Sorry I >> overlooked the note 2. >> Actually, the reason of my mail was to understand that the failures I got >> (including those two) are expected and can be explained by existing bugs. >> Totally there were about 300 test failures of java/awt and javax/swing tests >> in my environment. Most of them are explained by existing bugs. >> For others (about 70) no bugs were found - most probably they are somehow >> mentioned in Bug System but i could find no entries. >> >> Some of these 70 failures can be caused by specifics in my environment. But >> others like javax/swing/JPopupMenu/6800513/bug6800513.java may be considered >> as an issue in Java. > This could be - JDK-7184956 >> The main question here how can I report these issues (if I have no >> write-access to the Bug System)? >> It looks like time-consuming job doing it via emails... >> > > You can report issues via http://bugs.java.com/ <http://bugs.java.com/> > >> What can you recommend me? >> >> Thanks in advance, >> Vitaly >> >> P.S. now I am able to open number-links via >> http://download.java.net/openjdk/testresults/9/testresults.html >> <http://download.java.net/openjdk/testresults/9/testresults.html> >> But they can not be opened if starting page is >> http://download.java.net/openjdk/testresults/9 >> <http://download.java.net/openjdk/testresults/9>Thanks for that, I will get >> it looked at. > > Rgds,Rory > >> >>> On 04 Oct 2016, at 17:02, Rory O'Donnell <rory.odonn...@oracle.com >>> <mailto:rory.odonn...@oracle.com>> wrote: >>> >>> Hi Vitaly, >>> >>> A little bit of background, the purpose of the posting of EA build results >>> is to provide consistent, reproducible >>> results that people can generate and compare against in their own >>> environment. >>> >>> To this end, we do not run the awt, swing and 2d tests, as there are so >>> many variables and the results >>> can be inconsistent. >>> >>> On 04/10/2016 03:35, Vitaly Provodin wrote: >>>> Hello, >>>> >>>> My question relates to failures of the Regression tests (jdk/test) on >>>> OpenJDK8 and how to identify whether a failure is expected or not. >>>> >>>> Running java/awt and javax/swing tests against OpenJDK8 on various >>>> platforms I got some failures. Trying to find any explanations for these >>>> failures I looked through the JDK Bug System and could not find any bugs >>>> explaining them, for example, for the following two tests >>>> >>> As I mentioned above, we don't run these tests as part of the regular >>> testing see NOTE 2 >>> http://download.java.net/openjdk/testresults/9/testresults.html >>> <http://download.java.net/openjdk/testresults/9/testresults.html> >>> 2) ** Results do not includes test results for awt, swing and 2d. >>>> >>>> -java/awt/Frame/UnfocusableMaximizedFrameResizablity/UnfocusableMaximizedFrameResizablity.java >>>> >>>> Failed. >>>> Program >>>> `/Users/builduser/buildAgent/work/13ab0d4c51b0de0b/jbsdk/jdk/Contents/Home/bin/java' >>>> timed out (timeout set to 240000ms, elapsed time was 240786ms). >>>> >>>> - javax/swing/JPopupMenu/6800513/bug6800513.java >>>> >>>> >>>> Failed. >>>> Execution failed: `main' threw exception: java.lang.Exception: popup class >>>> is: javax.swing.PopupFactory$HeavyWeightPopup, expected: >>>> javax.swing.PopupFactory$LightWeightPopup >>>> >>>> I found out that these tests were excluded from runs against JDK 8 Update >>>> Releases Early Access Build (EAB) - see Test Results page at >>>> http://download.java.net/openjdk/testresults/8 >>>> <http://download.java.net/openjdk/testresults/8>. Please note, I do not >>>> know the status of these results, whether they are actual for OpenJDK8 >>>> final release or not. Also, I paid attention to that most of the failures >>>> I am talking about are not included into jdk/test/ProblemList.txt as for >>>> now. >>>> >>> As I mentioned above, we don't run these tests as part of the regular >>> testing see NOTE 2 >>> http://download.java.net/openjdk/testresults/9/testresults.html >>> <http://download.java.net/openjdk/testresults/9/testresults.html> >>> 2) ** Results do not includes test results for awt, swing and 2d. >>> >>>> Supposing once these tests were excluded from runs, there were some causes >>>> for that e.g. some known product or test issues which were not intended to >>>> be fixed for this particular release. My question is how can I get to know >>>> these causes in order to differentiate expected and unexpected failures - >>>> any of these tests, which were excluded from EAB runs because of one >>>> issue, may fail because of another. >>>> >>> With the exception of awt, swing and 2d, the test results or any >>> differences from one build to another are highlighted in the diff column. >>> >>>> I tried to follow the link at Summary column placed on the page >>>> http://download.java.net/openjdk/testresults/9 >>>> <http://download.java.net/openjdk/testresults/9> and could not get >>>> information about statuses of the tests (similar to OpenJDK8). >>>> >>> This link works for me, can you retry ? >>>> >>>> So another question, how can I get the list of regression tests (if any) >>>> which expectedly fail for some particular OpenJDK release? >>>> >>> In the test results page - for example >>> http://download.java.net/openjdk/testresults/9/testresults.html >>> <http://download.java.net/openjdk/testresults/9/testresults.html> >>> >>> The summary link provides a link to the tests that ran successfully. The >>> diff column provides a link to the failures that are different to the >>> previous build by component - jdk, hotspot & langtools >>> >>> I hope that helps. >>> >>> Rgds,Rory >>>> Thanks, >>>> >>>> Vitaly >>>> >>> >>> -- >>> Rgds,Rory O'Donnell >>> Quality Engineering Manager >>> Oracle EMEA, Dublin,Ireland >> > > -- > Rgds,Rory O'Donnell > Quality Engineering Manager > Oracle EMEA , Dublin, Ireland
