I fully agree! It would be nice to have a single test group and a single make target which could be run by everybody who succeeded to build the OpenJDK. And it should produce the same results like they are posted on http://www.java.net/download/jdk8/testresults/testresults.html . Actually, the tests results posted there should be run with the same make target and test group.
It's nice to have a test description an a website but reality shows that it is inevitable that the web page and the real scripts will get out of sync. This can not happen if you have a checked in make-file and group description. Of course it is also time to add test results for the other Oracle supported platforms (Mac, Win and Solaris) to http://www.java.net/download/jdk8/testresults/testresults.html. In my eyes, this is an absolute requirement if we want to get a stable set of tests which can be used to verify something like a port (or some other bigger changes). Concentrating on Linux will not help a lot here. The test group would be a running target of course and everybody who fixes and/or stabilizes a test should update it. Regards, Volker On Thu, Dec 12, 2013 at 12:10 AM, Jonathan Gibbons <jonathan.gibb...@oracle.com> wrote: > > On 12/11/2013 11:54 AM, Alan Bateman wrote: > > On 11/12/2013 19:26, Balchandra Vaidya wrote: > > > : > > Alternatively, that segment of the script could be a candidate for a target > in > one or more test/Makefile files. > > > This is good idea, but my experience with the 'make' is that if one target > critically fail, all > subsequent targets will not run. I thought it is a restriction of 'make'. > > I think Jon is suggesting that the subset that you run be added to > TEST.groups (which will automatically turn it a make targe as way of the > jdk_% rule ). On the surface this is a good idea but when I look at the > subset of the tests that you are running: > > :jdk_core > :jdk_svc > :jdk_beans > :jdk_imageio > :jdk_sound > :jdk_sctp > javax/accessibility > com/sun/java/swing > javax/print > sun/pisces > com/sun/awt > > then it's a bit ad hoc. I wouldn't object to adding a special group for this > but it really amounts to all jdk tests except for: > > java/awt > javax/swing > sun/awt > sun/java2d > com/apple/eawt > > If these tests aren't in your runs because of stability issues then we > should make sure that there are bugs submitted and that they get some focus. > In the interim then unstable tests can be @ignore-d or added to the exclude > list. I initially thought that part of the issue was the othervm vs. agentvm > discussion but I see in TEST.ROOT that othervm.dirs lists these directories > already. > > -Alan > > > > Yes, the high order point I was trying to make is that something is wrong if > you need to specify a long list of tests to run. While we all may take > whatever short cuts we choose to get our day to day work done, there should > be a standard set of tests[1] that we agree should be run, and which can be > run with reasonably concise command line args. > > -- Jon > > Ideally, "all" but maybe we're not there yet. > >
