Since some of the tests are marked generic-all and it's difficult to run
100 iterations on all platforms, I concentrated on the tests that are
marked failing on one platform.
I found that these tests are passing when ran for 100 iterations on windows
javax/swing/JComboBox/6607130/bug6607130.java
javax/swing/JPopupMenu/6675802/bug6675802.java
javax/swing/plaf/basic/BasicTextUI/8001470/bug8001470.java
javax/swing/plaf/basic/Test6984643.java
javax/swing/text/JTextComponent/5074573/bug5074573.java
javax/swing/SwingWorker/6493680/bug6493680.java
javax/swing/UIDefaults/8149879/InternalResourceBundle.java
javax/swing/PopupFactory/8048506/bug8048506.java
and these on mac
javax/swing/JButton/8151303/PressedIconTest.java
javax/swing/JWindow/ShapedAndTranslucentWindows/ShapedPerPixelTranslucentGradient.java
Proposed webrev:
http://cr.openjdk.java.net/~psadhukhan/8213181/webrev.0/
Regards
Prasanta
On 06-Nov-18 6:29 AM, Sergey Bylokhov wrote:
So shouldn't this test just be marked @headful ?
I can mark it headful if Sergey is ok with it.
This will skip the test in our nightly, but actually does not solve the existed
bug in the test.
fails last time due to some update dialog or antivirus popup)
javax/swing/JButton/8151303/PressedIconTest.java 8198689 macosx-all*
javax/swing/JWindow/ShapedAndTranslucentWindows/ShapedPerPixelTranslucentGradient.java
I guess there will be many more tests that will fail if we try to
run
for 100 iterations and Problemlist will swell, so I guess we should
keep only those tests in ProblemList that fail if we run
individually.
The ProblemList excludes the tests from the nightly and testing after commit,
the rest of tests(which are headless and not in this list) are quite stable.
Automated tests that are being run on a regular basis need to pass
reliably.
But we do not run each test in an batch for 100 iterations to find out
if it is reliable (should we?).
I do that, our sqe run the tests for 1000 iterations.
I guess we just run it individually in standalone mode and if it
pass
everytime, we consider it reliable, and all these 5 tests pass if run
I guess the problem here is that these tests were added to the problem list
without
investigation of root cause, at least It was not checked that it was a setup issue like
wrong "Display color profile".
As a solution I suggest to check the source code of the tests, and try to find
some issues which may affect
the stability of the tests, then run these tests in a bash loop for 100
iterations to prove that they
really stable.
