Hi,
Please see testing status[1] for r603534 below.
Note to my comments:
- I marked tests as regression if they passed on M3 [2]
- I question tests as intermittent if they passed on the previous
snapshot (r603433) [3]
- There are some tests that failed on M3 too. (move them to the exclude lists?)
- Not all failures have JIRA issue assigned so please help me to find
related JIRA number (if there is any).
All comments and clarifications are greatly appreciated!
=====================================================
We have the next suites failed on the snapshot.
* Eclipse Unit Tests 3.3
- Linux_x86: 4 failures
- Windows_x86: pass rate 96.92%
- Linux_x86_64: pass rate 95.29%
Would be nice to get failures evaluation.
We definitely improved stability of the Eclipse Unit Test suite since
M3 (Thanks to Vladimir Beliaev).
But the are still some issues such as intermittent suite crashes (that
make me very upset).
* EGAx48:
- Linux_x86 & Linux_x86_64 : failed. (HARMONY-5250 / regression?)
- Windows_x86:
Still no status – the scenario causes CC host down after running
several hours.
Currently I can not figure out if this is the host's issue or
problem introduced by Harmony.
* EHWA(GUI)
- Linux_x86 & Linux_x86_64 : failed.
The scenario is quite unstable. Most probably I'll drop the
scenario in M5 from snapshot test and use only EHWA-API.
* Functional:
F_KeyFactoryTest_01: regression on all platforms (HARMONY-4857)
- Linux_x86:
JComponent: intermittent?
- Linux_x86_64:
btest6353: fails on M3 too
- Windows_x86_64:
DataOutputStream, writeObjectReadObject0004, IfElse1, btest5717,
btest7214 : intermittent?
btest6353: fails on M3 too
* GUT:
ContainerTest, JAASSecurityTest - fail on all platforms
HARMONY-4895 says that this bug in Geronimo and not a regression.
- Linux_x86:
HARMONY-5302 - says that this a bug in tests (I don't understand
then why they passed on M3)
security\jaas\ConfigurationEntryTest
security\jaas\LoginKerberousNonGeronimoTest
security\jaas\LoginKerberousTest
security\jaas\LoginSQLTest
security\network\protocol\SubjectCarryingProtocolTest
security\remoting\jmx\RemoteLoginTest
- Linux_x86_64: 53 failures (11 on M3)
- Windows_x86_64: 22 failures (28 on M3)
* JDKTools
- Windows_x86_64: 3 failures by TimeoutException.
Regression? I previously incorrectly reported that they are
intermittent. They stably fail on M4 builds.
* Reliability
Failed on x86 platforms but they also failed on M3
api.net.HttpConnectionTest (I wonder why HARMONY-5266 didn't fixed it)
api.net.SingleConnectTest
- Linux_x86:
Regresions:
api.kernel.object.ObjectFinalizeTest
api.kernel.throwable.StackTraceExcptsTest
api.text.DecimalFormat_Locales
api.zip.ZipEntryIterateThreadingTest
api.zip.ZlibTest
vm.finalization.FinalizeThrowRestoreTest
vm.stack.StackUnwindTest
They also failed on M3:
api.nio.channels.filechannel.FileChannelMapTest
api.nio.channels.filechannel.MappedByteBufferForceTest
api.serialization.SerializableClassesTest
- Windows_x86
Regresions:
api.kernel.string.InternThreadingTest - regression
api.kernel.thread.Calculation.CalcTest
api.kernel.thread.RecursiveThreadTest
api.kernel.thread.StackTraceTest
api.kernel.thread.Synchronization
api.kernel.thread.ThreadArrayTest
api.kernel.threadgroup.EnumerateTest
api.net.DatagramTest
api.serialization.SerializableClassesTest
vm.stack.StackUnwindTest
- Linux_x86_64:
Regressions:
vm.classloading.ClassCastTest
* Stress
- Linux_x86:
StackUnwindingManyObjectsTests: 5 fails on M3 too (HARMONY-5158 /
test issue?)
Fragmentation, FragmentationFinalizer, FragmentationReference:
HARMONY-5159 (wasn't run during M3 so not regression)
- Windows_x86:
StackUnwindingManyObjectsTests: 5 fails on M3 too (HARMONY-5158 /
test issue?)
FragmentationReference: HARMONY-5159 (wasn't run during M3 so not
regression)
- Linux_x86_64:
jni.arguments,jni.arrays - fails on M3 too so not regression / HARMONY-????
MEMORY001, MEMORY003 - HARMONY-5047 (fails on M3 too so not regression)
- Windows_x86_64:
NotSynchThreads.ManyClasses - crash (HARMONY-5003), regression
Fragmentation, FragmentationFinalizer, FragmentationReference:
HARMONY-5159 (wasn't run during M3 so not regression)
gc.mem.MemoryTest3 - regression / HARMONY-????
jni.stack - fails on M3 too so not regression / HARMONY-????
EVENT014- crash (HARMONY-????)/ regression
MEMORY001, MEMORY003 - HARMONY-5047 (fails on M3 too so not regression)
StressThreads12Test - crash (HARMONY-????)/ regression
* VTS VM
- Windows_x86:
dstore_20701 - intermittent?
[1] http://people.apache.org/~mloenko/snapshot_testing/script/r603534/index.html
[2] http://people.apache.org/~mloenko/snapshot_testing/script/r580985/index.html
[3] http://people.apache.org/~mloenko/snapshot_testing/script/r603433/index.html
Thanks,
Stepan Mishura
Intel Enterprise Solutions Software Division
