Chunrong, thanks for the good work! You mentioned the M7 candidate has some new issues, does it have less old issues? I wonder why you think it is stabler.
Thanks, xiaofeng On Tue, Aug 12, 2008 at 1:26 AM, chunrong lai <[EMAIL PROTECTED]> wrote: > Hi, all: > > Here is r681495 snapshot testing status: > http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am using > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we will > have other two platforms in future for M8. Well, although we are testing > only two platforms for M7, my experience is that the status for another two > platforms should be not worse or just include some extra intermittent errors > which can be investigated in some later stages. > > The following suites passed on Linux x86/Windows x86_64 platforms: Ant > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite. > > Most of the failures are known issues (for M6). Although we can observe > 15~20 new issues, those issues happen only in 1 platform and they look more > like the intermittent/timeout issues (less reproducible) to me. Overall I > myself would like to think r681495 is more stable than M6. > > Please add your comments and clarifications (please also see M6 testing > results http://people.apache.org/~smishura/r653525/, > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/[EMAIL > PROTECTED] > integrity testing results > http://people.apache.org/~chunrong/harmony-integrity/ as a comparison). > > 1) Classlib: > 1.1) Since r644719 (which committed patch for HARMONY-5684) > > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest > > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest > > failed in both platforms > > 1.2) Two failures > > org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed > in windows_x86 running of M6) > tests.api.java.security.PermissionCollectionTest (Failed > in M6) > > are observed in Linux x86. > > 2) DRLVM tests: > 2.1) One failure > > java.lang.ClassGenericsTest.test_2 > > is observed in Linux x86 snapshot testing. > I can see some old discussion in the mailing list about that but I > am not sure the expected status here. > They should be intemittent errors since the integrity testing just > run well mostly. > > 3) EUTs: > > 3.1) Linux x86: 99.36% > A recorded JIRA for this suite is HARMONY-2914 which wastes file > handlers and makes system unstable. > > 4) Functional: > 4.1) Old regressions on both platforms: > api.java.text.MessageFormat (HARMONY-5430) > api.java.util.jar.Manifest (HARMONY-5473) > api.java.beans.beancontext.BeanContextTest (also in M6, recorded > as regression caused by changes in locale data) > api.java.beans.persistence.EncoderTest (also in M6, recorded as > regression in the beans module) > api.java.beans.persistence.EncoderDecoderTest (also in M6, > regression in the beans module) > reg.vm.btest5625 (also in M6, recorded as intermittent and not > reproducible manually) > > 4.2) Old regressions on 1 platform > api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823) > api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86, > HARMONY-5823) > jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6 it > is recorded as regression since M4) > java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86, > recorded as Timeout, not reproducible > in M6) > reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout, > the test is too heavy" in M6) > jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in > windows x86_64, recorded as "looks like > issue in test" in M6) > jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64, in > M6 it is recorded as not regression and started to fail on M5) > reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also > failed on M3 and might be issue with the test) > > 4.3) New regressions on 1 platform (looks intermittent) > reg.jit.btest8029.Btest8029 (FAILED in Linux x86) > func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86) > api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR > in Linux x86) > api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR > in Linux x86) > > 5) JDKTools Tests: > Several timeouts are observed in Linux x86 snapshot running. They are: > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001 > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002 > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003 > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004 > > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04 > > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001 > > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001 > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001 > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001 > The Linux-only timeouts are also observed in the integrity testing > results. > JIRA HARMONY-5833 has been filed for one of them. > > 6) JettyScenario: > The Linux x86 running failed because of the unresolved HARMONY-5219. > > 7) Reliability: > Several failures are observed in windows x86_64 running. > 7.1) Old regressions > api.net.DatagramTest (HARMONY-5531) > api.text.DecimalFormat_Locales - (in M6 it is recorded as also > intermittent in M5) > > 7.2) New/intemittent regressions > api.kernel.thread.ThreadLocalTest.ThreadLocalTest > api.kernel.exec.RunExec > > 8) Stress > Different test cases failed on different platforms. > 8.1) Timeouts on Linux x86. > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003 > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007 > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009 > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011 > > 8.2) Failed cases on Windows x86_64 with unknown reasons. > > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2 > > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses > > I have not found records for them. > > 9) Strut_test > Broken with same error report as M6. > > 10) Eclipse Hello World Application. > Although the testing framework just reports EUT-API status as "PASSED". > A fresh workspace running just failed in configuration stage ( > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/[EMAIL > PROTECTED]) > since SVN commit 641928 (which committed patch for HARMONY-4569). > > thanks, > chunrong > -- http://xiao-feng.blogspot.com
