I think none of these are blocker for M7 because they are basically old issues or intermettent errors. I also think that we are doing better than M6.
It is also noted that (as the comment in HARMONY-5747) the error in 10) is a new issue due to an old classlib checkin r641928 which can be stably reproduced . Maybe we need to have a look in it? On 8/14/08, Sian January <[EMAIL PROTECTED]> wrote: > > Does anyone have a feel for whether any of these failures should be > blockers for M7? In terms of the Harmony test suite I think the pass > rate is fine because we're doing better than M6, and EUTs look ok too, > but I'm not really sure if we need to be concerned about any of: > > 4) Functional > 5) JDKTools Tests > 6) JettyScenario > 7) Reliability > 8) Stress > > or > > 10) Eclipse Hello World Application. > > Any thoughts? > > If none of these are blockers then it would be good to start a vote in > the next day or two because the code has been frozen for nearly 3 > weeks now. > > Thanks, > > Sian > > > On 11/08/2008, chunrong lai <[EMAIL PROTECTED]> wrote: > > Hi, all: > > > > Here is r681495 snapshot testing status: > > http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am > using > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we > will > > have other two platforms in future for M8. Well, although we are testing > > only two platforms for M7, my experience is that the status for another > two > > platforms should be not worse or just include some extra intermittent > errors > > which can be investigated in some later stages. > > > > The following suites passed on Linux x86/Windows x86_64 platforms: Ant > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite. > > > > Most of the failures are known issues (for M6). Although we can observe > > 15~20 new issues, those issues happen only in 1 platform and they look > more > > like the intermittent/timeout issues (less reproducible) to me. Overall I > > myself would like to think r681495 is more stable than M6. > > > > Please add your comments and clarifications (please also see M6 testing > > results http://people.apache.org/~smishura/r653525/, > > > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/[EMAIL > PROTECTED] > > integrity testing results > > http://people.apache.org/~chunrong/harmony-integrity/ as a comparison). > > > > 1) Classlib: > > 1.1) Since r644719 (which committed patch for HARMONY-5684) > > > > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest > > > > > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest > > > > failed in both platforms > > > > 1.2) Two failures > > > > org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed > > in windows_x86 running of M6) > > tests.api.java.security.PermissionCollectionTest (Failed > > in M6) > > > > are observed in Linux x86. > > > > 2) DRLVM tests: > > 2.1) One failure > > > > java.lang.ClassGenericsTest.test_2 > > > > is observed in Linux x86 snapshot testing. > > I can see some old discussion in the mailing list about that but > I > > am not sure the expected status here. > > They should be intemittent errors since the integrity testing > just > > run well mostly. > > > > 3) EUTs: > > > > 3.1) Linux x86: 99.36% > > A recorded JIRA for this suite is HARMONY-2914 which wastes file > > handlers and makes system unstable. > > > > 4) Functional: > > 4.1) Old regressions on both platforms: > > api.java.text.MessageFormat (HARMONY-5430) > > api.java.util.jar.Manifest (HARMONY-5473) > > api.java.beans.beancontext.BeanContextTest (also in M6, recorded > > as regression caused by changes in locale data) > > api.java.beans.persistence.EncoderTest (also in M6, recorded as > > regression in the beans module) > > api.java.beans.persistence.EncoderDecoderTest (also in M6, > > regression in the beans module) > > reg.vm.btest5625 (also in M6, recorded as intermittent and not > > reproducible manually) > > > > 4.2) Old regressions on 1 platform > > api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823) > > api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86, > > HARMONY-5823) > > jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6 > it > > is recorded as regression since M4) > > java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86, > > recorded as Timeout, not reproducible > > in M6) > > reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout, > > the test is too heavy" in M6) > > jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in > > windows x86_64, recorded as "looks like > > issue in test" in M6) > > jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64, > in > > M6 it is recorded as not regression and started to fail on M5) > > reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded > also > > failed on M3 and might be issue with the test) > > > > 4.3) New regressions on 1 platform (looks intermittent) > > reg.jit.btest8029.Btest8029 (FAILED in Linux x86) > > func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86) > > api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR > > in Linux x86) > > api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR > > in Linux x86) > > > > 5) JDKTools Tests: > > Several timeouts are observed in Linux x86 snapshot running. They are: > > > > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001 > > > > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002 > > > > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003 > > > > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004 > > > > > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04 > > > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001 > > > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001 > > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume > > > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001 > > > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001 > > The Linux-only timeouts are also observed in the integrity testing > > results. > > JIRA HARMONY-5833 has been filed for one of them. > > > > 6) JettyScenario: > > The Linux x86 running failed because of the unresolved HARMONY-5219. > > > > 7) Reliability: > > Several failures are observed in windows x86_64 running. > > 7.1) Old regressions > > api.net.DatagramTest (HARMONY-5531) > > api.text.DecimalFormat_Locales - (in M6 it is recorded as also > > intermittent in M5) > > > > 7.2) New/intemittent regressions > > api.kernel.thread.ThreadLocalTest.ThreadLocalTest > > api.kernel.exec.RunExec > > > > 8) Stress > > Different test cases failed on different platforms. > > 8.1) Timeouts on Linux x86. > > > > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003 > > > > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007 > > > > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009 > > > > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011 > > > > 8.2) Failed cases on Windows x86_64 with unknown reasons. > > > > > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2 > > > > > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses > > > > I have not found records for them. > > > > 9) Strut_test > > Broken with same error report as M6. > > > > 10) Eclipse Hello World Application. > > Although the testing framework just reports EUT-API status as > "PASSED". > > A fresh workspace running just failed in configuration stage ( > > > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/[EMAIL > PROTECTED] > ) > > since SVN commit 641928 (which committed patch for HARMONY-4569). > > > > thanks, > > chunrong > > > > > -- > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >
