> Thanks for your report. Waiting for real issues analysis. Analysis details:
9 failures on Windows: api.kernel.hooks.AddRmPropertiesHooksTest timeout api.kernel.thread.ThreadArrayTest.ThreadArrayTest OutOfMemoryError - regression api.kernel.threadgroup.EnumerateTest OutOfMemoryError - regression api.net.DatagramTest timeout api.net.HttpConnectionTest configuration problem api.net.SingleConnectTest timeout api.nio.buffers.ByteBufferallocateTest IllegalArgumentException - regression api.serialization.SerializableClassesTest timeout vm.gc.WeakRefsTest timeout 13 failed on Linux: api.kernel.exec.RunExec too many files opened api.kernel.management.SystemResourceOrientedTest vm crash - SIGSEGV api.kernel.thread.ThreadArrayTest.ThreadArrayTest OutOfMemoryError api.net.DatagramTest timeout api.net.HttpConnectionTest configuration problem api.net.SingleConnectTest address is not available – busy socket(?) api.nio.buffers.ByteBufferallocateTest IllegalArgumentException api.nio.channels.filechannel.FileChannelMapTest timeout api.nio.channels.filechannel.MappedByteBufferForceTest vm crash - SIGSEGV api.serialization.SerializableClassesTest timeout vm.finalization.FinalizeThrowRestoreTest timeout vm.gc.WeakRefsTest timeout vm.stack.StackUnwindTest SIGABRT in VM code So it seems we have 3 regressions on Windows. I will check if we have all these failures submitted as JIRAs. Thanks, Andrey On 9/25/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > On 9/25/07, Andrey Yakushev <[EMAIL PROTECTED]> wrote: > > I also made some analysis for reliability test suite. > > I've found that several tests didn't fail but rather were killed by > > guarding timeout. > > > > They are: > > api.nio.channels.filechannel.FileChannelMapTest > > api.net.DatagramTest > > vm.finalization.FinalizeThrowRestoreTest > > vm.gc.WeakRefsTest > > api.serialization.SerializableClassesTest > > > > I think it is due to slowing down of the test host after repeated > > tests iteration. I suggest increasing the killing timeout. I've opened > > HARMONY-4851 for this. > > I'm OK with increasing timeout. > > > > > I've run excluded reliability tests several times on Windows and Linux > > and I don't see the failures in the following excluded tests now: > > > > api.kernel.thread.Calculation.CalcTest > > api.kernel.thread.StackTraceTest.StackTraceTest > > api.nio.charset.ContainsTest > > api.text.DecimalFormat_Locales > > api.zip.ZlibTest > > > > I suggest removing them from the exclude list. I've opened > > HARMONY-4850 for this. > > > > I'm OK with the exclude list clean up. > > > api.net.HttpConnectionTest fails due to unavailability of > > harmony.apache.org site from the test host network. It works fine if > > site is accessible. > > > > Why not to fix the test to be run against local http server (for > example, jetty)? > > > Other failures seem the real reliability issues. These tests fail only > > after several successful iterations. > > > > Thanks for your report. Waiting for real issues analysis. > > Thanks, > Stepan. > > > Thanks, > > Andrey > > > > > > > > > > On 9/25/07, Tim Ellison <[EMAIL PROTECTED]> wrote: > > > I'll be looking at some of the functional, reliability, and stress tests > > > failures and seeing if there is anything I can help with. I'll post to > > > the list for any areas that I pick up. > > > > > > Regards, > > > Tim > > > > > > Stepan Mishura wrote: > > > > On 9/24/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > >> On 9/22/07, Stepan Mishura wrote: > > > >>> Hi, > > > >>> > > > >>> According to the plan - Sept. 22 is code freeze date for M3 so let's > > > >>> follow policy: > > > >>> "no more commits please without agreement from two committers on the > > > >>> dev list." > > > >>> > > > >>> Let's do more testing and analyze if there any critical/blocker issues > > > >>> for consideration. > > > >>> Please raise here issues that you think MUST be fixed for M3. > > > >> Here is [1] the status of the last snapshot (r578410) that includes > > > >> last classlib updates. Unfortunately, not everything is green there. > > > >> I'm going to inspect all results to make sure that there are no > > > >> failures caused by CC configuration issues. > > > >> > > > > > > > > I've compared testing status for the last snapshot r578410 [1] with M2 > > > > [2][3]. > > > > > > > > We have the following short status for failed suites: > > > > > > > > Windows_x86: > > > > * classlib: 2 tests from pack200 module fail on snapshot (but pass > > > > on debug build) > > > > * Eclipse unit tests 3.2: there is no tests report like for M3. The > > > > pass rate was improved from 99.32%[3] to 99.7%[1] > > > > * Eclipse unit tests 3.3 are new and the pass rate is 99.77%. I > > > > think is acceptable > > > > * EGA x48 hours scenario fails. According to [4] it passed on M2. > > > > * Functional: need more analysis, currently I see that 2 tests were > > > > enabled and new 15 regressions. > > > > * Geronimo: 2 regressions > > > > * JDK tools: 1 test failed. It might be intermediate failure - the > > > > test failed due to timeout > > > > * Reliability: 65 tests passed for M2 and 64 for M3. Investigation > > > > is required. > > > > * Stress: 190 tests passed for M2 and 189 for M3. Investigation is > > > > required. > > > > > > > > Linux_x86: > > > > * classlib: 2 tests from pack200 (as for Windows), 1 luni tests > > > > failed and 1 crash of security test > > > > * Eclipse unit tests 3.2: 2 suites crashed so pass rate is 69.60%. I > > > > assume this may be CC host configuration issue. Going to investigate. > > > > * Eclipse unit tests 3.3 are new and the pass rate is 96.47%. I > > > > think is acceptable > > > > * EGA x48 hours scenario: the same as for Windows (scenario fails on > > > > M3) > > > > * Functional: need more analysis, similar to Windows - some test are > > > > passing now but there are new failures. > > > > * Geronimo: 2 regressions (the same as for Windows) > > > > * Reliability: need more analysis > > > > * Stress: need more analysis > > > > > > > > As I remember Sean took pack200 tests. And Alexei Zakharov took > > > > security test crash. > > > > I'm going to sort things out with Eclipse unit tests 3.2 crash on > > > > Linux. And to look info failed jdktools test. > > > > > > > > So volunteers are required for: EGAx48, Geronimo, functional, > > > > reliability and stress suites. > > > > > > > > Also we have 2 JIRAs to be resolved for M3 under milstone unmblella[5]: > > > > https://issues.apache.org/jira/browse/HARMONY-4844 > > > > https://issues.apache.org/jira/browse/HARMONY-4810 > > > > > > > > [1] > > > > http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html > > > > [2] > > > > http://people.apache.org/~mloenko/snapshot_testing/script/r551077/index.html > > > > [3] http://wiki.apache.org/harmony/milestones/M2 > > > > [4] > > > > http://mail-archives.apache.org/mod_mbox/harmony-dev/200706.mbox/[EMAIL > > > > PROTECTED] > > > > [5] https://issues.apache.org/jira/browse/HARMONY-4843 > > > > > > > > Thanks, > > > > Stepan. > > > > > > > > > > > >> Please feel free to pick up any issue for investigation. For example, > > > >> 2 pack200 classlib test failed. Also there is a crash of > > > >> org.apache.harmony.security.tests.java.security.cert.serialization.CertificateTest > > > >> on Linux x86 > > > >> > > > >> BTW, should we consider BTI's workspace (that we use for M3 testing) > > > >> frozen too? > > > >> > > > >> [1] > > > >> http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html > > > >> > > > >> Thanks, > > > >> Stepan. > > > > > > > > > > > > > -- > > Thanks, > > Andrey > -- Thanks, Andrey
