On 9/25/07, Stepan Mishura wrote: <SNIP> > I've compared testing status for the last snapshot r578410 [1] with M2 [2][3]. > > We have the following short status for failed suites: > > Windows_x86: > * classlib: 2 tests from pack200 module fail on snapshot (but pass > on debug build) > * Eclipse unit tests 3.2: there is no tests report like for M3. The > pass rate was improved from 99.32%[3] to 99.7%[1] > * Eclipse unit tests 3.3 are new and the pass rate is 99.77%. I > think is acceptable > * EGA x48 hours scenario fails. According to [4] it passed on M2. > * Functional: need more analysis, currently I see that 2 tests were > enabled and new 15 regressions. > * Geronimo: 2 regressions > * JDK tools: 1 test failed. It might be intermediate failure - the > test failed due to timeout > * Reliability: 65 tests passed for M2 and 64 for M3. Investigation > is required. > * Stress: 190 tests passed for M2 and 189 for M3. Investigation is required. > > Linux_x86: > * classlib: 2 tests from pack200 (as for Windows), 1 luni tests > failed and 1 crash of security test > * Eclipse unit tests 3.2: 2 suites crashed so pass rate is 69.60%. I > assume this may be CC host configuration issue. Going to investigate.
I'm looking into EUT 3.2 results on Linux_x86 for r579330. The good news that crashes for the next suites disappeared: - org.eclipse.jdt.core.tests.dom.RunAllTests - org.eclipse.jdt.core.tests.model.AllJavaModelTests And the bad news there is one new suite crash: - org.eclipse.jdt.core.tests.compiler.regression.TestAll This results are pretty confusing for me. I believe that last updates to classlib/drlvm/jdktool shouldn't influence on results in such dramatic way. I have to admit that CC host configuration was changed a bit. There is an assumption that the tests are sensitive to X-server. (currently VNC is used). So X-server (SLES9) was configured on CC host and runlevel was changed from 3 to 5. But IMHO it shouldn't affect current CC configuration (I didn't change it. i.e. VNC was used as usual) and testing result as well. Am I missing something? Thanks, Stepan. > * Eclipse unit tests 3.3 are new and the pass rate is 96.47%. I > think is acceptable > * EGA x48 hours scenario: the same as for Windows (scenario fails on M3) > * Functional: need more analysis, similar to Windows - some test are > passing now but there are new failures. > * Geronimo: 2 regressions (the same as for Windows) > * Reliability: need more analysis > * Stress: need more analysis > > As I remember Sean took pack200 tests. And Alexei Zakharov took > security test crash. > I'm going to sort things out with Eclipse unit tests 3.2 crash on > Linux. And to look info failed jdktools test. > > So volunteers are required for: EGAx48, Geronimo, functional, > reliability and stress suites. > > Also we have 2 JIRAs to be resolved for M3 under milstone unmblella[5]: > https://issues.apache.org/jira/browse/HARMONY-4844 > https://issues.apache.org/jira/browse/HARMONY-4810 > > [1] > http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html > [2] > http://people.apache.org/~mloenko/snapshot_testing/script/r551077/index.html > [3] http://wiki.apache.org/harmony/milestones/M2 > [4] http://mail-archives.apache.org/mod_mbox/harmony-dev/200706.mbox/[EMAIL > PROTECTED] > [5] https://issues.apache.org/jira/browse/HARMONY-4843 > > Thanks, > Stepan. >
