A few comments concerning Functional Test Suite. Actually I've found 4 regressions:
HARMONY-4866 - fixed by Mikhail Fursov<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mfursov> . HARMONY-4874 - fixed by Mikhail Fursov<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mfursov> . HARMONY-4857 - Alexey Varlamov<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=varlax>is working on this issue HARMONY-4861 - I've just filed a fix. Could someone commit it please? On 9/28/07, Alexei Zakharov <[EMAIL PROTECTED]> wrote: > > A short summary on classlib tests. > > > - Class library unit tests: > > 1 crash on Windows that is hard to reproduce: > > Was reported as > org.apache.harmony.security.tests.asn1.der.DerUTCTimeEDTest crash. > Unfortunately I'm unable to reproduce it anymore, I've tried several > times. > > > 1 crash on Linux that is reproducible and it is bug in DRLVM > (HARMONY-????): > > Was recognized as a DRLVM issue, filed as HARMONY-4873. > > > 1 test failure on Linux (incorrect test HARMONY-????) > > The tests.api.java.net.MulticastSocketTest failure. I'm unable to > reproduce it. May be it can be reproduced only on heavy-loaded > machines. Tim has filed HARMONY-4872 about one of the possible reasons > of this. In general, I don't think this is a blocking issue for > releasing M3. > > With Best Regards, > Alexei > > 2007/9/28, Stepan Mishura <[EMAIL PROTECTED]>: > > Hi all, > > > > OK. Here is testing status for r579330 M3 milestone candidate. > > It is based on testing status page[1] and investigation reports for > > suites on dev-list. > > It is provided to decide whether r579330 is OK to be released or we > > should fix found issues and build next release candidate. > > > > Status reporter please correct me if I miss something. > > > > I'd like to say that this in this milestone we used BTI for the first > > time to get 100% of testing results. Some issues in BTI were fixed but > > there are some still to be resolved, for example instability of > > several suites, mostly Eclipse unit tests on Linux. > > > > > > The following suites passed on Linux/Windows ia32 platform: Ant > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM tests, > > DRLVM regression tests, Jetty scenario, Scimark, Struts scenario, > > Tomcat scenario > > > > The the next list of issues we have (per suite): > > - Class library unit tests: > > 1 crash on Windows that is hard to reproduce: > > 1 crash on Linux that is reproducible and it is bug in DRLVM > (HARMONY-????): > > 1 test failure on Linux (incorrect test HARMONY-????) > > > > - Eclipse unit tests 3.2 > > Windows: 4 failures. 1 failure is expected (exclude list should be > > updated). 3 failure are not reproducible > > Linux: crash of > > org.eclipse.jdt.core.tests.compiler.regression.TestAll. It crashes due > > timeout > > > > - Eclipse unit tests 3.3 > > Windows: 49 failures, pass rate 99.77%. We haven't tried the suite > > on M2 so it is a good baseline for M3. > > Linux: the same problem as for 3.2. I've setup another CC Linux > > host for the suite and see no crashes, pass rate is 99.45% > > > > So I'm going to publish these reports for M3 and run 3.2 suite on > > these lucky host to obtain results without the crash. > > > > - Eclipse Geronimo Application (EGA) scenario x 48h > > Windows: there may be and issue with the scenario automation - the > > scenario run more then 8 hours. As I understand several iterations > > were done successfully. And it failed because of missing window, there > > are no crash/exceptions in the log. > > > > Also the scenario passed on Linux. > > > > - Eclipse TPTP Tests > > 1 failure on Linux - most probably intermittent > > > > - Functional Test Suite > > 21 failures on Windows most of then failure due to timeout. 1 > > regression (HARMONY-4857) > > Uploading Linux failed - I'm going to check why. From the first > > glance the results are same as for Windows. > > > > - Geronimo Unit Tests > > 2 possible(I haven't seen analysis report) regressions on > > Windows/Linux. These test passed on M2: > > org.apache.geronimo.tomcat.ContainerTest > > org.apache.geronimo.tomcat.JAASSecurityTest > > > > - JDKTools Tests > > 1 failure on Windows. The test stably fails when it runs in a bunch > > with others and may fail if run standalone : > > > > - Reliability Test Suite > > The status is a bit unclear to me. There is 1 test is incorrect > > test. And some tests failed because of timeout but its increase didn't > > help. > > Also some tests were removed from exclude list for r579330 but some > > of them failed. > > > > - Stress Test Suite > > 3 failures on Windows and 11 on Linux - I haven't seen analysis report > > > > - VTS VM Test Suite > > 1 failure on Linux - the test passed on previous snapshot should be > analysed. > > > > Any comments are welcome. > > > > [1] > http://people.apache.org/~mloenko/snapshot_testing/script/r579330/index.html > > > > Thanks, > > Stepan Mishura > > Intel Enterprise Solutions Software Division >
