Pavlenko, Andrey A wrote: > A few comments concerning Functional Test Suite. > Actually I've found 4 regressions: > > HARMONY-4866 - fixed by Mikhail > Fursov<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mfursov> > . > HARMONY-4874 - fixed by Mikhail > Fursov<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mfursov> > . > HARMONY-4857 - Alexey > Varlamov<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=varlax>is > working on this issue > HARMONY-4861 - I've just filed a fix. Could someone commit it please?
I'll start a new thread to get support for it... Regards, Tim > On 9/28/07, Alexei Zakharov <[EMAIL PROTECTED]> wrote: >> A short summary on classlib tests. >> >>> - Class library unit tests: >>> 1 crash on Windows that is hard to reproduce: >> Was reported as >> org.apache.harmony.security.tests.asn1.der.DerUTCTimeEDTest crash. >> Unfortunately I'm unable to reproduce it anymore, I've tried several >> times. >> >>> 1 crash on Linux that is reproducible and it is bug in DRLVM >> (HARMONY-????): >> >> Was recognized as a DRLVM issue, filed as HARMONY-4873. >> >>> 1 test failure on Linux (incorrect test HARMONY-????) >> The tests.api.java.net.MulticastSocketTest failure. I'm unable to >> reproduce it. May be it can be reproduced only on heavy-loaded >> machines. Tim has filed HARMONY-4872 about one of the possible reasons >> of this. In general, I don't think this is a blocking issue for >> releasing M3. >> >> With Best Regards, >> Alexei >> >> 2007/9/28, Stepan Mishura <[EMAIL PROTECTED]>: >>> Hi all, >>> >>> OK. Here is testing status for r579330 M3 milestone candidate. >>> It is based on testing status page[1] and investigation reports for >>> suites on dev-list. >>> It is provided to decide whether r579330 is OK to be released or we >>> should fix found issues and build next release candidate. >>> >>> Status reporter please correct me if I miss something. >>> >>> I'd like to say that this in this milestone we used BTI for the first >>> time to get 100% of testing results. Some issues in BTI were fixed but >>> there are some still to be resolved, for example instability of >>> several suites, mostly Eclipse unit tests on Linux. >>> >>> >>> The following suites passed on Linux/Windows ia32 platform: Ant >>> Scenario (or self-hosting), Axis application, Dacapo, DRLVM tests, >>> DRLVM regression tests, Jetty scenario, Scimark, Struts scenario, >>> Tomcat scenario >>> >>> The the next list of issues we have (per suite): >>> - Class library unit tests: >>> 1 crash on Windows that is hard to reproduce: >>> 1 crash on Linux that is reproducible and it is bug in DRLVM >> (HARMONY-????): >>> 1 test failure on Linux (incorrect test HARMONY-????) >>> >>> - Eclipse unit tests 3.2 >>> Windows: 4 failures. 1 failure is expected (exclude list should be >>> updated). 3 failure are not reproducible >>> Linux: crash of >>> org.eclipse.jdt.core.tests.compiler.regression.TestAll. It crashes due >>> timeout >>> >>> - Eclipse unit tests 3.3 >>> Windows: 49 failures, pass rate 99.77%. We haven't tried the suite >>> on M2 so it is a good baseline for M3. >>> Linux: the same problem as for 3.2. I've setup another CC Linux >>> host for the suite and see no crashes, pass rate is 99.45% >>> >>> So I'm going to publish these reports for M3 and run 3.2 suite on >>> these lucky host to obtain results without the crash. >>> >>> - Eclipse Geronimo Application (EGA) scenario x 48h >>> Windows: there may be and issue with the scenario automation - the >>> scenario run more then 8 hours. As I understand several iterations >>> were done successfully. And it failed because of missing window, there >>> are no crash/exceptions in the log. >>> >>> Also the scenario passed on Linux. >>> >>> - Eclipse TPTP Tests >>> 1 failure on Linux - most probably intermittent >>> >>> - Functional Test Suite >>> 21 failures on Windows most of then failure due to timeout. 1 >>> regression (HARMONY-4857) >>> Uploading Linux failed - I'm going to check why. From the first >>> glance the results are same as for Windows. >>> >>> - Geronimo Unit Tests >>> 2 possible(I haven't seen analysis report) regressions on >>> Windows/Linux. These test passed on M2: >>> org.apache.geronimo.tomcat.ContainerTest >>> org.apache.geronimo.tomcat.JAASSecurityTest >>> >>> - JDKTools Tests >>> 1 failure on Windows. The test stably fails when it runs in a bunch >>> with others and may fail if run standalone : >>> >>> - Reliability Test Suite >>> The status is a bit unclear to me. There is 1 test is incorrect >>> test. And some tests failed because of timeout but its increase didn't >>> help. >>> Also some tests were removed from exclude list for r579330 but some >>> of them failed. >>> >>> - Stress Test Suite >>> 3 failures on Windows and 11 on Linux - I haven't seen analysis report >>> >>> - VTS VM Test Suite >>> 1 failure on Linux - the test passed on previous snapshot should be >> analysed. >>> Any comments are welcome. >>> >>> [1] >> http://people.apache.org/~mloenko/snapshot_testing/script/r579330/index.html >>> Thanks, >>> Stepan Mishura >>> Intel Enterprise Solutions Software Division >
