I'd like to note about 1 "regression" in *DRLVM tests 99.98% (4350)<http://people.apache.org/~smishura/r580985/Windows_x86/drlvm-test> *
details are in http://people.apache.org/~smishura/r580985/Windows_x86/drlvm-test/ Exit code: -1 Err: ... Stack trace: 0: jthread_detach (??:-1)<end of stack trace> I confirm it is not a regression actually but a known intermitttent failure. The patch from HARMONY-4830<http://issues.apache.org/jira/browse/HARMONY-4830> is to fix it, still we may release M3 w/o this patch being committed. Thanks Vladimir Beliaev 2007/10/2, Mikhail Loenko <[EMAIL PROTECTED]>: > > Do we see any M3 blockers so far? > > Thanks, > Mikhail > > 2007/10/2, Stepan Mishura <[EMAIL PROTECTED]>: > > Hi, > > > > Currently, the next milestone candidate (r580985) is under testing. > > > > There were a number of investigation reports for r579330 and I'd like > > to update its summary to track the progress. All > > > > * Class library: > > - Windows: 1 crash (hard to reproduce) > > - Linux: 1 crash (HARMONY-4873), 1 failure (incorrect test - > HARMONY-4872) > > * Eclipse 3.2 unit tests: > > - Windows: > > 4 failures - 1 is expected, 3 hard to reproduce (exclude list > > was updated) > > - Linux: > > issue with > > org.eclipse.jdt.core.tests.compiler.regression.TestAllsuite > > timeout still remains. > > * Eclipse 3.3 unit tests: > > Success rate on Windows - 99.77% and Linux 99.46%. > > This results will be taken as baseline for tracking regressions. > > * Eclipse Geronimo Application (EGA) scenario x 48h > > - Windows: > > failed after 8 running hours, most probably the problem > > with scenario automation - no crashes/exceptions in the output > > * Eclipse TPTP Tests: > > - Linux: 1 intermittent failure > > * Functional Test Suite: > > 4 regressions were found - HARMONY-4866, HARMONY-4874, > > HARMONY-4857, HARMONY-4861 > > 6 tests are invalid - HARMONY-4864(2 cases), HARMONY-4859 (1 > > case), JIRA for others?? > > HARMONY-4867 - should be defined whether this is regression or not > > Also there are many failures due to timeout or intermittent failures. > > The suite definitely must be improved. > > * Geronimo Unit Tests > > 2 regressions on Windows/Linux - no evaluation yet > > * JDKTools tests: > > Windows: 1 failure - reproducible only if the whole suite is run. > > (HARMONY-4868) > > * Reliability Test Suite: > > - Windows: 13 failures - according to the evaluation report most > > of them are intermittent and additional investigation is required > > - Linux: 14 failures - the same as for Windows > > * Stress Test Suite: > > - Windows: 3 failures > > - Linux: 11 failures > > Most of test failures due to timeout or OutOfMemoryError - the > > suite must be improved. There is HARMONY-4114 with 1Mb patch for the > > suite and harness but it is too risky to apply it during code freeze - > > will be applied in M4. > > * VTS VM Test Suite > > - Linux: 1 intermittent failure - problems in DRLVM shutdown code > > (HARMONY-????). > > > > Thanks, > > Stepan. > > > > On 9/28/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > Hi all, > > > > > > OK. Here is testing status for r579330 M3 milestone candidate. > > > It is based on testing status page[1] and investigation reports for > > > suites on dev-list. > > > It is provided to decide whether r579330 is OK to be released or we > > > should fix found issues and build next release candidate. > > > > > > Status reporter please correct me if I miss something. > > > > > > I'd like to say that this in this milestone we used BTI for the first > > > time to get 100% of testing results. Some issues in BTI were fixed but > > > there are some still to be resolved, for example instability of > > > several suites, mostly Eclipse unit tests on Linux. > > > > > > > > > The following suites passed on Linux/Windows ia32 platform: Ant > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM tests, > > > DRLVM regression tests, Jetty scenario, Scimark, Struts scenario, > > > Tomcat scenario > > > > > > The the next list of issues we have (per suite): > > > - Classlibrary unit tests: > > > 1 crash on Windows that is hard to reproduce: > > > 1 crash on Linux that is reproducible and it is bug in DRLVM > (HARMONY-????): > > > 1 test failure on Linux (incorrect test HARMONY-????) > > > > > > - Eclipse unit tests 3.2 > > > Windows: 4 failures. 1 failure is expected (exclude list should be > > > updated). 3 failure are not reproducible > > > Linux: crash of > > > org.eclipse.jdt.core.tests.compiler.regression.TestAll. It crashes due > > > timeout > > > > > > - Eclipse unit tests 3.3 > > > Windows: 49 failures, pass rate 99.77%. We haven't tried the suite > > > on M2 so it is a good baseline for M3. > > > Linux: the same problem as for 3.2. I've setup another CC Linux > > > host for the suite and see no crashes, pass rate is 99.45% > > > > > > So I'm going to publish these reports for M3 and run 3.2 suite on > > > these lucky host to obtain results without the crash. > > > > > > - Eclipse Geronimo Application (EGA) scenario x 48h > > > Windows: there may be and issue with the scenario automation - the > > > scenario run more then 8 hours. As I understand several iterations > > > were done successfully. And it failed because of missing window, there > > > are no crash/exceptions in the log. > > > > > > Also the scenario passed on Linux. > > > > > > - Eclipse TPTP Tests > > > 1 failure on Linux - most probably intermittent > > > > > > - Functional Test Suite > > > 21 failures on Windows most of then failure due to timeout. 1 > > > regression (HARMONY-4857) > > > Uploading Linux failed - I'm going to check why. From the first > > > glance the results are same as for Windows. > > > > > > - Geronimo Unit Tests > > > 2 possible(I haven't seen analysis report) regressions on > > > Windows/Linux. These test passed on M2: > > > org.apache.geronimo.tomcat.ContainerTest > > > org.apache.geronimo.tomcat.JAASSecurityTest > > > > > > - JDKTools Tests > > > 1 failure on Windows. The test stably fails when it runs in a bunch > > > with others and may fail if run standalone : > > > > > > - Reliability Test Suite > > > The status is a bit unclear to me. There is 1 test is incorrect > > > test. And some tests failed because of timeout but its increase didn't > > > help. > > > Also some tests were removed from exclude list for r579330 but some > > > of them failed. > > > > > > - Stress Test Suite > > > 3 failures on Windows and 11 on Linux - I haven't seen analysis > report > > > > > > - VTS VM Test Suite > > > 1 failure on Linux - the test passed on previous snapshot should be > analysed. > > > > > > Any comments are welcome. > > > > > > [1] > http://people.apache.org/~mloenko/snapshot_testing/script/r579330/index.html > > > > > > Thanks, > > > Stepan Mishura > > > Intel Enterprise Solutions Software Division > > > -- Vladimir Beliaev Intel Middleware Products Division
