Helllo, Stepan, I confirm it is high probability that there is nothing to be fixed in Harmony for Eclipse Unit Tests now to significantly change the status. I mean the M3 snapshot is pretty "well done" in respect to EUT.
*Still the EUT running script still may be improved*. For example I can redo them to run each EUT Suite separately making sure all resources are cleaned up after each suite run (still this won't cleanup $TMP or $HOME directory because I do not have the exact list of file names created by EUT). Still the current issues wwith *jdtcorecompiler* better be resolved in Eclipse (but not workarounded in Harmony), so I'm not sure it is a good solution. Or I can incorporate to these scripts the creation, specifying and cleanup-ing $HOME and $TMP directories. So, please write if you have proposals for on EUT running scripts. Thanks Vladimir Beliaev 2007/9/28, Stepan Mishura <[EMAIL PROTECTED]>: > Hi all, > > OK. Here is testing status for r579330 M3 milestone candidate. > It is based on testing status page[1] and investigation reports for > suites on dev-list. > It is provided to decide whether r579330 is OK to be released or we > should fix found issues and build next release candidate. > > Status reporter please correct me if I miss something. > > I'd like to say that this in this milestone we used BTI for the first > time to get 100% of testing results. Some issues in BTI were fixed but > there are some still to be resolved, for example instability of > several suites, mostly Eclipse unit tests on Linux. > > > The following suites passed on Linux/Windows ia32 platform: Ant > Scenario (or self-hosting), Axis application, Dacapo, DRLVM tests, > DRLVM regression tests, Jetty scenario, Scimark, Struts scenario, > Tomcat scenario > > The the next list of issues we have (per suite): > - Classlibrary unit tests: > 1 crash on Windows that is hard to reproduce: > 1 crash on Linux that is reproducible and it is bug in DRLVM > (HARMONY-????): > 1 test failure on Linux (incorrect test HARMONY-????) > > - Eclipse unit tests 3.2 > Windows: 4 failures. 1 failure is expected (exclude list should be > updated). 3 failure are not reproducible > Linux: crash of > org.eclipse.jdt.core.tests.compiler.regression.TestAll. It crashes due > timeout > > - Eclipse unit tests 3.3 > Windows: 49 failures, pass rate 99.77%. We haven't tried the suite > on M2 so it is a good baseline for M3. > Linux: the same problem as for 3.2. I've setup another CC Linux > host for the suite and see no crashes, pass rate is 99.45% > > So I'm going to publish these reports for M3 and run 3.2 suite on > these lucky host to obtain results without the crash. > > - Eclipse Geronimo Application (EGA) scenario x 48h > Windows: there may be and issue with the scenario automation - the > scenario run more then 8 hours. As I understand several iterations > were done successfully. And it failed because of missing window, there > are no crash/exceptions in the log. > > Also the scenario passed on Linux. > > - Eclipse TPTP Tests > 1 failure on Linux - most probably intermittent > > - Functional Test Suite > 21 failures on Windows most of then failure due to timeout. 1 > regression (HARMONY-4857) > Uploading Linux failed - I'm going to check why. From the first > glance the results are same as for Windows. > > - Geronimo Unit Tests > 2 possible(I haven't seen analysis report) regressions on > Windows/Linux. These test passed on M2: > org.apache.geronimo.tomcat.ContainerTest > org.apache.geronimo.tomcat.JAASSecurityTest > > - JDKTools Tests > 1 failure on Windows. The test stably fails when it runs in a bunch > with others and may fail if run standalone : > > - Reliability Test Suite > The status is a bit unclear to me. There is 1 test is incorrect > test. And some tests failed because of timeout but its increase didn't > help. > Also some tests were removed from exclude list for r579330 but some > of them failed. > > - Stress Test Suite > 3 failures on Windows and 11 on Linux - I haven't seen analysis report > > - VTS VM Test Suite > 1 failure on Linux - the test passed on previous snapshot should be > analysed. > > Any comments are welcome. > > [1] > http://people.apache.org/~mloenko/snapshot_testing/script/r579330/index.html > > Thanks, > Stepan Mishura > Intel Enterprise Solutions Software Division > -- Vladimir Beliaev Intel Middleware Products Division
