> Thanks for capturing the requirements. IMO the requirements still lack
> good tests to ensure stability and reliability. I propose to include
> the following recently accepted tests:
>
> 1. VM validation test suite -
http://issues.apache.org/jira/browse/HARMONY-3206
> 2. Functional test suite - http://issues.apache.org/jira/browse/HARMONY-3528
> 3. Stress tests - http://issues.apache.org/jira/browse/HARMONY-3536
> 4. Reliability test - https://issues.apache.org/jira/browse/HARMONY-2918
>
> Olegs seems proposed a reasonable target for the tests- 98% pass rate
> of all valid tests from these test suites.
what is the % today? where can I find these numbers to make sure 98% is real?
Here are some approximate numbers I have (without details with some
probability of mistake) to estimate how far we are from 98% pass
rates:
Reliability test suite: valid 83, pass 59 (0,5 hour per test), fail
24, rate: 71%
Functional: valid 1328, pass 1254, fail 74, rate 94%
Stress: valid 376, pass 245, fail 131, rate 65%
VM validation: valid 4346, pass 4263, fail 83, rate 98%
TPTP profiler tests: valid 174, pass 174, rate 100%
Geronimo unit tests: valid 616, pass 603, fail 13, rate 97,8%
Eclipse unit tests (wrong test suites patched): valid 36642, pass
35598, fail 1044, rate 97%
Harmony class library tests: total ~24000, fail ~300, rate 98,8%
