This is incredibly useful. Thanks. We need to incorporate this as a
possible target in CC.
geir
tatyana doubtsova wrote:
I created lists of intermittent unit tests failures on Win XP in perTest
mode:
http://issues.apache.org/jira/browse/HARMONY-2282 - swing module
http://issues.apache.org/jira/browse/HARMONY-2284 - security module
Could anybody take a look at it?
I did not observe intermittent failures in other modules on Win XP so far.
I'm going to continue iterative classlib tests runs.
Thanks,
Tanya
On 11/22/06, Alexei Fedotov <[EMAIL PROTECTED]> wrote:
Tatiana,
That's great! Feel free to file JIRA issues about new problems.
This is quite interesting that you haven't discovered
org.apache.harmony.logging.tests.java.util.logging.SocketHandlerTest
failure using effectively the same revision as I. This makes me think
that my problems can be local to my computer.
--
Thank you,
Alexei
On 11/21/06, tatyana doubtsova <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I've run classlib unit tests on Win XP in Other VM mode 50 times for
each
> module in order to reveal sporadic failures.
>
>
>
> The script is
>
>
>
> #!sh
>
> cat modulesList | while read moduleName; do
>
> A=0
>
> while [ "$A" -lt "50" ]; do
>
> echo ITERATION $A
>
> ant -
>
Dtest.jre.home=c:/users/tdoubtso/harmony/drlvm/trunk/build/win_ia32_msvc_debug/deploy/jre\
>
> -Dbuild.module=$moduleName -
> Dhy.test.forkmode=perTest \
>
> test 2>&1 | tee log_${moduleName}_${A}.txt
>
> cp -rf build/test_report
build/test_report.${moduleName}_${A}
>
> A=`expr $A + 1`
>
> done
>
> done
>
>
>
>
>
> $ cat moduleList
>
>
>
> accessibility
>
> archive
>
> auth
>
> awt
>
> beans
>
> crypto
>
> instrument
>
> jndi
>
> lang-management
>
> logging
>
> luni
>
> math
>
> nio
>
> nio_char
>
> prefs
>
> print
>
> regex
>
> rmi
>
> security
>
> sound
>
> sql
>
> swing
>
> text
>
> tools
>
> x-net
>
>
>
>
>
> I've observed the following failures on svn=r476124
>
>
>
>
>
> *Module*
>
> *Test*
>
> *Number of failures*
>
> *issue*
>
> luni
>
>
org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectionTest
>
> 50 out of 50
>
> 1879 <http://issues.apache.org/jira/browse/HARMONY-1879>, updated
>
> luni
>
> tests.api.java.net.InetAddressTest
>
> 50 out of 50
>
> Test was eliminated from repository after Novenber,17
>
> prefs
>
> org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest
>
> 50 out of 50
>
> 1751 <http://issues.apache.org/jira/browse/HARMONY-1751>
>
> swing
>
> javax.swing.text.WrappedPlainView_SimpleTest
>
> 1 time out of 50
>
> 1)
>
> swing
>
> javax.swing.SpinnerDateModelTest
>
> 35 times out of 50
>
> 2)
>
>
>
>
> 1). Eror
>
> Timeout occurred
>
> junit.framework.AssertionFailedError: Timeout occurred
>
>
>
> 2). Failure
>
> expected:<0> but was:<7>
>
> junit.framework.AssertionFailedError: expected:<0> but was:<7> at
> javax.swing.SpinnerDateModelTest.testSpinnerDateModel(
> SpinnerDateModelTest.java:59) at
> java.lang.reflect.VMReflection.invokeMethod(Native
> Method) at javax.swing.BasicSwingTestCase.runBareSuper(
> BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase.runBareImpl(
> BasicSwingTestCase.java) at javax.swing.BasicSwingTestCase$1.run(
> BasicSwingTestCase.java) at java.lang.Thread.run(Unknown Source)
>
>
>
> 2070 <http://issues.apache.org/jira/browse/HARMONY-2070> was not
reproduced
> during this test run
>
>
>
> I can't reproduce 1) and 2) today on svn = r477507
>
>
>
> I've excluded failed tests and restarted multiple test run on Win XP in
> order to make sure there is no other sporadic failures.
>
> Also I'm going to start the multiple classlib tests run on SUSE 9
>
>
>
>
>
> To extract the results I've used commands:
>
> cd classlib/trunk/buid
>
> # find errors
>
> find . -name *Test.xml -exec grep -l "<testsuite errors=\"[1-9]*\" " {}
\; >
> ERRORS
>
> # find failures
>
> find . -name *Test.xml -exec grep -l "<testsuite errors=\"*\"
> failures=\"[1-9]*\"" {} \; > FAILURES
>
> find . -type f -size 0 | grep *.xml > ZERO_LENGTH
>
>
>
> Also I've compared the number of files in test_report directories:
>
> #!sh
>
> cat modulesList | while read moduleName; do
>
> echo $moduleName
>
> A=1
>
> while [ "$A" -lt "51" ]; do
>
> L=`ls -1 build/test_report.${moduleName}_${A} | grep xml
| wc
> -l`
>
> echo build/test_report.${moduleName}_${A} $L >>
> test_report_length
>
> A=`expr $A + 1`
>
> done
> done
>
> Thanks,
> Tanya
>
>
