Hi Ivan, I think that the issue is common for all testing suites so I'd like to hear other opinions.
Summary: junit timeout works for per forked VM not for an individual test in a test suite. For example, if each test in a test suite fits in timeout but overall time for testsuite exceeds timeout then testsuite passes when forkmode=perTest and fails with timeout when forkmode=once. (I think that is possible for any classlib module as far as the number of tests is growing). Currently we had such situation for JDPA tests[1] - the test suite run did't fit into timeout so JPDA test suite failed if forkmode=once. The most simple and quick solution was to increase timeout (current timeout=15min). Another way was to split the test suite into sub-suites that was suggested by the JIRA patch (by adding a macrodef to run two junit tasks). I think that timeout value should have reasonable value (not too big) so I desided to split the testsuite but did it in different way - I created two batchtest junit elements (to split the suite) and run tests in forkmode=perBatch. Are there opinions about: 1) Reasonable value for timeout? Should we define any limit for it? 2) Clear and understandable way for splitting the suite: macrodef with several junit tasks OR one junit task and several batchtests elements within. [1] https://issues.apache.org/jira/browse/HARMONY-3319 Thanks, Stepan. On 3/9/07, Ivan Popov (JIRA) <[EMAIL PROTECTED]> wrote:
[ https://issues.apache.org/jira/browse/HARMONY-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12479590] Ivan Popov commented on HARMONY-3319: ------------------------------------- I don't think that your solution with adding special property " hy.jpda.test.forkmode" which value is calculated depending on the value of standard property "hy.test.forkmode" according to not trivial rules: <!-- run JPDA tests with forkmode=perBatch if common forkmode=once --> <condition property="hy.jpda.test.forkmode" value="perBatch" else="perTest"> <equals arg1="${hy.test.forkmode}" arg2="once" /> </condition> is better than was proposed in the patch. You still have to divide tests into two groups, but also introduce not so evident calculation of a special property, which one need to take into account. I think the previous suggestion with dividing tests into two junit tasks is more clear and understandable. > [jdktools][jpda] JDWP unit tests fail with timeout > -------------------------------------------------- > > Key: HARMONY-3319 > URL: https://issues.apache.org/jira/browse/HARMONY-3319 > Project: Harmony > Issue Type: Bug > Components: JDK > Environment: Windows/ia32, Linux/ia32, Harmony-jdk-r515478 > Reporter: Ivan Popov > Assigned To: Stepan Mishura > Attachments: jpda_build.patch > > > JDWP unit tests from JPDA component started with ant build usually fail with timeout: > <...> > [junit] TEST org.apache.harmony.jpda.tests.jdwp.ReferenceType.GetValues003Test FAILED > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 9.578sec > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 9.543sec > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 9.537sec > [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec > [junit] Tests FAILED (timeout) > <...> > full-report: > [junitreport] [Fatal Error] :-1:-1: Premature end of file. > [junitreport] The file .../trunk/working_jdktools/build/test_report/TEST- org.apache.harmony.jpda.tests.jdwp.ReferenceType.InterfacesTest.xml is not a valid XML document. It is possibly corrupted. > The problem is in default timeout (120000 milliseconds, or 15 minutes) specified for junit task in 'timeout" option: > timeout="${hy.test.timeout}" > JUnit task reference reads for this option: > Cancel the individual tests if they don't finish in the given time (measured in milliseconds). Ignored if fork is disabled. > However, it seems that this option is applied to the whole tests set specified in <batchtest> item. All JDWP tests are specified in a single <batchtest> and their execution usually exceeds 15 minutes, so junit task cancels them with timeout. > There are several "long-running" tests, each takes about 1 minute. These tests can be moved to a separate <batchtest> item to satisfy the default timeout. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
-- Stepan Mishura Intel Enterprise Solutions Software Division
