[
https://issues.apache.org/jira/browse/LUCENE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Weiss updated LUCENE-9472:
--------------------------------
Description:
I toyed with this a bit because it's hacker's delight... Forked the default
Gradle test task and made it emit the same test/ suite name multiple times.
This results in the same test being executed multiple times:
{code}
# ./gradlew -p lucene\analysis\kuromoji testy --tests
TestToStringUtil.testHepburn -Ptests.dups=10
> Task :randomizationInfo
Running tests with randomization seed: tests.seed=88C21B0E8ABB2D46
> Task :lucene:analysis:kuromoji:testy
:lucene:analysis:kuromoji:testy (SUCCESS): 10 test(s)
BUILD SUCCESSFUL in 13s
21 actionable tasks: 6 executed, 15 up-to-date
{code}
Upsides:
* Everything configures itself because the custom task is a subtype of Gradle's
Test task,
* Tests are run in parallel, on multiple VMs,
* Opens up the possibility of reordering test suites (for load-balancing).
Downsides:
* Requires a custom(ized) clone of Gradle's internal classes. Will require
maintenance when upgrading gradle. There is no certainty it'll work.
* Some things visibly don't like duplicated suite names - the progress bar
shows the suite once, even though it is executed concurrently in multiple VMs,
* XML reports will drive jenkins and other tooling crazy:
{code}
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="org.apache.lucene.analysis.ja.util.TestToStringUtil"
tests="10" skipped="0" failures="0" errors="0" timestamp="2020-08-20T12:54:27"
hostname="DWEISS-DESKTOP" time="3.177">
<properties/>
<testcase name="testHepburn"
classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.04"/>
<testcase name="testHepburn"
classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.045"/>
<testcase name="testHepburn"
classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.043"/>
<testcase name="testHepburn"
classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.038"/>
<testcase name="testHepburn"
classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.008"/>
<testcase name="testHepburn"
classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.005"/>
....
{code}
* not sure what happens with sysouts from such concurrent runs...
> True concurrent test suite duplication and execution (beasting)
> ---------------------------------------------------------------
>
> Key: LUCENE-9472
> URL: https://issues.apache.org/jira/browse/LUCENE-9472
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Dawid Weiss
> Priority: Major
>
> I toyed with this a bit because it's hacker's delight... Forked the default
> Gradle test task and made it emit the same test/ suite name multiple times.
> This results in the same test being executed multiple times:
> {code}
> # ./gradlew -p lucene\analysis\kuromoji testy --tests
> TestToStringUtil.testHepburn -Ptests.dups=10
> > Task :randomizationInfo
> Running tests with randomization seed: tests.seed=88C21B0E8ABB2D46
> > Task :lucene:analysis:kuromoji:testy
> :lucene:analysis:kuromoji:testy (SUCCESS): 10 test(s)
> BUILD SUCCESSFUL in 13s
> 21 actionable tasks: 6 executed, 15 up-to-date
> {code}
> Upsides:
> * Everything configures itself because the custom task is a subtype of
> Gradle's Test task,
> * Tests are run in parallel, on multiple VMs,
> * Opens up the possibility of reordering test suites (for load-balancing).
> Downsides:
> * Requires a custom(ized) clone of Gradle's internal classes. Will require
> maintenance when upgrading gradle. There is no certainty it'll work.
> * Some things visibly don't like duplicated suite names - the progress bar
> shows the suite once, even though it is executed concurrently in multiple VMs,
> * XML reports will drive jenkins and other tooling crazy:
> {code}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite name="org.apache.lucene.analysis.ja.util.TestToStringUtil"
> tests="10" skipped="0" failures="0" errors="0"
> timestamp="2020-08-20T12:54:27" hostname="DWEISS-DESKTOP" time="3.177">
> <properties/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.04"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.045"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.043"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.038"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.008"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.005"/>
> ....
> {code}
> * not sure what happens with sysouts from such concurrent runs...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
