[
https://issues.apache.org/jira/browse/SOLR-14857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris M. Hostetter updated SOLR-14857:
--------------------------------------
Attachment: SOLR-14857.patch
Status: Open (was: Open)
I took a stab at implementing this...
The gradle {{WorkerExecutor}} is that lovely combination of seemingly simple,
extremely bare bones in functionality, and has confusingly undocumented
limitations.
Most problematic: there doesn't seem to be any way to control the level of
parallelization independently of the project level {{org.gradle.workers.max}}
setting – and while I typically run with {{org.gradle.workers.max=5}} on my
laptop, when each of those workers is spinning up docker containers running
full JVMs, that pretty much crushes my box, and causes multiple (docker) test
failures – most seem to be because background tasks (like starting solr, or
committing documents) don't finish as quickly as the bash test scripts expect.
One thing that can be done fairly easily is to {{await()}} on the {{WorkQueue}}
– which can be done after every task is submitted as quick& dirty way to ensure
only one test is run at a time, so that's how I implemented this as an optional
feature...
{noformat}
# default: run tests one at a time (regardless of `org.gradle.workers.max`) ...
./gradlew testDocker
# optional: run concurrent tests, up to `org.gradle.workers.max`
./gradlew testDocker -Dorg.gradle.workers.max=3
-Psolr.docker.tests.concurrent=true
{noformat}
As part of this work, I changed how the bash scripts are invoked by gradle so
that the stdout/stderr is always written to test case specific files, never to
the gradle console. (but if any tests fail, the name of the test case work dir,
which includes those files, is mentioned in the failure message)
[~houston] - WDYT?
> Parallelize Solr Docker tests
> -----------------------------
>
> Key: SOLR-14857
> URL: https://issues.apache.org/jira/browse/SOLR-14857
> Project: Solr
> Issue Type: Improvement
> Components: Docker, Tests
> Affects Versions: main (9.0)
> Reporter: Houston Putman
> Priority: Minor
> Attachments: SOLR-14857.patch
>
>
> Currently the Solr docker tests are run concurrently. The contexts and image
> names are all unique, so there is no reason they can't be run in parallel.
> I would imagine that this would be a configuration passed to the gradle task,
> which already has command line options for specifying tests to skip or tests
> to run.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
