[
https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220065#comment-14220065
]
Josh Rosen commented on SPARK-3431:
-----------------------------------
[~nchammas]: It's been a while since I tried that patch, so I don't remember
offhand, but it seemed significantly faster. Tests completed ~20 minutes-ish
on my laptop, maybe? Thats just a guess though; I could be misremembering.
The port contention issues should have been solved; even without
parallelization in Maven, we still run into the risk of multiple Jenkins builds
running on the same box contending for ports. AFAIK we haven't seen any recent
failures due to port contention, so I think it should be safe to increase the
degree of parallelism.
> Parallelize execution of tests
> ------------------------------
>
> Key: SPARK-3431
> URL: https://issues.apache.org/jira/browse/SPARK-3431
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Reporter: Nicholas Chammas
>
> Running all the tests in {{dev/run-tests}} takes up to 2 hours. A common
> strategy to cut test time down is to parallelize the execution of the tests.
> Doing that may in turn require some prerequisite changes to be made to how
> certain tests run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
