Michael Armbrust created SPARK-3534:
---------------------------------------
Summary: Avoid running MLlib and Streaming tests when testing SQL
PRs
Key: SPARK-3534
URL: https://issues.apache.org/jira/browse/SPARK-3534
Project: Spark
Issue Type: Bug
Components: Project Infra, SQL
Reporter: Michael Armbrust
Priority: Blocker
We are bumping up against the 120 minute time limit for tests pretty regularly
now. Since we have decreased the number of shuffle partitions and up-ed the
parallelism I don't think there is much low hanging fruit to speed up the SQL
tests. (The tests that are listed as taking 2-3 minutes are actually 100s of
tests that I think are valuable). Instead I propose we avoid running tests
that we don't need to.
This will have the added benefit of eliminating failures in SQL due to flaky
streaming tests.
Note that this won't fix the full builds that are run for every commit. There
I think we just just up the test timeout.
cc: [~joshrosen] [~pwendell]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
