This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 921daa8 [SPARK-33439][INFRA] Use SERIAL_SBT_TESTS=1 for SQL modules
921daa8 is described below
commit 921daa8ef817c56918612eec3e8ea4d9967a33ad
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Nov 12 21:19:51 2020 -0800
[SPARK-33439][INFRA] Use SERIAL_SBT_TESTS=1 for SQL modules
### What changes were proposed in this pull request?
This PR aims to decrease the parallelism of `SQL` module like `Hive` module.
### Why are the changes needed?
GitHub Action `sql - slow tests` become flaky.
- https://github.com/apache/spark/runs/1393670291
- https://github.com/apache/spark/runs/1393088031
### Does this PR introduce _any_ user-facing change?
No. This is dev-only feature.
Although this will increase the running time, but it's better than
flakiness.
### How was this patch tested?
Pass the GitHub Action stably.
Closes #30365 from dongjoon-hyun/SPARK-33439.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit a70a2b02ce7d18947778d37c8fffb3f1b1b5b154)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/build_and_test.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 9b4f41a..574e1bf 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -132,8 +132,8 @@ jobs:
# Run the tests.
- name: Run tests
run: |
- # Hive tests become flaky when running in parallel as it's too
intensive.
- if [[ "$MODULES_TO_TEST" == "hive" ]]; then export SERIAL_SBT_TESTS=1;
fi
+ # Hive and SQL tests become flaky when running in parallel as it's too
intensive.
+ if [[ "$MODULES_TO_TEST" == "hive" ]] || [[ "$MODULES_TO_TEST" ==
"sql" ]]; then export SERIAL_SBT_TESTS=1; fi
mkdir -p ~/.m2
./dev/run-tests --parallelism 2 --modules "$MODULES_TO_TEST"
--included-tags "$INCLUDED_TAGS" --excluded-tags "$EXCLUDED_TAGS"
rm -rf ~/.m2/repository/org/apache/spark
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
