This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4f8d1f575e99 [SPARK-50394][PYTHON][INFRA][FOLLOW-UP] Reduce
parallelism further in Pure Python library builds
4f8d1f575e99 is described below
commit 4f8d1f575e99aeef8990c63a9614af0fc5479330
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Sun Nov 24 17:31:22 2024 -0800
[SPARK-50394][PYTHON][INFRA][FOLLOW-UP] Reduce parallelism further in Pure
Python library builds
### What changes were proposed in this pull request?
This PR is a followup of https://github.com/apache/spark/pull/48932 that
reduces parallelism further
### Why are the changes needed?
In order to make the tests more robust:
https://github.com/apache/spark/actions/workflows/build_python_connect35.yml
It fails because of OOM.
FWIW,
https://github.com/apache/spark/actions/workflows/build_python_connect.yml is
fixed by the original PR.
### Does this PR introduce _any_ user-facing change?
No, test-only.
### How was this patch tested?
Will monitor the build.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #48948 from HyukjinKwon/SPARK-50394-followup.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/build_python_connect35.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/build_python_connect35.yml
b/.github/workflows/build_python_connect35.yml
index 4b7a6b82b952..ad250d95fb84 100644
--- a/.github/workflows/build_python_connect35.yml
+++ b/.github/workflows/build_python_connect35.yml
@@ -98,7 +98,7 @@ jobs:
# Run branch-3.5 tests
./python/run-tests --parallelism=1 --python-executables=python3
--modules pyspark-connect
# None of tests are dependent on each other in Pandas API on Spark
so run them in parallel
- ./python/run-tests --parallelism=2 --python-executables=python3
--modules pyspark-pandas-connect,pyspark-pandas-slow-connect
+ ./python/run-tests --parallelism=1 --python-executables=python3
--modules pyspark-pandas-connect,pyspark-pandas-slow-connect
- name: Upload test results to report
if: always()
uses: actions/upload-artifact@v4
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
