This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 60d68034e6a7 [SPARK-50187][INFRA] Use latest Python dependencies for
the build of Python client 3.5 <> 4.0 server
60d68034e6a7 is described below
commit 60d68034e6a7eb4ebade4217b398c95bd8b06eb2
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Thu Oct 31 17:04:21 2024 +0900
[SPARK-50187][INFRA] Use latest Python dependencies for the build of Python
client 3.5 <> 4.0 server
### What changes were proposed in this pull request?
This PR proposes to use the same latest Python dependencies in Python
client 3.5 <> 4.0 server build. This is a sort of a followup of
https://github.com/apache/spark/pull/48524.
### Why are the changes needed?
https://github.com/apache/spark/actions/runs/11543780375/job/32128570166
fails apparently because of different grpc versions. While it is a legitimate
failure to fix, that CI does not target to test all different dependency
versions. It only targets 3.5cleint <> 4.0server. We should better make the
test on its purpose.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Will monitor the jobs
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #48719 from HyukjinKwon/SPARK-50015-followup.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.github/workflows/build_python_connect35.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/build_python_connect35.yml
b/.github/workflows/build_python_connect35.yml
index d4eb6d02a286..276c9ad9d2f9 100644
--- a/.github/workflows/build_python_connect35.yml
+++ b/.github/workflows/build_python_connect35.yml
@@ -70,7 +70,7 @@ jobs:
pip install 'numpy==1.25.1' 'pyarrow==12.0.1' 'pandas<=2.0.3' scipy
unittest-xml-reporting plotly>=4.8 'mlflow>=2.3.1' coverage 'matplotlib==3.7.2'
openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*'
# Add Python deps for Spark Connect.
- pip install 'grpcio>=1.48,<1.57' 'grpcio-status>=1.48,<1.57'
'protobuf==3.20.3' 'googleapis-common-protos==1.56.4' 'graphviz==0.20.3'
+ pip install 'grpcio==1.67.0' 'grpcio-status==1.67.0'
'protobuf==5.28.3' 'googleapis-common-protos==1.65.0' 'graphviz==0.20.3'
# Add torch as a testing dependency for TorchDistributor
pip install 'torch==2.0.1' 'torchvision==0.15.2' torcheval
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
