This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d16b128d8215 [SPARK-54245][INFRA] Install `mlflow` at Python 3.14
Docker image
d16b128d8215 is described below
commit d16b128d8215a3a6ac75a84d91cf6ef24a55e6e5
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Fri Nov 7 17:50:58 2025 -0800
[SPARK-54245][INFRA] Install `mlflow` at Python 3.14 Docker image
### What changes were proposed in this pull request?
This PR aims to install `mlflow` at Python 3.14 Docker image.
### Why are the changes needed?
MLFlow 3.6.0 is released with Python 3.14 support via PyArrow 22.0.0.
- https://pypi.org/project/mlflow/3.6.0/
Like the other images, we will add the same condition `mlflow>=2.8.1`, but
it will install `MLFlow>=3.6.0` effectively.
https://github.com/apache/spark/blob/a871ba4464e07cb0229b1289b125bb07da7b6265/dev/spark-test-image/python-313/Dockerfile#L71
### Does this PR introduce _any_ user-facing change?
No behavior change.
### How was this patch tested?
Manual review.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #52948 from dongjoon-hyun/SPARK-54245.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/spark-test-image/python-314/Dockerfile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/dev/spark-test-image/python-314/Dockerfile
b/dev/spark-test-image/python-314/Dockerfile
index 236e95e3f52c..07916fc35a0d 100644
--- a/dev/spark-test-image/python-314/Dockerfile
+++ b/dev/spark-test-image/python-314/Dockerfile
@@ -68,7 +68,7 @@ RUN apt-get update && apt-get install -y \
&& rm -rf /var/lib/apt/lists/*
-ARG BASIC_PIP_PKGS="numpy pyarrow>=22.0.0 six==1.16.0 pandas==2.3.3 scipy
plotly<6.0.0 coverage matplotlib openpyxl memory-profiler>=0.61.0
scikit-learn>=1.3.2"
+ARG BASIC_PIP_PKGS="numpy pyarrow>=22.0.0 six==1.16.0 pandas==2.3.3 scipy
plotly<6.0.0 mlflow>=2.8.1 coverage matplotlib openpyxl memory-profiler>=0.61.0
scikit-learn>=1.3.2"
# Python deps for Spark Connect
ARG CONNECT_PIP_PKGS="grpcio==1.76.0 grpcio-status==1.76.0 protobuf==6.33.0
googleapis-common-protos==1.71.0 graphviz==0.20.3"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
