uros-b commented on code in PR #58060: URL: https://github.com/apache/spark/pull/58060#discussion_r3801763921
########## dev/spark-test-image/python-minimum/Dockerfile: ########## @@ -64,8 +64,9 @@ ENV VIRTUAL_ENV=/opt/spark-venv RUN python3.11 -m venv $VIRTUAL_ENV ENV PATH="$VIRTUAL_ENV/bin:$PATH" -ARG BASIC_PIP_PKGS="numpy==1.23.2 pyarrow==18.0.0 pandas==2.2.0 six==1.16.0 scipy scikit-learn coverage unittest-xml-reporting psutil" -ARG CONNECT_PIP_PKGS="grpcio==1.76.0 grpcio-status==1.76.0 googleapis-common-protos==1.71.0 zstandard==0.25.0 graphviz==0.20 protobuf==6.33.5" +COPY --from=root pyproject.toml ./pyproject.toml Review Comment: Please see `.github/workflows/build_infra_images_cache.yml`. This PR adds COPY --from=root pyproject.toml to the Dockerfile but does not add build-contexts: root=./ to the corresponding Build and push (PySpark with old dependencies) step in build_infra_images_cache.yml. Without the named build context the Docker daemon cannot resolve --from=root and the scheduled infra image cache build will fail. Functional CI via build_and_test.yml continues to work (it already provides build-contexts: root=./ for all PYSPARK_IMAGE_TO_TEST images), so tests themselves are unaffected, only the pre-built image cache is broken. The same omission was introduced for python-312-classic-only in SPARK-58771 but was not caught at review time; it should be fixed here rather than carried forward again. Fix: add build-contexts: |\n root=./ to the python-minimum build step in that workflow, matching the docs/lint/python-311/python-312/python-313/python-314 steps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
