[
https://issues.apache.org/jira/browse/SPARK-26704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750323#comment-16750323
]
Rob Vesse commented on SPARK-26704:
-----------------------------------
For me it's a question of build reproducibility (I've been following an
interesting discussion around this on legal-discuss -
https://lists.apache.org/thread.html/d578819f1afa6b8fb697ea72083e0fb05e43938a23d6e7bb804069b8@%3Clegal-discuss.apache.org%3E).
If I crack open the image and start poking around and find a Dockerfile
present do I have a reasonable expectation that the Dockerfile I find there is
the one used to build the image?
If Yes, then we should ensure we include the correct Dockerfile's in the build
context and thus the image.
If No, then we should probably not bother including the Dockerfile's at all.
However since as you point out when building from a Spark release distribution
they will be present and thus packaged into the image I would suspect we want
to continue doing this even for developer builds.
> docker-image-tool.sh should copy custom Dockerfiles into the build context
> for inclusion in images
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-26704
> URL: https://issues.apache.org/jira/browse/SPARK-26704
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 2.4.0
> Reporter: Rob Vesse
> Priority: Major
>
> As surfaced in the discussion on the PR for SPARK-26687
> (https://github.com/apache/spark/pull/23613) when using custom Dockerfiles
> these are not copied into the build context. Rather the build context
> includes the default Dockerfiles from Spark regardless of what Dockerfiles
> the end user actually used to build the images.
> The suggestion in the PR was that the script should copy in the custom
> Dockerfiles over the stock Dockerfiles. This potentially aids in
> reproducing the images later because someone with an image can get the exact
> Dockerfile used to build that image.
> A related issue is that the script allows for and even in some cases
> implicitly uses Docker build arguments as part of building the images. In
> the case where build arguments are used these should probably also be
> captured in the image to aid reproducibility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
