Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22897#discussion_r230476657
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh ---
@@ -96,22 +96,6 @@ case "$SPARK_K8S_CMD" in
"$@"
)
;;
- driver-py)
--- End diff --
I think this is fine and what we want to do. But at some point we're going
to want to make the API between spark submit and launched containers stable.
Using this as an example, if a user upgraded their spark-submit version to
3.0 but didn't upgrade the version of Spark in their docker image, the docker
container's command will attempt to look up these old environment variables
that are no longer being set by spark submit.
We should be thinking about making this contract stable from 3.0 onwards.
For this coming release I think this is fine.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
