Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20669#discussion_r175238936
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh ---
@@ -53,14 +53,10 @@ fi
case "$SPARK_K8S_CMD" in
driver)
CMD=(
- ${JAVA_HOME}/bin/java
- "${SPARK_JAVA_OPTS[@]}"
- -cp "$SPARK_CLASSPATH"
- -Xms$SPARK_DRIVER_MEMORY
- -Xmx$SPARK_DRIVER_MEMORY
- -Dspark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS
- $SPARK_DRIVER_CLASS
- $SPARK_DRIVER_ARGS
+ "$SPARK_HOME/bin/spark-submit"
+ --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS"
+ --deploy-mode client
+ "$@"
--- End diff --
If one is going to override the docker image and provide their own
implementation, the customizing developer would like to know all of the
parameters that they are being passed up-front. Receiving "$@" makes it less
obvious that we should be expecting a properties file and the
`SparkLauncher.NO_RESOURCE` from spark-submit.
We could do argument parsing here to verify that `--properties-file` etc.
are explicitly provided.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
