Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/19717#discussion_r154503509
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/driver/Dockerfile ---
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+FROM spark-base
+
+# If this docker file is being used in the context of building your images
from a Spark distribution, the docker build
+# command should be invoked from the top level directory of the Spark
distribution. E.g.:
+# docker build -t spark-driver:latest -f dockerfiles/driver/Dockerfile .
+
+COPY examples /opt/spark/examples
+
+CMD SPARK_CLASSPATH="${SPARK_HOME}/jars/*" && \
+ env | grep SPARK_JAVA_OPT_ | sed 's/[^=]*=\(.*\)/\1/g' >
/tmp/java_opts.txt && \
+ readarray -t SPARK_DRIVER_JAVA_OPTS < /tmp/java_opts.txt && \
+ if ! [ -z ${SPARK_MOUNTED_CLASSPATH+x} ]; then
SPARK_CLASSPATH="$SPARK_MOUNTED_CLASSPATH:$SPARK_CLASSPATH"; fi && \
+ if ! [ -z ${SPARK_SUBMIT_EXTRA_CLASSPATH+x} ]; then
SPARK_CLASSPATH="$SPARK_SUBMIT_EXTRA_CLASSPATH:$SPARK_CLASSPATH"; fi && \
+ ${JAVA_HOME}/bin/java "${SPARK_DRIVER_JAVA_OPTS[@]}" -cp
"$SPARK_CLASSPATH" -Xms$SPARK_DRIVER_MEMORY -Xmx$SPARK_DRIVER_MEMORY
-Dspark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS $SPARK_DRIVER_CLASS
$SPARK_DRIVER_ARGS
--- End diff --
what are the core differences in driver vs executor image, other than the
`SPARK_DRIVER*` env?
we could consider switching in a docker-entrypoint.sh script like in
https://github.com/apache/flink/blob/master/flink-contrib/docker-flink/docker-entrypoint.sh
and have one image instead.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
