Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19717#discussion_r154566070
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/executor/Dockerfile ---
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+FROM spark-base
+
+# If this docker file is being used in the context of building your images
from a Spark distribution, the docker build
+# command should be invoked from the top level directory of the Spark
distribution. E.g.:
+# docker build -t spark-executor:latest -f dockerfiles/executor/Dockerfile
.
+
+COPY examples /opt/spark/examples
+
+CMD SPARK_CLASSPATH="${SPARK_HOME}/jars/*" && \
+ env | grep SPARK_JAVA_OPT_ | sed 's/[^=]*=\(.*\)/\1/g' >
/tmp/java_opts.txt && \
+ readarray -t SPARK_EXECUTOR_JAVA_OPTS < /tmp/java_opts.txt && \
+ if ! [ -z ${SPARK_MOUNTED_CLASSPATH}+x} ]; then
SPARK_CLASSPATH="$SPARK_MOUNTED_CLASSPATH:$SPARK_CLASSPATH"; fi && \
+ if ! [ -z ${SPARK_EXECUTOR_EXTRA_CLASSPATH+x} ]; then
SPARK_CLASSPATH="$SPARK_EXECUTOR_EXTRA_CLASSPATH:$SPARK_CLASSPATH"; fi && \
+ ${JAVA_HOME}/bin/java "${SPARK_EXECUTOR_JAVA_OPTS[@]}"
-Dspark.executor.port=$SPARK_EXECUTOR_PORT -Xms$SPARK_EXECUTOR_MEMORY
-Xmx$SPARK_EXECUTOR_MEMORY -cp "$SPARK_CLASSPATH"
org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url
$SPARK_DRIVER_URL --executor-id $SPARK_EXECUTOR_ID --cores
$SPARK_EXECUTOR_CORES --app-id $SPARK_APPLICATION_ID --hostname
$SPARK_EXECUTOR_POD_IP
--- End diff --
`SPARK_EXECUTOR_PORT` is no longer set as `spark.executor.port` is no
longer used by Spark. Removed `SPARK_EXECUTOR_PORT`. `SPARK_MOUNTED_CLASSPATH`
is set in `DependencyResolutionStep`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
