Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22748#discussion_r225614967
--- Diff: bin/docker-image-tool.sh ---
@@ -44,28 +44,41 @@ function image_ref {
function build {
local BUILD_ARGS
local IMG_PATH
+ local JARS
if [ ! -f "$SPARK_HOME/RELEASE" ]; then
# Set image build arguments accordingly if this is a source repo and
not a distribution archive.
IMG_PATH=resource-managers/kubernetes/docker/src/main/dockerfiles
+ JARS=assembly/target/scala-$SPARK_SCALA_VERSION/jars
BUILD_ARGS=(
${BUILD_PARAMS}
--build-arg
img_path=$IMG_PATH
--build-arg
- spark_jars=assembly/target/scala-$SPARK_SCALA_VERSION/jars
+ spark_jars=$JARS
--build-arg
k8s_tests=resource-managers/kubernetes/integration-tests/tests
)
else
- # Not passed as an argument to docker, but used to validate the Spark
directory.
+ # Not passed as arguments to docker, but used to validate the Spark
directory.
IMG_PATH="kubernetes/dockerfiles"
+ JARS=jars
BUILD_ARGS=(${BUILD_PARAMS})
fi
+ # Verify that the Docker image content directory is present
if [ ! -d "$IMG_PATH" ]; then
error "Cannot find docker image. This script must be run from a
runnable distribution of Apache Spark."
fi
+
+ # Verify that Spark has actually been built/is a runnable distribution
+ #Â i.e. the Spark JARs that the Docker files will place into the image
are present
+ local TOTAL_JARS=$(ls $JARS/spark-* | wc -l)
+ TOTAL_JARS=$(( $TOTAL_JARS ))
+ if [ "${TOTAL_JARS}" -eq 0 ]; then
+ error "Cannot find Spark JARs. This script assumes that Apache Spark
has first been built locally."
--- End diff --
Double white spaces after `JARS.`. Also it's better to say `Apache Spark
has first been built locally or it is run from inside a distribution`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
