Repository: spark Updated Branches: refs/heads/branch-1.5 56f0bb6ed -> 1cd2d9ced
[SPARK-11264] bin/spark-class can't find assembly jars with certain GREP_OPTIONS set Temporarily remove GREP_OPTIONS if set in bin/spark-class. Some GREP_OPTIONS will modify the output of the grep commands that are looking for the assembly jars. For example, if the -n option is specified, the grep output will look like: 5:spark-assembly-1.5.1-hadoop2.4.0.jar This will not match the regular expressions, and so the jar files will not be found. We could improve the regular expression to handle this case and trim off extra characters, but it is difficult to know which options may or may not be set. Unsetting GREP_OPTIONS within the script handles all the cases and gives the desired output. Author: Jeffrey Naisbitt <jnaisb...@familysearch.org> Closes #9231 from naisbitt/unset-GREP_OPTIONS. (cherry picked from commit 28132ceb10d0c127495ce8cb36135e1cb54164d7) Signed-off-by: Sean Owen <so...@cloudera.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1cd2d9ce Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1cd2d9ce Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/1cd2d9ce Branch: refs/heads/branch-1.5 Commit: 1cd2d9ced55ab52899a4e57b5c3da6b444ec9ae4 Parents: 56f0bb6 Author: Jeffrey Naisbitt <jnaisb...@familysearch.org> Authored: Sat Oct 24 18:21:36 2015 +0100 Committer: Sean Owen <so...@cloudera.com> Committed: Sat Oct 24 18:21:47 2015 +0100 ---------------------------------------------------------------------- bin/spark-class | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/1cd2d9ce/bin/spark-class ---------------------------------------------------------------------- diff --git a/bin/spark-class b/bin/spark-class index 2b59e5d..7db8cb5 100755 --- a/bin/spark-class +++ b/bin/spark-class @@ -42,6 +42,7 @@ else ASSEMBLY_DIR="$SPARK_HOME/assembly/target/scala-$SPARK_SCALA_VERSION" fi +GREP_OPTIONS= num_jars="$(ls -1 "$ASSEMBLY_DIR" | grep "^spark-assembly.*hadoop.*\.jar$" | wc -l)" if [ "$num_jars" -eq "0" -a -z "$SPARK_ASSEMBLY_JAR" ]; then echo "Failed to find Spark assembly in $ASSEMBLY_DIR." 1>&2 --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
