Mark Grover created SPARK-13670: ----------------------------------- Summary: spark-class doesn't bubble up error from launcher command Key: SPARK-13670 URL: https://issues.apache.org/jira/browse/SPARK-13670 Project: Spark Issue Type: Bug Components: Spark Submit Affects Versions: 2.0.0 Reporter: Mark Grover Priority: Minor
There's a particular snippet in spark-class [here|https://github.com/apache/spark/blob/master/bin/spark-class#L86] that runs the spark-launcher code in a subshell. {code} # The launcher library will print arguments separated by a NULL character, to allow arguments with # characters that would be otherwise interpreted by the shell. Read that in a while loop, populating # an array that will be used to exec the final command. CMD=() while IFS= read -d '' -r ARG; do CMD+=("$ARG") done < <("$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@") {code} The problem is that the if the launcher Main fails, this code still still returns success and continues, even though the top level script is marked {{set -e}}. This is because the launcher.Main is run within a subshell. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org