[
https://issues.apache.org/jira/browse/SPARK-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin resolved SPARK-21399.
------------------------------------
Resolution: Invalid
Sorry, but you not understanding shell code is not a reason to rewrite it...
there are reasons why the code does what it does and why simpler approaches
don't work.
> Difficult to debug code in spark-class
> --------------------------------------
>
> Key: SPARK-21399
> URL: https://issues.apache.org/jira/browse/SPARK-21399
> Project: Spark
> Issue Type: Improvement
> Components: Spark Shell, Spark Submit
> Affects Versions: 2.1.1
> Environment: Running spark on a version of hottub.
> Reporter: Mahesh
> Priority: Minor
>
> The code below is difficult to debug, and its not clear what is happening, it
> appears to use some kind of shell magic, if possible it should be replaced
> with an easier to understand code. Specifically there needs to be a log of
> the exact java command that is finally invoked.
> Using input redirection means that adding comments to the build_command will
> break the code ,I am not sure but the below code for build_command is not
> equivalent to one in the code, although semantically it should be.
> build_command() {
> "$RUNNER" -Xmx128m -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main
> "$@"
> printf "%d\0" $?
> #printf "Something\0"
> }
> *Original code below*
> *build_command() {
> "$RUNNER" -Xmx128m -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main
> "$@"
> printf "%d\0" $?
> }
> CMD=()
> while IFS= read -d '' -r ARG; do
> CMD+=("$ARG")
> done < <(build_command "$@")*
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
