[
https://issues.apache.org/jira/browse/SPARK-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132164#comment-14132164
]
Andrew Or commented on SPARK-3107:
----------------------------------
I see. Yes, setting it to an empty value is semantically different from not
setting it, and that is exactly the reason why we shouldn't set these to empty
strings when what we really mean to do is to not set it. I think we still do
that in a few places here and there.
> Don't pass null jar to executor in yarn-client mode
> ---------------------------------------------------
>
> Key: SPARK-3107
> URL: https://issues.apache.org/jira/browse/SPARK-3107
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.1.0
> Reporter: Andrew Or
>
> In the following line, ExecutorLauncher's `--jar` takes in null.
> {code}
> 14/08/18 20:52:43 INFO yarn.Client: command: $JAVA_HOME/bin/java -server
> -Xmx512m ... org.apache.spark.deploy.yarn.ExecutorLauncher --class 'notused'
> --jar null --arg 'ip-172-31-0-12.us-west-2.compute.internal:56838'
> --executor-memory 1024 --executor-cores 1 --num-executors 2
> {code}
> Also it appears that we set a bunch of system properties to empty strings
> (not shown). We should avoid setting these if they don't actually contain
> values.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
