[
https://issues.apache.org/jira/browse/SPARK-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941869#comment-15941869
]
Samuel Soubeyran commented on SPARK-15665:
------------------------------------------
When trying to do the same thing using the SparkLauncher class, it doesn't work:
new SparkLauncher(sparkEnvMap).setSparkHome(sparkHome).addSparkArg("--kill",
submissionId).launch().
This is because SparkLauncher call the empty constructor in
SparkSubmitCommandBuilder, bypassing all the logic to handle the kill and
status command present in the constructor
SparkSubmitCommandBuilder(List<String> args).
I also want to make the case that this logic shouldn't be present in the
constructor in the first place since it's a builder pattern. Instead it should
be in the buildSparkSubmitCommand. The default should be the empty constructor,
the full constructor is just a shortcut. Else what the point of having a
builder pattern in the first place.
I'd be happy to send a PR to solve this,
Also there might be a easier way to kill/get the status of a job (cluster mode)
but i couldn't figure it out.
Thanks,
Sam
> spark-submit --kill and --status are not working
> -------------------------------------------------
>
> Key: SPARK-15665
> URL: https://issues.apache.org/jira/browse/SPARK-15665
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Devaraj K
> Assignee: Devaraj K
> Fix For: 2.0.0
>
>
> {code:xml}
> [devaraj@server2 spark-master]$ ./bin/spark-submit --kill
> driver-20160531171222-0000 --master spark://xx.xx.xx.xx:6066
> Exception in thread "main" java.lang.IllegalArgumentException: Missing
> application resource.
> at
> org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:241)
> at
> org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitArgs(SparkSubmitCommandBuilder.java:160)
> at
> org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitCommand(SparkSubmitCommandBuilder.java:276)
> at
> org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.java:151)
> at org.apache.spark.launcher.Main.main(Main.java:86)
> {code}
> {code:xml}
> [devaraj@server2 spark-master]$ ./bin/spark-submit --status
> driver-20160531171222-0000 --master spark://xx.xx.xx.xx:6066
> Exception in thread "main" java.lang.IllegalArgumentException: Missing
> application resource.
> at
> org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:241)
> at
> org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitArgs(SparkSubmitCommandBuilder.java:160)
> at
> org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitCommand(SparkSubmitCommandBuilder.java:276)
> at
> org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.java:151)
> at org.apache.spark.launcher.Main.main(Main.java:86)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
