[
https://issues.apache.org/jira/browse/SPARK-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212993#comment-15212993
]
liyan commented on SPARK-14173:
-------------------------------
hi Owen:
What I used is java *SparkLauncher* interface, the code is as below:
{code:title=MySparkLauncher.java|borderStyle=solid}
// set conf
public SparkLauncher setConf(){
return new org.apache.spark.launcher.SparkLauncher()
.setConf("spark.streaming.backpressure.enabled", "true")
.setConf("spark.executor.extraJavaOptions",
"-Xloggc:/home/streaming/gc.log");
//other conf
//.....
}
{code}
All confs work fine except "spark.executor.extraJavaOptions".
The warning log is {quote} {color:red} "Ignoring none-spark config property :
"spark.executor.extraJavaOptions= -Xloggc:/home/streaming/gc.log"{color}{quote}
> Ignoring config property “spark.executor.extraJavaOptions”
> ----------------------------------------------------------
>
> Key: SPARK-14173
> URL: https://issues.apache.org/jira/browse/SPARK-14173
> Project: Spark
> Issue Type: Bug
> Components: Streaming, YARN
> Affects Versions: 1.5.2
> Reporter: liyan
>
> when i submit streaming application on *yarn cluster* mode, i can't find
> "spark.executor.extraJavaOptions" in Spark UI.
> the log prints warning：
> {quote}Ignoring none-spark config property :
> "spark.executor.extraJavaOptions= -Xloggc:/home/streaming/gc.log"{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
