Hi Chen, spark.executor.extraJavaOptions is introduced in Spark 1.0, not in Spark 0.9. You need to
export SPARK_JAVA_OPTS=" -Dspark.config1=value1 -Dspark.config2=value2" in conf/spark-env.sh. Let me know if that works. Andrew 2014-07-17 18:15 GMT-07:00 Tathagata Das <tathagata.das1...@gmail.com>: > Can you check in the environment tab of Spark web ui to see whether this > configuration parameter is in effect? > > TD > > > On Thu, Jul 17, 2014 at 2:05 PM, Chen Song <chen.song...@gmail.com> wrote: > >> I am using spark 0.9.0 and I am able to submit job to YARN, >> https://spark.apache.org/docs/0.9.0/running-on-yarn.html. >> >> I am trying to turn on gc logging on executors but could not find a way >> to set extra Java opts for workers. >> >> I tried to set spark.executor.extraJavaOptions but that did not work. >> >> Any idea on how I should do this? >> >> -- >> Chen Song >> >> >
