using spark 0.8.0 I have this line in my spark-env.sh
SPARK_JAVA_OPTS+=" -Dspark.local.dir=/tmp/spark -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+DisableExplicitGC -Xms1024m -Xmx2048m -XX:MaxPermSize=256m" export SPARK_JAVA_OPTS logs shows it used but also -Xms512m -Xmx512m, how I can remove any reference to use -Xms512m -Xmx512m when I am setting explicitly -Xms1024m -Xmx2048m in my spark-env.sh ? Spark Command: /opt/jdk1.7.0_40/bin/java -cp :/opt/spark-0.8.0/conf:/opt/spark-0.8.0/assembly/target/scala-2.9.3/spark-assembly_2.9.3-0.8.0-incubating-hadoop1.0.4.jar -Dspark.local.dir=/tmp/spark -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+DisableExplicitGC -Xms1024m -Xmx2048m -XX:MaxPermSize=256m -Dspark.local.dir=/tmp/spark -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+DisableExplicitGC -Xms1024m -Xmx2048m -XX:MaxPermSize=256m -Dspark.local.dir=/tmp/spark -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+DisableExplicitGC -Xms1024m -Xmx2048m -XX:MaxPermSize=256m -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.master.Master --ip poc1.stc1lab.local --port 7077 --webui-port 8080 ======================================== if I set in my spark-env.sh export SPARK_JAVA_OPTS=" -Dspark.local.dir=/tmp/spark -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+DisableExplicitGC -Xms1024m -Xmx2048m -XX:MaxPermSize=256m" logs shows: Spark Command: /opt/jdk1.7.0_40/bin/java -cp :/opt/spark-0.8.0/conf:/opt/spark-0.8.0/assembly/target/scala-2.9.3/spark-assembly_2.9.3-0.8.0-incubating-hadoop1.0.4.jar -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.master.Master --ip poc1.stc1lab.local --port 7077 --webui-port 8080 ======================================== can someone please update on the correct way of setting SPARK_JAVA_OPTS in spark-env.sh? Thanks,
