That command line you mention in your e-mail doesn't look like something started by Spark. Spark would start one of ApplicationMaster, ExecutableRunner or CoarseGrainedSchedulerBackend, not "org.apache.hadoop.mapred.YarnChild".
On Wed, Aug 20, 2014 at 6:56 PM, centerqi hu <cente...@gmail.com> wrote: > Spark memory settings let me very misunderstanding. > > My code is as follows. > > spark-1.0.2-bin-2.4.1/bin/spark-submit --class SimpleApp \ > --master yarn \ > --deploy-mode cluster \ > --queue sls_queue_1 \ > --num-executors 3 \ > --driver-memory 6g \ > --executor-memory 10g \ > --executor-cores 5 \ > target/scala-2.10/simple-project_2.10-1.0.jar \ > /user/www/abc/output/2014-08-*/* > > > Set executor-memory 10g, > But, see the nodemanager java process. > Why is Xmx3072m instead of 10G? > > jdk1.7.0_67//bin/java -Djava.net.preferIPv4Stack=true > -Dhadoop.metrics.log.level=WARN -Xmx3072m > -Djava.io.tmpdir=/data/hadoop/nodemanager/usercache/www/appcache/application_1408182086233_0013/container_1408182086233_0013_01_000004/tmp > -Dlog4j.configuration=container-log4j.properties > -Dyarn.app.container.log.dir=/data/hadoop/logs/nodemanager/application_1408182086233_0013/container_1408182086233_0013_01_000004 > -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA > org.apache.hadoop.mapred.YarnChild 10.1.13.11 35389 > attempt_1408182086233_0013_m_000002_0 4 > > > Thx > > cente...@gmail.com|齐忠 > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
