For MapReduce jobs, you should use you configuration options in mapred-site.xml or Configuration object in your Driver class.
mapreduce.map.memory.mb1536Larger resource limit for maps. mapreduce.map.java.opts-Xmx1024MLarger heap-size for child jvms of maps. mapreduce.reduce.memory.mb3072Larger resource limit for reduces. mapreduce.reduce.java.opts-Xmx2560MLarger heap-size for child jvms of reduces. You can read more here http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Configuring_the_Hadoop_Daemons_in_Non-Secure_Mode and http://grokbase.com/t/hadoop/hdfs-user/1324x8nt6d/how-mapred-map-child-java-opts-influence-the-container-resource-allocation 2013/11/26 ch huang <[email protected]> > hi,maillist: > i find each my container just use 200M heap space,how can i > resize it? > > # ps -ef|grep -i yarnchild > yarn 24333 8210 99 14:09 ? 00:00:05 > /usr/java/jdk1.7.0_25/bin/java -Djava.net.preferIPv4Stack=true > -Dhadoop.metrics.log.level=WARN -Xmx200m > -Djava.io.tmpdir=/data/mrlocal/1/yarn/local/usercache/hdfs/appcache/application_1385445543402_0006/container_1385445543402_0006_01_000017/tmp > -Dlog4j.configuration=container-log4j.properties > -Dyarn.app.mapreduce.container.log.dir=/data/mrlocal/2/yarn/logs/application_1385445543402_0006/container_1385445543402_0006_01_000017 > -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA > org.apache.hadoop.mapred.YarnChild 192.168.10.224 59237 > attempt_1385445543402_0006_m_000015_0 17 >
