13/12/03 11:46:56 INFO mapreduce.Job: map 100% reduce 19% 13/12/03 11:47:33 INFO mapreduce.Job: map 100% reduce 20% 13/12/03 11:47:54 INFO mapreduce.Job: map 100% reduce 21% 13/12/03 11:48:06 INFO mapreduce.Job: map 100% reduce 22% 13/12/03 11:48:17 INFO mapreduce.Job: map 100% reduce 23% 13/12/03 11:48:29 INFO mapreduce.Job: map 100% reduce 24% 13/12/03 11:49:23 INFO mapreduce.Job: map 83% reduce 25% 13/12/03 11:49:39 INFO mapreduce.Job: map 84% reduce 25% 13/12/03 11:49:52 INFO mapreduce.Job: map 85% reduce 25% 13/12/03 11:50:08 INFO mapreduce.Job: map 86% reduce 25% 13/12/03 11:50:16 INFO mapreduce.Job: map 87% reduce 25% 13/12/03 11:50:21 INFO mapreduce.Job: map 88% reduce 25% 13/12/03 11:50:30 INFO mapreduce.Job: map 89% reduce 25% 13/12/03 11:50:42 INFO mapreduce.Job: map 90% reduce 25% 13/12/03 11:50:57 INFO mapreduce.Job: map 91% reduce 25% 13/12/03 11:51:10 INFO mapreduce.Job: map 92% reduce 25% 13/12/03 11:51:18 INFO mapreduce.Job: map 92% reduce 26% 13/12/03 11:51:20 INFO mapreduce.Job: map 93% reduce 26% 13/12/03 11:51:25 INFO mapreduce.Job: map 94% reduce 26% 13/12/03 11:51:31 INFO mapreduce.Job: map 95% reduce 26% 13/12/03 11:51:43 INFO mapreduce.Job: map 96% reduce 26% 13/12/03 11:51:50 INFO mapreduce.Job: map 97% reduce 26% 13/12/03 11:51:59 INFO mapreduce.Job: map 98% reduce 26% 13/12/03 11:52:19 INFO mapreduce.Job: map 99% reduce 26% 13/12/03 11:52:29 INFO mapreduce.Job: map 100% reduce 26%
On Tue, Dec 3, 2013 at 10:25 AM, ch huang <[email protected]> wrote: > another question is why the map process progress will back when it reach > 100%? > > > > > On Tue, Dec 3, 2013 at 10:07 AM, ch huang <[email protected]> wrote: > >> hi,maillist: >> i run a job on my CDH4.4 yarn framework ,it's map task >> finished very fast,but reduce is very slow, i check it use ps command >> find it's work heap size is 200m,so i try to increase heap size used by >> reduce task,i add "YARN_OPTS="$YARN_OPTS >> -Dmapreduce.reduce.java.opts=-Xmx1024m -verbose:gc -XX:+PrintGCDetails >> -XX:+PrintGCDateStamps >> -Xloggc:$YARN_LOG_DIR/gc-$(hostname)-resourcemanager.log >> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=15M >> -XX:-UseGCOverheadLimit" in yarn-env.sh file ,but when i restart the >> nodemanager ,i find new reduce task still use 200m heap ,why? >> >> # jps >> 2853 DataNode >> 19533 Jps >> 10949 YarnChild >> 10661 NodeManager >> 15130 HRegionServer >> # ps -ef|grep 10949 >> yarn 10949 10661 99 09:52 ? 00:19:31 >> /usr/java/jdk1.7.0_45/bin/java -Djava.net.preferIPv4Stack=true >> -Dhadoop.metrics.log.level=WARN -Xmx200m >> -Djava.io.tmpdir=/data/1/mrlocal/yarn/local/usercache/hdfs/appcache/application_1385983958793_0022/container_1385983958793_0022_01_005650/tmp >> -Dlog4j.configuration=container-log4j.properties >> -Dyarn.app.mapreduce.container.log.dir=/data/2/mrlocal/yarn/logs/application_1385983958793_0022/container_1385983958793_0022_01_005650 >> -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA >> org.apache.hadoop.mapred.YarnChild 192.168.11.10 48936 >> attempt_1385983958793_0022_r_000000_14 5650 >> >> >> >> >> > >
