see mapreduce.job.reduce.slowstart.completedmaps It gives hint of when reduce tasks could kick off.
2014-09-26 8:36 GMT+08:00 Abdul Navaz <[email protected]>: > Hello, > > I am having a Hadoop cluster with 1 name node and 3 data nodes. I running > sample word count job on 1GB of file which is distributed among the HDFS. > > When I run the map reduce job, before even completing the mapping 100 % > reduce starts. Say for eg map 40% reduce 10% etc. > > I would like to know when the shuffling traffic starts ? > > -> Is there any way to find out when exactly shuffling started ? Does it > generate any syslog in the logs . > -> How to find the total amount of shuffling traffic? > > > > Thanks & Regards, > > Abdul Navaz > Research Assistant > University of Houston Main Campus, Houston TX > Ph: 281-685-0388 > > -- Bing Jiang Tel：(86)134-2619-1361 weibo: http://weibo.com/jiangbinglover BLOG: www.binospace.com BLOG: http://blog.sina.com.cn/jiangbinglover Focus on distributed computing, HDFS/HBase
