Sorry for sending this email again but I got no answers from the first one. Anyone please help or forward it to mail-list that would help.
2011-06-15 *********************************************** * Hailong Yang, PhD. Candidate * Sino-German Joint Software Institute, * School of Computer Science&Engineering, Beihang University * Phone: (86-010)82315908 * Email: hailong.yang1...@gmail.com * Address: G413, New Main Building in Beihang University, * No.37 XueYuan Road,HaiDian District, * Beijing,P.R.China,100191 *********************************************** 发件人： hailong.yang1115 发送时间： 2011-06-10 13:28:46 收件人： general 抄送： 主题： Problems about the job counters Dear all, I am trying to the built-in example wordcount with nearly 15GB input. When the Hadoop job finished, I got the following counters. CounterMapReduceTotal Job CountersLaunched reduce tasks001 Rack-local map tasks0035 Launched map tasks002,318 Data-local map tasks002,283 FileSystemCountersFILE_BYTES_READ22,863,580,65617,654,943,34140,518,523,997 HDFS_BYTES_READ154,400,997,4590154,400,997,459 FILE_BYTES_WRITTEN33,490,829,40317,654,943,34151,145,772,744 HDFS_BYTES_WRITTEN02,747,356,7042,747,356,704 My question is what does the FILE_BYTES_READ counter mean? And what is the difference between FILE_BYTES_READ and HDFS_BYTES_READ? In my opinion, all the input is located in HDFS, so where does FILE_BYTES_READ come from during the map phase? Any help will be appreciated! Hailong 2011-06-10 *********************************************** * Hailong Yang, PhD. Candidate * Sino-German Joint Software Institute, * School of Computer Science&Engineering, Beihang University * Phone: (86-010)82315908 * Email: hailong.yang1...@gmail.com * Address: G413, New Main Building in Beihang University, * No.37 XueYuan Road,HaiDian District, * Beijing,P.R.China,100191 ***********************************************
