Hi guys ! I see that hadoop doesn't capture the Map task I/O time and Reduce task I/O time and captures only map runtime and reduce runtime. Am i right ?
By I/O time for map task i meant time taken by the map task to read the input chunk allocated to it for processing and the time for it to write the O/P data to the local disk. By I/O time for Reduce task i meant time for reduce task to transfer map O/Ps to reduce task(shuffle phase) and writing reduce O/Ps to DFS. > What does that runtime of Map and reduce tasks mean ? Does it mean time taken to execute the Map method and reduce method respectively ? (or) Does it mean time taken from the start of the Map/Reduce task to the completion of the Map/Reduce task(i.e including time to read,sort ,compute map or reduce ,merge,etc.) ? > Which files do i need to look at and modify in Hadoop if i want to capture > the map and reduce I/O time's ? > If i want to capture these values for few jobs of applications like > wordcount,sort,etc. what is the best way to do ? Can anyone guide me in this regard ? Thanks, Arun -- View this message in context: http://lucene.472066.n3.nabble.com/Capturing-Map-and-Reduce-I-O-time-tp3545298p3545298.html Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
