Hi guys ! I see that hadoop doesn't capture the Map task I/O time and Reduce task I/O time and captures only map runtime and reduce runtime. Am i right ?
By I/O time for map task i meant time taken by the map task to read the input chunk allocated to it for processing and the time for it to write the O/P data to the local disk. By I/O time for Reduce task i meant time for reduce task to transfer map O/Ps to reduce task(shuffle phase) and writing reduce O/Ps to DFS. > What does that runtime of Map and reduce tasks mean ? Does it mean time taken to execute the Map method and reduce method respectively ? (or) Does it mean time taken from the start of the Map/Reduce task to the completion of the Map/Reduce task(i.e including time to read,sort ,compute map or reduce ,merge,etc.) ? > Which files do i need to look at and modify in Hadoop if i want to capture the map and reduce I/O time's ? > If i want to capture these values for few jobs of applications like wordcount,sort,etc. what is the best way to do ? Can anyone guide me in this regard ? Thanks, Arun
