Hello Raj, On Fri, Apr 29, 2011 at 5:52 AM, Raj V <[email protected]> wrote: > I am trying to understand the relationship,if any, between HDFS_BYTES_READ > and FILE_BYTES_READ. Since HDFS is laid on top of the local file system is > FILE_BYTES_READ include the bytes read by HDFS_BYTES_READ.. For example in my > word count program > HDFS_BYTES_READ is 1,000,000,415,344 and FILE_BYTES_READ is 679,397,984,899. > Does it mean that 679,397,984,899 bytes were read by the data local map jobs > or are the two counters completely independent of each other?
They are individual counters, independent of each other. Although HDFS may, in lower level terms, be a layer atop the 'local filesystem', the MapReduce components do not see it in such a granular manner. -- Harsh J
