The input is effectively split by lines, but under the covers, the actual splits are by byte. Each mapper will cleverly scan from the specified start to the next line after the start point. At then end, it will over-read to the end of line that is at or after the end of its specified region. This can make the last split be a bit smaller than the others and the first be a bit larger.
Practically speaking, however, your 2000 line file is extremely unlikely to be split at all because it is sooo small. On Fri, Feb 18, 2011 at 11:14 AM, maha <[email protected]> wrote: > Hi all, > > I want to check if the following statement is right: > > If I use TextInputFormat to process a text file with 2000 lines (each > ending with \n) with 20 mappers. Then each map will have a sequence of > COMPLETE LINES . > > In other words, the input is not split byte-wise but by lines. > > Is that right? > > > Thank you, > Maha
