last split generated by FileInputFormat.getSplits may not have the best locality
--------------------------------------------------------------------------------
Key: MAPREDUCE-3710
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3710
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv1, mrv2
Affects Versions: 1.0.0, 0.23.0
Reporter: Siddharth Seth
Assignee: Siddharth Seth
The last split generated by FileInputFormat.getSplits considers
{{blkLocations.length-1}} to be the hosts for the split.
The last split may be larger than the rest (SPLIT_SLOP=1.1 by default) - in
which case locality is picked up from a smaller block.
e.g. 1027MB file with a 128MB split size. The last split ends up being 131MB.
The hosts for locality end up being the nodes containing the 3MB block instead
of the 128MB block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
