Infinite loop at creating splits using CombineFileInputFormat
-------------------------------------------------------------
Key: MAPREDUCE-2185
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2185
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: job submission
Reporter: Hairong Kuang
Assignee: Hairong Kuang
Fix For: 0.22.0
This is caused by a missing block in HDFS. So the block's locations are empty.
The following code adds the block to blockToNodes map but not to rackToBlocks
map. Later on when generating splits, only blocks in rackToBlocks are removed
from blockToNodes map. So blockToNodes map can never become empty therefore
causing infinite loop
{code}
// add this block to the block --> node locations map
blockToNodes.put(oneblock, oneblock.hosts);
// add this block to the rack --> block map
for (int j = 0; j < oneblock.racks.length; j++) {
..
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
