Corner-case in IFile leads to failed tasks
------------------------------------------
Key: HADOOP-3647
URL: https://issues.apache.org/jira/browse/HADOOP-3647
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Affects Versions: 0.18.0
Reporter: Arun C Murthy
Assignee: Arun C Murthy
Priority: Blocker
Fix For: 0.18.0
A couple of reduce tasks failed at IFile.Reader.next, one with:
{noformat}
java.lang.NegativeArraySizeException
at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:246)
at org.apache.hadoop.mapred.IFile$Reader.next(IFile.java:298)
at org.apache.hadoop.mapred.Merger$Segment.next(Merger.java:134)
at
org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:225)
at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:242)
at org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:720)
at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:679)
at
org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:225)
at
org.apache.hadoop.mapred.lib.IdentityReducer.reduce(IdentityReducer.java:39)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:316)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2157)
{noformat}
On a related note, another failed at:
{code:title=IFile.java:380}
// Position for the next record
long skipped = dataIn.skip(recordLength);
if (skipped != recordLength) {
throw new IOException("Failed to skip past record of length: " +
recordLength);
}
{code}
where recordLength was *-17*.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
