Bug in MapTask.MapOutputBuffer.collect leads to an unnecessary and harmful
'reset'
----------------------------------------------------------------------------------
Key: HADOOP-3931
URL: https://issues.apache.org/jira/browse/HADOOP-3931
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Affects Versions: 0.17.1
Reporter: Arun C Murthy
Assignee: Chris Douglas
Priority: Blocker
Fix For: 0.17.2
The unnecessary reset of the io.sort.mb buffer when bufindex==bufvoid leads to
wrong, very large, temporary key buffer being created and passed along to the
Buffer.write. This causes MapBufferTooSmallException as the Buffer thinks there
isn't enough space to write this record into. If a combiner is defined the
record does not go through the combiner and hits the disk directly.
It's already fixed in hadoop-0.18.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
