[
https://issues.apache.org/jira/browse/HADOOP-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12600980#action_12600980
]
Chris Douglas commented on HADOOP-3443:
---------------------------------------
+1 Looks good. Some of the logging could be at the debug level, particularly
the logging in the synchronized section of BlockingBuffer.Buffer::write, but it
shouldn't be a real issue.
bq. Is it a regression?
No; it looks pretty ancient, but if there's a 0.17.1, including this would only
make things better.
> map outputs should not be renamed between partitions
> ----------------------------------------------------
>
> Key: HADOOP-3443
> URL: https://issues.apache.org/jira/browse/HADOOP-3443
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Priority: Critical
> Attachments: hadoop-3443-1.patch
>
>
> If a map finishes with out having to spill its data buffer, the map outputs
> are sorted and written to disk. However, no care is taken to make sure that
> the same partition is used to write it out before it is renamed. On nodes
> with multiple disks assigned to the task trackers, this will likely cause an
> addition read/write cycle to disk that is very expensive.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
