[
https://issues.apache.org/jira/browse/HADOOP-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705909#action_12705909
]
Jothi Padmanabhan commented on HADOOP-5572:
-------------------------------------------
Some initial comments:
# Ensure that the sum of weights for a phase does not cross 1
# Having a boolean variable to keep track of whether the weights are fixed or
variable is a better option
# Merger -- Sort the segments only if numSegments > factor
# Relying on writesCounter to decide includeFinalMerge variable is not a good
idea.
# computeBytesInMerges should disregard empty segments -- we probably need to
add a isEmpty() API to Segment.
> The map progress value should have a separate phase for doing the final sort.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-5572
> URL: https://issues.apache.org/jira/browse/HADOOP-5572
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5572.patch
>
>
> Currently, the final spill and sort doesn't record any progress while it
> runs, leading to the perception that the map is done, but "stuck".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
