[
https://issues.apache.org/jira/browse/MAPREDUCE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13595550#comment-13595550
]
Mariappan Asokan commented on MAPREDUCE-3685:
---------------------------------------------
Hi Ravi,
I guess I am too late to comment since your patch has been committed already.
In any case, I have the following comments since you asked:)
* In {{closeOnDiskFile()}} the following lines of code
{code}
if (onDiskMapOutputs.size() >= (2 * ioSortFactor - 1)) {
onDiskMerger.startMerge(onDiskMapOutputs);
}
{code}
can be changed to
{code}
if (onDiskMapOutputs.size() >= ioSortFactor) {
onDiskMerger.startMerge(onDiskMapOutputs);
}
{code}
Please confirm.
* In the class {{CompressAwarePath}} there is a nit in {{compareTo().}} The
following lines:
{code}
} else if (this.getCompressedSize() > compPath.getCompressedSize()) {
return 1;
{code}
can be simplified as:
{code}
} else {
return 1;
{code}
The set will be partially ordered without an additional compare and without
executing the line
{code}
return super.compareTo(obj);
{code}
* Since the patch fixes some performance issues, did you have a chance to run
some benchmarks that show improvements? I know this will take some time. I
will leave it to you.
-- Asokan
> There are some bugs in implementation of MergeManager
> -----------------------------------------------------
>
> Key: MAPREDUCE-3685
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3685
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.1
> Reporter: anty.rao
> Assignee: anty
> Priority: Critical
> Fix For: 0.23.7, 2.0.4-beta
>
> Attachments: MAPREDUCE-3685-branch-0.23.1.patch,
> MAPREDUCE-3685-branch-0.23.1.patch, MAPREDUCE-3685-branch-0.23.1.patch,
> MAPREDUCE-3685.branch-0.23.patch, MAPREDUCE-3685.branch-0.23.patch,
> MAPREDUCE-3685.branch-0.23.patch, MAPREDUCE-3685.branch-0.23.patch,
> MAPREDUCE-3685.branch-0.23.patch, MAPREDUCE-3685.patch, MAPREDUCE-3685.patch,
> MAPREDUCE-3685.patch, MAPREDUCE-3685.patch, MAPREDUCE-3685.patch,
> MAPREDUCE-3685.patch, MAPREDUCE-3685.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
