[
https://issues.apache.org/jira/browse/HADOOP-868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj Das updated HADOOP-868:
-------------------------------
Status: Patch Available (was: Open)
The patch fixes the following:
1) Removes a redundant file "open" method call from mergeParts
2) All index files are not kept open at once. Now, the index file for a spill
is opened, the offsets are read after "seek"ing to the correct place, and then
the file is immediately closed.
3) The hardcoding of merge factor to the number of spills is not done; merge
factor used for the merges is the default (configured via io.sort.factor).
> Fix the merge method on Maps to limit the number of open files
> --------------------------------------------------------------
>
> Key: HADOOP-868
> URL: https://issues.apache.org/jira/browse/HADOOP-868
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Devaraj Das
> Assigned To: Devaraj Das
> Attachments: 868.patch
>
>
> The mergeParts method should not hardcode the factor for the merge and
> instead rely on the configured default. Also, there is an unnecessary open
> call made in the method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
