[
https://issues.apache.org/jira/browse/HADOOP-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sharad Agarwal updated HADOOP-3326:
-----------------------------------
Attachment: 3326_1.patch
Attaching a patch. It does following:-
1. LocalFSMerger and InMemFSMergeThread threads are spawned only once in
fetchOutputs() method. The thread gets notified when something is ready for
merge. The merge happens when thresholds are met.
2. Instead of sleeping (in loop) to wait for finishing the merger threads, now
it does join.
3. The in memory merge code was duplicate at 2 places, now the common code has
been moved to doInMemMerge() method.
> ReduceTask should not sleep for 200 ms while waiting for merge to finish
> ------------------------------------------------------------------------
>
> Key: HADOOP-3326
> URL: https://issues.apache.org/jira/browse/HADOOP-3326
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Sharad Agarwal
> Attachments: 3326_1.patch
>
>
> Currently the merge code in Reduce task does:
> {code}
> // Wait for the on-disk merge to complete
> while (localFSMergeInProgress) {
> Thread.sleep(200);
> }
>
> //wait for an ongoing merge (if it is in flight) to complete
> while (mergeInProgress) {
> Thread.sleep(200);
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
