[
https://issues.apache.org/jira/browse/HADOOP-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sharad Agarwal updated HADOOP-3326:
-----------------------------------
Attachment: 3326_3.patch
Mahadev, Thanks for the review. Have incorporated your comments. Instead of
interrupt, using boolean flags to exit from the thread.
Also, removed the last inMemory merge from fetchOutputs. Now last inMem merge
is also handled by the InMemFSMergeThread.
Tested sort+validate on 50 nodes cluster.
> ReduceTask should not sleep for 200 ms while waiting for merge to finish
> ------------------------------------------------------------------------
>
> Key: HADOOP-3326
> URL: https://issues.apache.org/jira/browse/HADOOP-3326
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Sharad Agarwal
> Fix For: 0.18.0
>
> Attachments: 3326_1.patch, 3326_2.patch, 3326_3.patch
>
>
> Currently the merge code in Reduce task does:
> {code}
> // Wait for the on-disk merge to complete
> while (localFSMergeInProgress) {
> Thread.sleep(200);
> }
>
> //wait for an ongoing merge (if it is in flight) to complete
> while (mergeInProgress) {
> Thread.sleep(200);
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
