[
https://issues.apache.org/jira/browse/MAPREDUCE-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249084#comment-13249084
]
Luke Lu commented on MAPREDUCE-3612:
------------------------------------
See MAPREDUCE-2450 for why this is needed. We saw a fairly significant
performance regression (vs. 0.20.2) in Hadoop 1.x for some small jobs workloads
due to this issue as well.
Regarding the patch, I don't think it address the issue in MAPREDUCE-2450. Will
post a patch later.
> Task.TaskReporter.done method blocked for some time when task is finishing
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-3612
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3612
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Binglin Chang
> Assignee: Binglin Chang
> Attachments: MAPREDUCE-3612.patch
>
>
> We recently have done some tests to evaluate performances of different Hadoop
> versions(1.0, 0.23, Baidu internal version), and found some weird results.
> One of them is in 1.0 Task.TaskReporter.done() takes too much time, about 2s,
> this is bad for small tasks. After reviewing source code and add some log,
> the following code block Task.TaskReporter.done
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java}
> 658 try {
> 659 Thread.sleep(PROGRESS_INTERVAL);
> 660 }
> 723 public void stopCommunicationThread() throws InterruptedException {
> 724 // Updating resources specified in ResourceCalculatorPlugin
> 725 if (pingThread != null) {
> 726 synchronized(lock) {
> 727 while(!done) {
> 728 lock.wait();
> 729 }
> 730 }
> 731 pingThread.interrupt();
> 732 pingThread.join();
> 733 }
> 734 }
> {code}
> Originally line 724-730 don't exists, I don't know why it is added. If it is
> needed, we can replace Thread.sleep with Object.wait(timeout) and
> Object.notify instead, so it won't block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
