[
https://issues.apache.org/jira/browse/MAPREDUCE-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Binglin Chang updated MAPREDUCE-3612:
-------------------------------------
Description:
We recently have done some tests to evaluate performances of different Hadoop
versions(1.0, 0.23, Baidu internal version), and found some weird results. One
of them is in 1.0 Task.TaskReporter.done() takes too much time, about 2s, this
is bad for small tasks. After reviewing source code and add some log, the
following code block Task.TaskReporter.done
{code:title=src/mapred/org/apache/hadoop/mapred/Task.java}
658 try {
659 Thread.sleep(PROGRESS_INTERVAL);
660 }
723 public void stopCommunicationThread() throws InterruptedException {
724 // Updating resources specified in ResourceCalculatorPlugin
725 if (pingThread != null) {
726 synchronized(lock) {
727 while(!done) {
728 lock.wait();
729 }
730 }
731 pingThread.interrupt();
732 pingThread.join();
733 }
734 }
{code}
Originally line 724-730 don't exists, I don't know why it is added. If it is
needed, we can replace Thread.sleep with Object.wait(timeout) and Object.notify
instead, so it won't block.
was:
We recently have done some tests to evaluate performances of different Hadoop
versions(1.0, 0.23, Baidu internal version), and found some weird results. One
of them is in 1.0 Task.TaskReporter.done() takes too much time, about 2s, this
is bad for small tasks. After reviewing source code and add some log, the
following code block Task.TaskReporter.done
src/mapred/org/apache/hadoop/mapred/Task.java
658 try {
659 Thread.sleep(PROGRESS_INTERVAL);
660 }
723 public void stopCommunicationThread() throws InterruptedException {
724 // Updating resources specified in ResourceCalculatorPlugin
725 if (pingThread != null) {
726 synchronized(lock) {
727 while(!done) {
728 lock.wait();
729 }
730 }
731 pingThread.interrupt();
732 pingThread.join();
733 }
734 }
Originally line 724-730 don't exists, I don't know why it is added. If it is
needed, we can replace Thread.sleep with Object.wait(timeout) and Object.notify
instead, so it won't block.
Target Version/s: 1.0.0, 0.20.205.0 (was: 0.20.205.0, 1.0.0)
> Task.TaskReporter.done method blocked for some time when task is finishing
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-3612
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3612
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Binglin Chang
>
> We recently have done some tests to evaluate performances of different Hadoop
> versions(1.0, 0.23, Baidu internal version), and found some weird results.
> One of them is in 1.0 Task.TaskReporter.done() takes too much time, about 2s,
> this is bad for small tasks. After reviewing source code and add some log,
> the following code block Task.TaskReporter.done
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java}
> 658 try {
> 659 Thread.sleep(PROGRESS_INTERVAL);
> 660 }
> 723 public void stopCommunicationThread() throws InterruptedException {
> 724 // Updating resources specified in ResourceCalculatorPlugin
> 725 if (pingThread != null) {
> 726 synchronized(lock) {
> 727 while(!done) {
> 728 lock.wait();
> 729 }
> 730 }
> 731 pingThread.interrupt();
> 732 pingThread.join();
> 733 }
> 734 }
> {code}
> Originally line 724-730 don't exists, I don't know why it is added. If it is
> needed, we can replace Thread.sleep with Object.wait(timeout) and
> Object.notify instead, so it won't block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
