Benoit Perroud created MAPREDUCE-5736:
-----------------------------------------
Summary: Jobtracker to hang when jobs with lot of tasks running
Key: MAPREDUCE-5736
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5736
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Reporter: Benoit Perroud
Priority: Minor
The jobtracker (in MRv1) is progressing slowly when a job with lot of tasks is
running. The reason is that JT.getJobCounters is holding a global lock, and
with a big job (like 50+K mappers for instance), it could take while to
instanciate the ``Counters`` class. This global lock prevent all other
activities to run normally, queuing them and degrading the normal activities of
the JT.
I was wondering if job.getCounters(), which is synchronized on a finer
granularity (i.e. per job and not global) couldn't be taken out of the
synchronized block.
On a more general idea, I was wondering if the usage of the synchronized
statement in the JT shouldn't be re-thought. Or maybe all this has already been
addressed in YARN.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
