reduce or remove usage of String.format() usage in
CapacityTaskScheduler.updateQSIObjects
-----------------------------------------------------------------------------------------
Key: MAPREDUCE-1533
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1533
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.20.1
Reporter: Rajesh Balamohan
When short jobs are executed in hadoop with OutOfBandHeardBeat=true, JT
executes heartBeat() method heavily. This internally makes a call to
CapacityTaskScheduler.updateQSIObjects().
CapacityTaskScheduler.updateQSIObjects(), internally calls String.format() for
setting the job scheduling information. Based on the datastructure size of
"jobQueuesManager" and "queueInfoMap", the number of times String.format() gets
executed becomes very high. String.format() internally does pattern matching
which turns to be out very heavy (This was revealed while profiling JT. Almost
57% of time was spent in CapacityScheduler.assignTasks(), out of which
String.format() took 46%.
Would it be possible to do String.format() only at the time of invoking
JobInProgress.getSchedulingInfo?. This might reduce the pressure on JT while
processing heartbeats.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
