Improve the performance efficiency of task initialization at the JobTracker
---------------------------------------------------------------------------
Key: HADOOP-6026
URL: https://issues.apache.org/jira/browse/HADOOP-6026
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Reporter: dhruba borthakur
Assignee: Zheng Shao
The JobTracker reads the splits for a job at Job Initialization time. Then, for
each location in the split, it invokes DNSToSwitchMapping.resolve(). This, in
turn, typically invokes an external script that resolves the hostname to a
network rack location. The time spent in invoking this external script can be
reduced if the hostname and their rack locations are inserted into a cache.
JobTracker.resolveAndAddToTopology() can look up this cache first and avoid
invoking the external "resolve" script is most cases.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
