Hadoop does not implemented the clever task scheduler, when a data node heartbeat with the namenode, and if the data node wants a job, simply get one for it. The selection does not consider the task's input file at all.
Best regards, Yongqiang He 2008-06-25 发件人: Saptarshi Guha 发送时间: 2008-06-30 21:12:24 收件人: [email protected] 抄送: 主题: Data-local tasks Hello, I recall asking this question but this is in addition to what I'ev askd. Firstly, to recap my question and Arun's specific response: -- On May 20, 2008, at 9:03 AM, Saptarshi Guha wrote: > Hello, > -- Does the "Data-local map tasks" counter mean the number of tasks that the had the input data already present on the machine on they are running on? -- i.e the wasn't a need to ship the data to them. Response from Arun -- Yes. Your understanding is correct. More specifically it means that the map-task got scheduled on a machine on which one of the -- replicas of it's input-split-block was present and was served by the datanode running on that machine. *smile* Arun Now, Is Hadoop designed to schedule a map task on a machine which has one of the replicas of it's input split block? Failing that, does then assign a map task on machine close to one that contains a replica of it's input split block? Are there any performance metrics for this? Many thanks Saptarshi Saptarshi Guha | [EMAIL PROTECTED] | http://www.stat.purdue.edu/~sguha
