On Fri, Jun 27, 2008 at 08:57, Chris Anderson <[EMAIL PROTECTED]> wrote:
> The problem is that when there are a large number of map tasks to > complete, Hadoop doesn't seem to obey the map.tasks.maximum. Instead, > it is spawning 8 map tasks per tasktracker (even when I change the > mapred.tasktracker.map.tasks.maximum in hadoop-site.xml to 2, on the > master). The cluster was booted with the setting at 8. Do I need to > change hadoop-site.xml on all the slaves, and restart the task > trackers, in order to make the limit apply? That seems unlikely - I'd > really like to manage this parameter on a per-job level. > Yes, mapred.tasktracker.map.tasks.maximum is configured per tasktracker on startup. It can't be configured per job because it's not a job-scope parameter (if there are multiple concurrent jobs, they have to share the task limit). rick
