This has been discussed in past. There is no current dynamic way to control the parallel execution on a per-node basis.
Scheduler configurations will let you control overall parallelism (# of simultaneous tasks) of specific jobs on a cluster-level basis, but not on a per-node level. On Sat, May 31, 2014 at 4:08 AM, jeremy p <[email protected]> wrote: > Hello all, > > I have two jobs, Job A and Job B. Job A is not very CPU-intensive, and so > we would like to run it with 50 mappers per node. Job B is very > CPU-intensive, and so we would like to run it with 25 mappers per node. How > can we request a different number of mappers per node for each job? From > what I've read, mapred.tasktracker.map.tasks.maximum and > mapred.tasktracker.reduce.tasks.maximum cannot be overridden from the > client. > > --Jeremy -- Harsh J
