Yes, use the CapacityScheduler and ask for multiple slots per map or reduce task: http://hadoop.apache.org/docs/stable/capacity_scheduler.html#Resource+based+scheduling
Arun On Oct 30, 2012, at 8:49 AM, Marco Zühlke wrote: > Hi, > > on our cluster our jobs usually satisfied with less than 2 GB of heap space. > so we have on our 8 GB computers 3 maps maximum and on our 16 GB > computers 4 maps maximum (we only have quad core CPUs and to have > memory left for reducers). This works very well. > > But now we have a new kind of jobs. Each mapper requires at lest 4 GB > of heap space. > > Is it possible to limit the number of tasks (mapper) per computer to 1 or 2 > for > these kinds of jobs ? > > Regards, > Marco > -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
