On Feb 27, 2008, at 7:29 PM, Zhang, Guibin wrote:
Question: How can I configure the hadoop to distribute less
tasks to weak nodes and distribute more tasks to strong nodes?
mapred.tasktracker.map.tasks.maximum is the number of tasks to run on
that task track simultaneously. 75 is almost certainly too high.
mapred.map.tasks is only relevant on the submitting node, because
that is where the planning takes place. More reasonable values
(depending on your hardware) are:
strong:
mapred.tasktracker.map.tasks.maximum = 8
mapred.tasktracker.reduce.tasks.maximum = 4
weak:
mapred.tasktracker.map.tasks.maximum = 4
mapred.tasktracker.reduce.tasks.maximum = 2
client:
mapred.map.tasks = nodes * avgerage
(mapred.tasktracker.map.tasks.maximum)
mapred.reduce.tasks = 95% * nodes * average
(mapred.tasktracker.reduce.tasks.maximum)
-- Owen
