Hey Mark, Yes, you can limit the memory for each task with "mapred.child.java.opts" property. Set this to final if no developer has to change it .
Little intro to "mapred.task.default.maxvmem" This property has to be set on both the JobTracker for making scheduling decisions and on the TaskTracker nodes for the sake of memory management. If a job doesn't specify its virtual memory requirement by setting mapred.task.maxvmem to -1, tasks are assured a memory limit set to this property. This property is set to -1 by default. This value should in general be less than the cluster-wide configuration mapred.task.limit.maxvmem. If not or if it is not set, TaskTracker's memory management will be disabled and a scheduler's memory based scheduling decisions may be affected. On Wed, Feb 15, 2012 at 5:57 PM, Mark question <[email protected]> wrote: > Hi, > > My question is what's the difference between the following two settings: > > 1. mapred.task.default.maxvmem > 2. mapred.child.java.opts > > The first one is used by the TT to monitor the memory usage of tasks, while > the second one is the maximum heap space assigned for each task. I want to > limit each task to use upto say 100MB of memory. Can I use only #2 ?? > > Thank you, > Mark -- -- Srinivas [email protected]
