Praveen, The mapred.job.map.memory.mb is a config that Hadoop uses to decided if your task is using too much memory and needs to be killed. The memory used by this includes not only the memory for java itself but also any other child processes that may be launched by java. The -Xmx in mapred.child.java.opts tells the JVM being launched how big of a heap to use. Typically java itself will use a bit more memory then just what is stored in the heap so -Xmx should be slightly smaller then mapred.job.map.memory.mb. But, if you are going to be launching sub-processes, like streaming tends to do, then you will typically want to have -Xmx be much smaller then mapred.job.map.memory.mb so that it can allow for the RAM that the native process will be using.
--Bobby Evans On 11/7/11 7:29 PM, "Praveen Sripati" <praveensrip...@gmail.com> wrote: Hi, Can someone please clarify me on the below query? Thanks, Praveen On Sun, Nov 6, 2011 at 8:47 PM, Praveen Sripati <praveensrip...@gmail.com> wrote: Hi, What is the difference between setting the mapred.job.map.memory.mb and mapred.child.java.opts using -Xmx to control the maximum memory used by a Mapper and Reduce task? Which one takes precedence? Thanks, Praveen
