On Tue, Apr 26, 2011 at 6:02 AM, Juwei Shi <[email protected]> wrote:
As you know, Hadoop MapReduce start child JVM processes to run task. I want > to start the process of m/r task myself thus I can pass some OS level > parameters to the JVM process. > That currently isn't possible. The best that is available is mapred.child.java.opts, which adds options to the child jvm. Alternatively, you could fork a subprocess out of the map or reduce like streaming does. Actually, I guess you could use streaming itself and get control over how your code runs. In MapReduce v2, which is described here http://developer.yahoo.com/blogs/hadoop/posts/2011/02/mapreduce-nextgen/ , the user gets a lot more control over the task launching. In particular, mapreduce becomes a user-land library controlled by the user. -- Owen
