Owen, thanks a lot! Is is possible to do this by changing some source code of current MR?
2011/4/26 Owen O'Malley <[email protected]> > On Tue, Apr 26, 2011 at 6:02 AM, Juwei Shi <[email protected]> wrote: > > As you know, Hadoop MapReduce start child JVM processes to run task. I want >> to start the process of m/r task myself thus I can pass some OS level >> parameters to the JVM process. >> > > That currently isn't possible. The best that is available > is mapred.child.java.opts, which adds options to the child jvm. > Alternatively, you could fork a subprocess out of the map or reduce like > streaming does. Actually, I guess you could use streaming itself and get > control over how your code runs. > > In MapReduce v2, which is described here > http://developer.yahoo.com/blogs/hadoop/posts/2011/02/mapreduce-nextgen/ , > the user gets a lot more control over the task launching. In particular, > mapreduce becomes a user-land library controlled by the user. > > -- Owen > -- - Juwei Shi (史巨伟)
