For streaming, you just need enough space to the run the child that forks and handles the streaming mapper/reducer and enough space to handle sorting the date with out the task getting stuck in GC hell.
On Fri, Sep 11, 2009 at 12:30 PM, Mayuran Yogarajah < [email protected]> wrote: > Is there any sense in setting mapred.child.java.opts to a high value > if all we're using is Hadoop streaming ? We've set it to 512MB but > I don't know if it even matters.. > > thanks > -- Pro Hadoop, a book to guide you from beginner to hadoop mastery, http://www.amazon.com/dp/1430219424?tag=jewlerymall www.prohadoopbook.com a community for Hadoop Professionals
