Re: PageRank OOM Exception

2011-11-18 Thread Claudio Martella
Thanks, we'll fix that. Meanwhile use this patch to get trunk to build. On Fri, Nov 18, 2011 at 9:28 AM, Yingyi Bu wrote: > Could anyone fix the trunk:  two files miss headers so that build fails... > > Attached is the target/rat.txt from the failed build. > I fixed them locally anyway... > Than

Re: PageRank OOM Exception

2011-11-17 Thread Avery Ching
Hi Yingyi, Here are some ideas you might want to try: 1) Limit the thread stack size. 2 You can set the heap available to the mapper jvm. I.e. Here's a setting to get 10 GB of heap and use a smaller stack (64k) for the threads. -Dmapred.child.java.opts="-Xms10g -Xmx10g -Xss64k" Also, you

PageRank OOM Exception

2011-11-17 Thread Yingyi Bu
Hi, I'm running a Giraph PageRank job. I tried with 8GB input text data over 10 nodes (each has 4 core, 4 disks, and 12GB physical memory), that is 800MB input-data/machine.However, Giraph job fails because of high GC costs and Out-of-Memory exception. Do I set some special things