Hi Yingyi,

Here are some ideas you might want to try:

1)  Limit the thread stack size.

2  You can set the heap available to the mapper jvm.

I.e. Here's a setting to get 10 GB of heap and use a smaller stack (64k) for the threads.

-Dmapred.child.java.opts="-Xms10g -Xmx10g -Xss64k"

Also, you might want to try using the EdgeListVertex instead of Vertex (i.e. GiraphJob.setVertexClass(EdgeListVertex.class)), it is quite a bit smaller.

Let us know if that helps you. You should also check to see if your Hadoop installation is using a 32-bit of 64-bit JVM. If it's 32-bit you will be limited in how much heap you can use.


On 11/17/11 9:38 PM, Yingyi Bu wrote:

I'm running a Giraph PageRank job. I tried with 8GB input text data over 10 nodes (each has 4 core, 4 disks, and 12GB physical memory), that is 800MB input-data/machine. However, Giraph job fails because of high GC costs and Out-of-Memory exception. Do I set some special things in Hadoop configurations, for example, maximum heap size for map task vm ?

Best regards,

Reply via email to