[
https://issues.apache.org/jira/browse/GIRAPH-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406816#comment-13406816
]
Jan van der Lugt commented on GIRAPH-234:
-----------------------------------------
I guess you could try to increase the maximum amount of memory used by
mapreduce jobs in Hadoop. By default it is only 200 MB, I believe. Add this to
your mapred-site.xml:
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx4000m</value>
</property>
Since Giraph holds the whole graph in memory, you need a lot of it. Make sure
you are not oversubscribing the amount of memory: If you run two mappers on a
machine with 8GB of memory, set the maximum to about 3500 MB
> Error: GC overhead limit exceeded. Child Error
> org.apache.hadoop.mapred.TaskRunner.run
> --------------------------------------------------------------------------------------
>
> Key: GIRAPH-234
> URL: https://issues.apache.org/jira/browse/GIRAPH-234
> Project: Giraph
> Issue Type: Bug
> Components: bsp, examples, mapreduce
> Affects Versions: 0.1.0
> Reporter: Amani
> Labels: example, giraph
>
> Hello everyone,
> I'm trying to run pregel style implementation of strongly connected
> components algorithm on top of giraph. A user defined writable
> (LongPairWritable) used as the type of the messages between the vertices. The
> algorithm works well in single node. However, whenever I tested using large
> graph (around 1 million vertices) in cluster, a GC overhead limit exceeded
> caused by out of memory exception. The error comes after attempt=0
> super-step=1 with 2 minutes pending. I'm wondering how to solve the problem
> and what causes it?
> Is it possible that LongPairWritable cause the problem?
> I tried a simple test:
> Succeeded job:
> Comput(Iterator<LongPairWritable> msgIterator){
> If(getSuperstep() >=20){
> voteToHalt();
> } else {
> //Do nothing
> }
> }
> Failed job:
> Comput(Iterator<LongPairWritable> msgIterator){
> If(getSuperstep() >=20){
> voteToHalt();
> } else {
> sendToAll(new LongPairWritable(new LongWritable(0),new LongWritable(0));
> }
> }
> Thanks in advance.
> Amani
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
