On Mon, Aug 18, 2014 at 6:29 AM, Yifan LI <iamyifa...@gmail.com> wrote:
> I am testing our application(similar to "personalised page rank" using > Pregel, and note that each vertex property will need pretty much more space > to store after new iteration) [...] But when we ran it on larger graph(e.g. LiveJouranl), it always end at the > error "GC overhead limit exceeded", even the partitions number is increased > to 48 from 8. If the graph (including vertex properties) is too large to fit in memory, you might try allowing it to spill to disk. When constructing the graph, you can set vertexStorageLevel and edgeStorageLevel to StorageLevel.MEMORY_AND_DISK. This should allow the algorithm to finish. Ankur <http://www.ankurdave.com/>
