Jianfeng created SPARK-1531:
-------------------------------
Summary: GraphX should have messageRDD to enable OutOfCore messages
Key: SPARK-1531
URL: https://issues.apache.org/jira/browse/SPARK-1531
Project: Spark
Issue Type: Improvement
Components: GraphX
Affects Versions: 0.9.1
Reporter: Jianfeng
There is no such `messageRDD` in Pregel function.
Most of the sendMessage is directly sent one Scala Iterator. Like the below one
in staticPageRank:
```
def sendMessage(edge: EdgeTriplet[Double, Double]) =
Iterator((edge.dstId, edge.srcAttr * edge.attr))
```
For some message intensively computation on some bigger graph, it will throw
OOM exceptions. If we have some more general messageRDD, at lease we can set
MessageRDD.persist(DISK) to enable it flush onto the disk.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
