Ankur Dave created SPARK-1991:
---------------------------------
Summary: Support custom StorageLevels for vertices and edges
Key: SPARK-1991
URL: https://issues.apache.org/jira/browse/SPARK-1991
Project: Spark
Issue Type: Improvement
Components: GraphX
Reporter: Ankur Dave
Assignee: Ankur Dave
Large graphs may not fit entirely in memory. If we supported custom storage
levels for the vertices and edges of a graph, the user could specify
MEMORY_AND_DISK and then repartition the graph to use many small partitions,
each of which does fit in memory. Spark would then automatically load
partitions from disk as needed.
Also, the replicated storage levels would be helpful for fault tolerance, and
the serialized ones would improve efficiency for non-primitive vertex and edge
attributes.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
