[
https://issues.apache.org/jira/browse/TINKERPOP3-925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marko A. Rodriguez closed TINKERPOP3-925.
-----------------------------------------
Resolution: Fixed
> Use persisted SparkContext to persist an RDD across Spark jobs.
> ---------------------------------------------------------------
>
> Key: TINKERPOP3-925
> URL: https://issues.apache.org/jira/browse/TINKERPOP3-925
> Project: TinkerPop 3
> Issue Type: Improvement
> Components: hadoop
> Affects Versions: 3.0.2-incubating
> Reporter: Marko A. Rodriguez
> Assignee: Marko A. Rodriguez
> Fix For: 3.1.0-incubating
>
>
> If a provider is using Spark, they are currently forced to have HDFS be used
> to store intermediate RDD data. However, if they plan on using that data in a
> {{GraphComputer}} "job chain," then they should be able to lookup a
> {{.cached()}} RDD by name.
> Create a {{inputGraphRDD.name}} and {{outputGraphRDD.name}} to make it so
> that the configuration references {{SparkContext.getPersitedRDDs()}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
