[
https://issues.apache.org/jira/browse/TINKERPOP-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046009#comment-15046009
]
ASF GitHub Bot commented on TINKERPOP-1027:
-------------------------------------------
Github user okram commented on the pull request:
https://github.com/apache/incubator-tinkerpop/pull/172#issuecomment-162709345
I made it so that SparkGremlin works like Spark JobServer
(https://github.com/spark-jobserver/spark-jobserver/). It ensures that RDDs are
not garbage collected by maintaining a static `Spark` class that holds a
`ConcurrentHashMap` of RDDs. Thus, Spark is like a "file system" in that RDDs
can be `ls()`, `rm()`, etc. This was necessary to get "slow" `mvn clean
install` to build correctly where RDDs are NOT GC'd by Spark context cleaner.
> Merge view prior to writing graphRDD to output format/rdd
> ---------------------------------------------------------
>
> Key: TINKERPOP-1027
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1027
> Project: TinkerPop
> Issue Type: Bug
> Components: hadoop
> Affects Versions: 3.1.0-incubating
> Reporter: Marko A. Rodriguez
> Assignee: Marko A. Rodriguez
> Fix For: 3.1.1-incubating
>
>
> [~dalaro] noted that DSEGraph was not happy with the current {{graphRDD}}
> model when it comes to writing. To make it happy, the view merge needs to
> happen prior to {{graphRDD}} output. Thus, move the {{mapReduceRDD}} view
> merge to before {{graphRDD}} writing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
