[
https://issues.apache.org/jira/browse/TINKERPOP-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088323#comment-15088323
]
ASF GitHub Bot commented on TINKERPOP-1033:
-------------------------------------------
Github user okram commented on the pull request:
https://github.com/apache/incubator-tinkerpop/pull/192#issuecomment-169832850
Here is the thing. `SparkLoader` was introduced in 3.1.1-SNAPSHOT :D so it
okay to drop. `HadoopLoader` is all meta-programming Groovy stuff to get ls(),
rm(), etc. to work in Gremlin Console. We can keep the the class, but we can't
have it loaded else it will interfere with the new `FileSystemStorage`.
However, I say we just drop it. Its so low level and all meta-programmy that if
someone is using it, they are retarded.
Finally, `HDFSTools`. Again, low level.... I can bring that class back, but
people really shouldn't be using it. This is like an internal utility and so
specific to TinkerPop filesystem stuff. ??
> Store sideEffects as a persisted RDD
> ------------------------------------
>
> Key: TINKERPOP-1033
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1033
> Project: TinkerPop
> Issue Type: Improvement
> Components: hadoop
> Affects Versions: 3.1.0-incubating
> Reporter: Marko A. Rodriguez
> Assignee: Marko A. Rodriguez
> Fix For: 3.1.1-incubating
>
>
> I think we can completely get away from HDFS for {{SparkGraphComputer}}. We
> will need something like {{PesistedSideEffectsRDD}}. Once we do that, if the
> user wants to use Spark without Hadoop, its possible.
> This would beg the question -- do we go all the way and support
> {{SparkGraph}} ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
