[
https://issues.apache.org/jira/browse/TINKERPOP3-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716734#comment-14716734
]
Marko A. Rodriguez commented on TINKERPOP3-616:
-----------------------------------------------
This is going to be nasty. I think this is the way forward.
{code}
spark-gremlin
giraph-gremlin
hadoop1-gremlin
hadoop2-gremlin
{code}
I think we can make it so that {{spark-gremlin}} and {{giraph-gremlin}} do NOT
depend on any Hadoop libraries. Next, {{hadoop1-gremlin}} and
{{hadoop2-gremlin}} would (hopefully) be able to depend on something like
{{hadoop-core-gremlin}} so all the HDFS, InputFormat, etc. classes don't have
to be copied over (it all depends on API compatability between the Hadoop1 and
Hadoop2). If I remember correctly, there aren't compatibility issues at this
level (HDFS API stayed the same). So, we might be lucky. MAYBE we don't even
need to make a distinction between Hadoop1 and Hadoop2 cause MAYBE there is a
single library we can depend on in Hadoop's ecosystem that crosses both
versions... ??????? That'd be bad ass.
Next, we should rewrite {{giraph-gremlin}} to support Giraph 1.1.0 (dah).
Finally {{spark-gremlin}} at this point can version with Spark's evolution
without that nasty convolution of jars with Giraph and Hadoop.
> Use Spark 1.3.0 in Hadoop-Gremlin.
> ----------------------------------
>
> Key: TINKERPOP3-616
> URL: https://issues.apache.org/jira/browse/TINKERPOP3-616
> Project: TinkerPop 3
> Issue Type: Improvement
> Components: hadoop
> Affects Versions: 3.0.0-incubating
> Reporter: Marko A. Rodriguez
> Assignee: Marko A. Rodriguez
> Fix For: 3.1.0-incubating
>
>
> Spark 1.3.0 was released approximately 2 weeks ago. Release M9 with 1.3.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
