[
https://issues.apache.org/jira/browse/TINKERPOP-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322811#comment-15322811
]
ASF GitHub Bot commented on TINKERPOP-1331:
-------------------------------------------
GitHub user dkuppitz opened a pull request:
https://github.com/apache/incubator-tinkerpop/pull/334
TINKERPOP-1331 HADOOP_GREMLIN_LIBS can only point to local file system
https://issues.apache.org/jira/browse/TINKERPOP-1331
It's now possible to have DFS paths in `HADOOP_GREMLIN_LIBS`.
VOTE: +1
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-tinkerpop TINKERPOP-1331
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-tinkerpop/pull/334.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #334
----
commit 447941ded4d5fecd731036e7f5df748edca92273
Author: Daniel Kuppitz <[email protected]>
Date: 2016-06-09T11:46:34Z
Allow DFS paths in `HADOOP_GREMLIN_LIBS`.
commit 91475d3e85946e400264448d0a9730e7301ca9ed
Author: Daniel Kuppitz <[email protected]>
Date: 2016-06-09T11:48:17Z
Updated CHANGELOG
commit 2ccae55e893b92c1f58b7b878ffa5a27b4828afa
Author: Daniel Kuppitz <[email protected]>
Date: 2016-06-09T16:28:30Z
Minor tweaks and support for more path separators in `HADOOP_GREMLIN_LIBS`
(e.g. `;` on Windows systems).
----
> HADOOP_GREMLIN_LIBS can only point to local file system
> -------------------------------------------------------
>
> Key: TINKERPOP-1331
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1331
> Project: TinkerPop
> Issue Type: Improvement
> Components: hadoop
> Affects Versions: 3.2.0-incubating, 3.1.2-incubating
> Reporter: Daniel Kuppitz
> Assignee: Daniel Kuppitz
>
> These two lines in {{SparkGraphComputer}} assume that {{HADOOP_GREMLIN_LIBS}}
> will only contain local file system references (although it seems that the
> rest of the code could handle DFS references):
> {code}
> final String[] paths = hadoopGremlinLocalLibs.split(":");
> final FileSystem fs = FileSystem.get(hadoopConfiguration);
> {code}
> If, for example, {{HADOOP_GREMLIN_LIBS}} would be set to
> {{hdfs:///spark-gremlin-libs:/foo/bar}}, the {{split(":")}} call would
> obviously separate the file system scheme ({{hdfs://}}) from the path
> ({{/spark-gremlin-libs}}).
> Next, {{FileSystem.get(hadoopConfiguration)}} will always only return a
> reference to the {{FileSystem}} that is defined as the default file system.
> The same is probably true for {{GiraphGraphComputer}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
