[
https://issues.apache.org/jira/browse/MESOS-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199464#comment-14199464
]
Vinod Kone commented on MESOS-1711:
-----------------------------------
commit c7227471f98c0dc62c8700d41534be142a3fcfad
Author: Ankur Chauhan <[email protected]>
Date: Wed Nov 5 16:33:00 2014 -0800
Fetcher uses Hadoop to fetch URIs with unknown schemes.
Review: https://reviews.apache.org/r/27483
> Create method for users to identify HDFS compatible protocols in fetcher.cpp
> ----------------------------------------------------------------------------
>
> Key: MESOS-1711
> URL: https://issues.apache.org/jira/browse/MESOS-1711
> Project: Mesos
> Issue Type: Improvement
> Components: general
> Affects Versions: 0.19.1
> Environment: All
> Reporter: John Omernik
> Assignee: Timothy St. Clair
> Priority: Minor
> Labels: fetcher, hadoop, hdfs
> Fix For: 0.22.0
>
> Original Estimate: 6h
> Remaining Estimate: 6h
>
> In fetcher.cpp, the code to get the Mesos packages uses a hard coded list of
> protocols to determine if the Hadoop copytoLocal method is used or if another
> method (such as standard filecopy). This limits the addition of new
> protocols that are HDFS compatible until the next release of Mesos. Tachyon
> Filesystem (tachyonfs://), MapR FS (maprfs://) and glusterfs:// are three
> examples that could make use of this.
> Instead of just adding those file systems in the hard coded list, I recommend
> following the lead of the Tachyon Project. In tachyon-0.6.0-SNAPSHOT, they
> have added an environment variable of allowed hdfs compatible protocols. This
> comma-separated list allows the user/admin to specify which protocols are
> HDFS compatible, without hard coding it in the fetcher.cpp.
> I don't have access to the Tachyon issues list for linking, but the code is
> on line 75 of
> https://github.com/amplab/tachyon/blob/master/core/src/main/java/tachyon/UnderFileSystem.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
