[
https://issues.apache.org/jira/browse/YARN-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997281#comment-15997281
]
Jason Lowe commented on YARN-6556:
----------------------------------
Wouldn't the new HTTP filesystem appear in the Hadoop Common project rather
than YARN? I don't see why it would need to be something specific to YARN.
> Implement a FileSystem that reads from HTTP
> -------------------------------------------
>
> Key: YARN-6556
> URL: https://issues.apache.org/jira/browse/YARN-6556
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Haohui Mai
> Assignee: Haohui Mai
>
> We have a use case where YARN applications would like to localize resources
> from Artifactory. Putting the resources on HDFS itself might not be ideal as
> we would like to leverage Artifactory to manage different versions of the
> resources.
> It would be nice to have something like {{HttpFileSystem}} that implements
> the Hadoop filesystem API and reads from a HTTP endpoint.
> Note that Samza has implemented the proposal by themselves:
> https://github.com/apache/samza/blob/master/samza-yarn/src/main/scala/org/apache/samza/util/hadoop/HttpFileSystem.scala
> The downside of this approach is that it requires the YARN cluster to put the
> Samza jar into the classpath for each NM.
> It would be much nicer for Hadoop to have this feature built-in.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
