Jean-Baptiste Onofré commented on BEAM-2005:

I think there's two topics:
- where to put the code itself
- what dependency artifacts do we bring with the core (via a BoM for example)

Definitely, I think it's better to have HDFS filesystem outside of the core, in 
an extension.

Now, from an user perspective, this extension can come by default with the 
dependency set (again using a BoM for instance).

> Add a Hadoop FileSystem implementation of Beam's FileSystem
> -----------------------------------------------------------
>                 Key: BEAM-2005
>                 URL: https://issues.apache.org/jira/browse/BEAM-2005
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-extensions
>            Reporter: Stephen Sisk
>            Assignee: Stephen Sisk
>             Fix For: First stable release
> Beam's FileSystem creates an abstraction for reading from files in many 
> different places. 
> We should add a Hadoop FileSystem implementation 
> (https://hadoop.apache.org/docs/r2.8.0/api/org/apache/hadoop/fs/FileSystem.html)
>  - that would enable us to read from any file system that implements 
> FileSystem (including HDFS, azure, s3, etc..)
> I'm investigating this now.

This message was sent by Atlassian JIRA

Reply via email to