Steve Loughran created HADOOP-15229:
---------------------------------------

             Summary: Add FileSystem builder-based open API to match create()
                 Key: HADOOP-15229
                 URL: https://issues.apache.org/jira/browse/HADOOP-15229
             Project: Hadoop Common
          Issue Type: New Feature
          Components: fs, fs/azure, fs/s3
    Affects Versions: 3.0.0
            Reporter: Steve Loughran


Replicate HDFS-1170 and HADOOP-14365 with an API to open files.

A key requirement of this is not HDFS, it's to put in the fadvise policy for 
working with object stores, where getting the decision to do a full GET and TCP 
abort on seek vs smaller GETs is fundamentally different: the wrong option can 
cost you minutes. S3A and Azure both have adaptive policies now (first backward 
seek), but they still don't do it that well.

Columnar formats (ORC, Parquet) should be able to say "fs.input.fadvise" 
"random" as an option when they open files; I can imagine other options too.

The Builder model of [~eddyxu] is the one to mimic, method for method. Ideally 
with as much code reuse as possible



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

Reply via email to