[
https://issues.apache.org/jira/browse/HADOOP-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-19200.
-------------------------------------
Resolution: Duplicate
> Reduce the number of headObject when opening a file with the s3 file system
> ---------------------------------------------------------------------------
>
> Key: HADOOP-19200
> URL: https://issues.apache.org/jira/browse/HADOOP-19200
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Affects Versions: 3.4.0, 3.3.6
> Reporter: Oliver Caballero Alvarez
> Priority: Major
>
> In the implementation of the S3 filesystem, of the hadoop aws package, if you
> use it with spark, every time you open a file for anything you will have to
> send two Head Objects, since to open the file, you will first look to see if
> this file exists, executing a HeadObject, and then when opening it, the
> implementation, both of sdk1 and sdk2, forces you to make a head object
> again. This is not the fault of the implementation of this class
> (S3AFileSystem), but of the abstract FileSystem class of the Hadoop core,
> since it does not allow the FileStatus to be passed but only allows the use
> of Path.
> If the FileSystem implementation is changed, it could be used to not have to
> request that HeadObject again.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
