[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18684:
------------------------------------
Description:
Allow s3a filesystem to be bindable to other filesystem schemas, especially
s3://
* FileContext API has hard coded use of "s3a"
* S3AFileSystem.getScheme() needs to pick up the scheme of the URI passed to
initialize()
* plus tests
was:Certain codepaths use the FileContext API's to perform FS based
operations such as yarn log aggregations. While trying to reuse the S3A
connector for GCS based workloads the yarn log aggregation was not happening.
Upon further investigation it was observed that FileContext API have hardcoded
URI scheme checks that need to disabled/updated to make S3A compatible with non
AWS stores.
> S3A filesystem to support binding to to other URI schemes
> ---------------------------------------------------------
>
> Key: HADOOP-18684
> URL: https://issues.apache.org/jira/browse/HADOOP-18684
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 3.3.5
> Reporter: Harshit Gupta
> Priority: Major
> Labels: pull-request-available
>
> Allow s3a filesystem to be bindable to other filesystem schemas, especially
> s3://
> * FileContext API has hard coded use of "s3a"
> * S3AFileSystem.getScheme() needs to pick up the scheme of the URI passed to
> initialize()
> * plus tests
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
