DJ Hoffman created HADOOP-15314:
-----------------------------------
Summary: Scheme assertion in DynamoDBMetadataStore::checkPath is
unnecessarily restrictive
Key: HADOOP-15314
URL: https://issues.apache.org/jira/browse/HADOOP-15314
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Affects Versions: 3.0.0
Reporter: DJ Hoffman
In version 3.0.0, the checkPath method for dealing with paths prevents us from
using the s3:// scheme when utilizing S3Guard. However, in our core-site.xml we
have included
{noformat}
<property>
<name>fs.s3.impl</name>
<value>org.apache.hadoop.fs.s3a.S3AFileSystem</value>
</property>{noformat}
which should enforce that s3 prefixed paths go through s3a and are properly
compatible with s3guard. We removed the assertion that paths use the s3a scheme
(some of our paths use the s3 scheme) and our testing thus far with S3Guard
enabled have been positive. We believe the assertion in checkPath is
unnecessary and could be expanded to include the s3 and s3n schemes if not
dropped altogether or altered in some other way. We're happy to develop and
test a patch if the community is amenable to the change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
