[
https://issues.apache.org/jira/browse/HADOOP-13516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427740#comment-15427740
]
Shaik Idris Ali edited comment on HADOOP-13516 at 8/19/16 7:19 AM:
-------------------------------------------------------------------
Hi [[email protected]], [~cnauroth],
Thanks for checking. Chris, it is typo in copy paste, sorry for the confusion.
STEP 1:
Here is the environment setup:
Hadoop-version: 2.7.2 Release
Here is the stack-trace while calling getFileStatus from test-code
programatically:
java.io.FileNotFoundException: No such file or directory:
{code}
s3a://<mask>:<mask>@dataplatform-prod-store/dev/empty
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at
com.olacabs.dp.utils.commonutils.io.S3ACustomFileSystem.main(S3ACustomFileSystem.java:29)
{code}
STEP 2: Checked out Trunk
I checked out the code and tried to work with the Trunk. However, I am hitting
into build another issue,:
{code}
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden
(Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID:
F8394B728D82563E), S3 Extended Request ID:
AXVEW+4p4f2XaByJyPPhSsSlnRVvmsvCjEgKju4NJ7Y4yenIus3IQreW8RwEjqK7tnPt6lHeK70=
at
com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
at
com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:770)
at
com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1050)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1027)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:902)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1414)
{code}
STEP 3: Checkout 2.7.2 and tried to patch with the "else fix"
{code}
} else if (key.isEmpty()) {
LOG.debug("Found root directory");
return new S3AFileStatus(true, true, path);
}
{code}
But looks like since the key is not empty, it is not helping here. However when
I changed to code to without this line //key = maybeAddTrailingSlash(key);
getCommonPrefixes() is non-empty.
was (Author: shaik.idris):
Hi [[email protected]], [~cnauroth],
Thanks for checking. Chris, it is typo in copy paste, sorry for the confusion.
STEP 1:
Here is the environment setup:
Hadoop-version: 2.7.2 Release
Here is the stack-trace while calling getFileStatus:
java.io.FileNotFoundException: No such file or directory:
{code}
s3a://<mask>:<mask>@dataplatform-prod-store/dev/empty
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at
com.olacabs.dp.utils.commonutils.io.S3ACustomFileSystem.main(S3ACustomFileSystem.java:29)
{code}
STEP 2: Checked out Trunk
I checked out the code and tried to work with the Trunk. However, I am hitting
into build another issue,:
{code}
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden
(Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID:
F8394B728D82563E), S3 Extended Request ID:
AXVEW+4p4f2XaByJyPPhSsSlnRVvmsvCjEgKju4NJ7Y4yenIus3IQreW8RwEjqK7tnPt6lHeK70=
at
com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
at
com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:770)
at
com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1050)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1027)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:902)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1414)
{code}
STEP 3: Checkout 2.7.2 and tried to patch with the "else fix"
{code}
} else if (key.isEmpty()) {
LOG.debug("Found root directory");
return new S3AFileStatus(true, true, path);
}
{code}
But looks like since the key is not empty, it is not helping here. However when
I changed to code to without this line //key = maybeAddTrailingSlash(key);
getCommonPrefixes() is non-empty.
> Listing an empty s3a NON root directory throws FileNotFound.
> ------------------------------------------------------------
>
> Key: HADOOP-13516
> URL: https://issues.apache.org/jira/browse/HADOOP-13516
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 2.7.0
> Reporter: Shaik Idris Ali
> Assignee: Steve Loughran
> Priority: Minor
> Fix For: 2.8.0
>
>
> With an empty s3 bucket and run
> {code}
> $ hadoop fs -D... -ls s3a://hdfs-s3a-test/emptyDirectory
> 15/05/04 15:21:34 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> ls: `s3a://hdfs-s3a-test/emtpyDirectory': No such file or directory
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
