Marcin Zukowski created HADOOP-13572:
----------------------------------------
Summary: fs.s3native.mkdirs does not work if the user is only
authorized to a subdirectory
Key: HADOOP-13572
URL: https://issues.apache.org/jira/browse/HADOOP-13572
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Reporter: Marcin Zukowski
Noticed that when working with Spark. I have an S3 bucket with top directories
having protected access, and a dedicated open directory deeper in the tree for
Spark temporary data.
Writing to this directory fails with the following stack
{noformat}
[info] org.apache.hadoop.fs.s3.S3Exception:
org.jets3t.service.S3ServiceException: S3 HEAD request failed for
'/SPARK-SNOWFLAKEDB' - ResponseCode=403, ResponseMessage=Forbidden
[info] at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:245)
[info] at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:119)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] at java.lang.reflect.Method.invoke(Method.java:497)
[info] at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
[info] at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
[info] at org.apache.hadoop.fs.s3native.$Proxy34.retrieveMetadata(Unknown
Source)
[info] at
org.apache.hadoop.fs.s3native.NativeS3FileSystem.getFileStatus(NativeS3FileSystem.java:414)
[info] at
org.apache.hadoop.fs.s3native.NativeS3FileSystem.mkdir(NativeS3FileSystem.java:539)
[info] at
org.apache.hadoop.fs.s3native.NativeS3FileSystem.mkdirs(NativeS3FileSystem.java:532)
[info] at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1933)
[info] at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:291)
[info] at
org.apache.hadoop.mapred.FileOutputCommitter.setupJob(FileOutputCommitter.java:131)
{noformat}
I believe this is because mkdirs in NativeS3FileSystem.java tries to create
directories starting "from the root", and so if the process can't "list"
objects on a given level, it fails. Perhaps it should accept this kind of
failures, or go "from the leaf" first to find the level from which it needs to
start creating directories. That might also be good for performance assuming
the directories exist most of the time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
