[
https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192296#comment-17192296
]
Bharat Viswanadham edited comment on HDDS-4209 at 9/8/20, 4:08 PM:
-------------------------------------------------------------------
[~elek]
I am not sure if this is the corner case.
Because the use case here is straight forward
/d11/d12 -> create a directory, but for ozone, it is considered as a file. (As
create directory is put Request for a file with 0 bytes from S3A, it does not
call createDirectory)
"/d11/d12/file1 -> So, when trying to create file1 it considers "/d11/d12" as
file instead of directory that is causing this error.
{quote}I think the intermediate directory creation should be more
permissive.{quote}
What do you mean by this?
was (Author: bharatviswa):
[~elek]
I am not sure if this is the corner case.
Because the use case here is straight forward
/d11/d12 -> create a directory, but for ozone, it is considered as a file. (As
create directory is put Request for a file with 0 bytes)
"/d11/d12/file1 -> So, when trying to create file1 it considers "/d11/d12" as
file instead of directory that is causing this error.
{quote}I think the intermediate directory creation should be more
permissive.{quote}
What do you mean by this?
> S3A Filesystem does not work with Ozone S3
> ------------------------------------------
>
> Key: HDDS-4209
> URL: https://issues.apache.org/jira/browse/HDDS-4209
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Blocker
>
> When *ozone.om.enable.filesystem.paths* is enabled
>
> hdfs dfs -mkdir -p s3a://b12345/d11/d12 -> Success
> hdfs dfs -put /tmp/file1 s3a://b12345/d11/d12/file1 -> fails with below error
>
> {code:java}
> 2020-09-04 03:53:51,377 ERROR
> org.apache.hadoop.ozone.om.request.key.OMKeyCreateRequest: Key creation
> failed. Volume:s3v, Bucket:b1234, Keyd11/d12/file1._COPYING_. Exception:{}
> NOT_A_FILE org.apache.hadoop.ozone.om.exceptions.OMException: Can not create
> file: cp/k1._COPYING_ as there is already file in the given path
> at
> org.apache.hadoop.ozone.om.request.key.OMKeyCreateRequest.validateAndUpdateCache(OMKeyCreateRequest.java:256)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleWriteRequest(OzoneManagerRequestHandler.java:227)
> at
> org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.runCommand(OzoneManagerStateMachine.java:428)
> at
> org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.lambda$applyTransaction$1(OzoneManagerStateMachine.java:246)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
> *Reason for this*
> S3A filesystem when create directory creates an empty file
> *Now entries in Ozone KeyTable after create directory*
> d11/
> d11/d12
> Because of this in OMFileRequest.VerifyInFilesPath fails with
> FILE_EXISTS_IN_GIVEN_PATH because d11/d12 is considered as file not a
> directory. (As in ozone currently, directories end with trailing "/")
> So, when d11/d12/file is created, we check parent exists, now d11/d12 is
> considered as file and fails with NOT_A_FILE
> When disabled it works fine, as when disabled during key create we do not
> check any filesystem semantics and also does not create intermediate
> directories.
> {code:java}
> [root@bvoz-1 ~]# hdfs dfs -mkdir -p s3a://b12345/d11/d12
> [root@bvoz-1 ~]# hdfs dfs -put /etc/hadoop/conf/ozone-site.xml
> s3a://b12345/d11/d12/k1
> [root@bvoz-1 ~]# hdfs dfs -ls s3a://b12345/d11/d12
> Found 1 items
> -rw-rw-rw- 1 systest systest 2373 2020-09-04 04:45
> s3a://b12345/d11/d12/k1
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
