[
https://issues.apache.org/jira/browse/HDDS-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190989#comment-17190989
]
Marton Elek commented on HDDS-4155:
-----------------------------------
Thanks the answer [~arp].
Sorry, If I used the wrong words in my question ("support") let's talk about
behavior instead of how to name it (bug or not).
Let's say I have an S3 bucket:
{code}
aws s3api list-objects --bucket ozonetest --prefix=a/b | jq '.Contents[] |
.Key'
"a/b/../c"
"a/b/../e"
"a/b/./c"
"a/b/./f"
"a/b//c"
"a/b//y"
"a/b/c"
"a/b/c/file1"
"a/b/h"
"a/b/h/"
"a/b/i/"
"a/b/x/README.md"
{code}
Using s3a we can see that some (invalid) path are hidden, others (for example
the directory and file with the same name):
{code}
drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/a
-rw-rw-rw- 1 elek elek 3841 2020-08-28 11:18 s3a://ozonetest/a/b/c
drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/c
-rw-rw-rw- 1 elek elek 3841 2020-09-02 12:34 s3a://ozonetest/a/b/h
drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/h
drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/i
drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/x
{code}
I don't think it's an "unfixable" bug, but it's a decision about the mapping:
to display both file and directories instead of hiding one (or throwing an
exception).
I understand that HDFS couldn't support it as it's not an object store. But
Ozone is an object store, and I asked (without the intention to suggest
anything different) **why** do we choose to follow the behavior of HDFS instead
of S3A which seems to be close to S3FS.
Second question: what is the behavior of some other object store connectors
(ADLS, google ....)?
In general (as I wrote in the other thread) I think we should consider some
level of compatibility between S3A (and other object store connectors) and
ofs/o3fs to support seamless move between on-prem and cloud in case of hybrid
cloud.
> Directory and filename can end up with same name in a path
> ----------------------------------------------------------
>
> Key: HDDS-4155
> URL: https://issues.apache.org/jira/browse/HDDS-4155
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: pull-request-available
>
> Scenario:
> Create Key via S3, and Create Directory through Fs.
> # open key -> /a/b/c
> # CreateDirectory -> /a/b/c
> # CommitKey -> /a/b/c
> So, now in Ozone we will have directory and file with name "c"
> When created through Fs interface.
> # create file -> /a/b/c
> # CreateDirectory -> /a/b/c
> # CommitKey -> /a/b/c
> So, now in Ozone we will have directory and file with name "c"
>
> # InitiateMPU /a/b/c
> # Create Part1 /a/b/c
> # Commit Part1 /a/b/c
> # Create Directory /a/b/c
> # Complete MPU /a/b/c
> So, now in Ozone, we will have directory and file with name "c". In MPU this
> is one example scenario.
>
> Few proposals/ideas to solve this:
> # Check during commit whether a directory already exists with same name. But
> disadvantage is after user uploads the entire data during last stage we fail.
> (File system with create in progress acts similarly. Scenario: 1. vi t1 2.
> mkdir t1 3. Save t1: (Fail:"t1" is a directory)
> # During create directory check are there any open key creation with same
> name and fail.
>
> Any of the above approaches are not final, this Jira is opened to discuss
> this issue and come up with solution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
