[
https://issues.apache.org/jira/browse/HDDS-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655171#comment-17655171
]
Sammi Chen edited comment on HDDS-7585 at 1/6/23 12:39 AM:
-----------------------------------------------------------
# If I create a bucket named {{x_y-z}} and then list buckets, how does Ozone
know which {{-}} to convert back to {{{}_{}}}? Or is it OK to get back
different bucket name?
# If "posix.compliant" is enabled, will Ozone reject creation of {{x-y-z}}
with "already exists" if there is a bucket named {{{}x_y-z{}}}?
Given the property is enabled, If user requests to create a bucket
x_y-z, the bucket with name x-y-z will be created in Ozone. If user requests to
create another bucket with name x-y-z, or x_y-z, or x-y_z, ozone will reject
with "already exists". Ozone doesn't know which - to convert back to _, so we
need to put the original bucket name somewhere so it will be used later.
In this case, probably we can leverage the bucket link function. If user
creates a bucket with name x_y-z, we create two buckets, one is x-y-z, another
is x_y-z which is a bucket link, linking to x-y-z. Then we will have two path,
(a) is /vol/x-y-z which follows s3 naming rule, (b) is /vol/x_y-z which follows
posix naming rule. If upper layer application need s3 access, it uses path
(a), if need posix access, it uses path(b).
was (Author: sammi):
# If I create a bucket named {{x_y-z}} and then list buckets, how does Ozone
know which {{-}} to convert back to {{{}_{}}}? Or is it OK to get back
different bucket name?
# If "posix.compliant" is enabled, will Ozone reject creation of {{x-y-z}}
with "already exists" if there is a bucket named {{{}x_y-z{}}}?
Given the property is enabled, If user requests to create a bucket
x_y-z, the bucket with name x-y-z will be created in Ozone. If user requests to
create another bucket with name x-y-z, or x_y-z, or x-y_z, ozone will reject
with "already exists". Ozone doesn't know which - to convert back to _, so we
need to put the original bucket name somewhere so it will be used later.
In this case, probably we can leverage the bucket link function. If user
creates a bucket with name x_y-z, we create two buckets, one is x-y-z, another
is x_y-z which is a bucket link, linking to x-y-z. Then we will have two path,
(a) is /vol/x-y-z which follows s3 naming rule, (b) is /vol/x_y-z which follows
posix naming rule. If upper layer application need s3 access, it uses path
(a), if need posix access, it uses path(b). Just a quick thinking, not sure if
bucket link will bring in other issues.
> Allow Ozone bucket to use non-S3-compliance naming rule
> -------------------------------------------------------
>
> Key: HDDS-7585
> URL: https://issues.apache.org/jira/browse/HDDS-7585
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone CLI, Ozone Filesystem, Ozone Manager
> Reporter: Dave Teng
> Priority: Major
>
> We notice many HDFS migration cases which translate the second level
> directory names to Ozone bucket names.
> However, some characters in the name of directory are +compliant with POSIX+
> convention, but are +not compliant with S3+ naming rule, such as
> {*}underscore{*}.
> Thus the original HDFS path which contains *underscore* _character are not
> allowed to put into Ozone because by default Ozone's path validation is same
> as S3. For such users, this proposal wish to provide a way that when the user
> gives a flag while creating the bucket via Ozone CLI (not from S3 interface),
> Ozone will allow *underscore*_ only for such buckets. The idea is it'd be
> user's choice whether to create a non-s3 compliant bucket path or not. Ozone
> will keep equally supporting both OFS and S3 interfaces. For pure file system
> back ground users, they may want to go ahead with this flag by having the
> awareness of that they may not be able to access the bucket through S3
> interface.
>
> This Jira is an umbrella ticket that makes Ozone compatible with these two
> types of naming convention, both POSIX and S3, to enable the HDFS to Ozone
> migration with non s3 compliant paths.
>
> (ps:
> - This proposal wouldn't change the default behavior of S3 bucket naming
> semantic in Ozone.
> - The documentation for new flag will be provided! )
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
