[
https://issues.apache.org/jira/browse/HDDS-17521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18133970#comment-18133970
]
Aleksei Ieshin commented on HDDS-17521:
---------------------------------------
Thanks [~deveshsingh]. This overlaps with HDDS-17303, which I filed on Oct 22
with the background: the regression from HDDS-10979 (2.0.0 gave an EC bucket
precedence over the header), the production impact we hit after upgrading to
2.2.1, and the same override at the OM level in
{{{}OzoneConfigUtil.resolveReplicationConfigPreference{}}}. Could we link the
two?
A few points where the two descriptions differ and I'd like to agree on the
behaviour:
# *Conflicts: make ignore vs. reject an operator choice.* Rather than deciding
for everyone, a second setting could choose what happens when the switch is on
and a valid storage class conflicts with the bucket:
** {{ignore}} (suggested default): accept the request and store the object
with the bucket's settings. Many copy tools send a storage class on every
request, so this keeps them working. Responses report the class derived from
the stored replication, so clients can see what they got.
** {{{}reject{}}}: fail the request with a clear error, for operators who want
clients to notice.
Unknown classes are rejected with {{InvalidStorageClass}} in both modes, as
today. The two settings could also be one, e.g.
{{ozone.s3g.storage-class.client-override = allow | ignore | reject}} with
{{allow}} (today's behaviour) as the default, which avoids a combination where
reject is set but enforcement is off.
# *Which buckets are covered.* Some EC buckets get their replication from the
cluster's server-side default, not from a value set on the bucket. If only
explicit bucket settings count, a {{STANDARD}} header still writes RATIS/THREE
into them. Could the switch use the bucket's effective replication (bucket,
then server default), or at least document clearly which buckets are covered?
# *All classes and {{{}x-amz-meta-storage-config{}}}.* With the switch on,
{{{}REDUCED_REDUNDANCY{}}}, {{STANDARD_IA}} and the
{{x-amz-meta-storage-config}} header should also stop choosing replication, not
only {{{}STANDARD{}}}.
# *Precedence over the gateway's {{{}ozone.replication{}}}.* Today a request
without a header uses the gateway's {{ozone.replication}} if it is set, ahead
of the bucket. With the switch on, should the bucket win there too? The same
applies to {{{}ozone.s3g.standard.storage-class.use-client-default{}}}.
Happy to review the patch and test it on an EC-only setup.
> Add an optional S3 Gateway setting to prefer bucket replication and storage
> policy
> ----------------------------------------------------------------------------------
>
> Key: HDDS-17521
> URL: https://issues.apache.org/jira/browse/HDDS-17521
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Client
> Reporter: Devesh Kumar Singh
> Assignee: Devesh Kumar Singh
> Priority: Major
>
> An explicit S3 storage-class header can override a bucket’s replication
> configuration. For example:
> 1. A bucket uses EC replication.
> 2. A copy tool uploads an object with x-amz-storage-class: STANDARD.
> 3. The gateway selects RATIS/THREE instead of the bucket’s EC configuration.
> This can produce objects with a different replication configuration than the
> operator intended.
> Storage-policy support introduces a related case: an archive-class request
> could select COLD for a bucket configured as HOT. This needs to be addressed
> when the new resolver is integrated into S3 write operations.
> Introduce an optional S3 Gateway setting that gives explicitly configured
> bucket replication and storage policy priority. Keep the option disabled by
> default to preserve existing behavior. When a bucket setting is absent,
> retain the existing resolution rules for that setting.
> *Decision needed*
> Agree on how to handle an archive-class request that conflicts with the
> bucket’s settings. The proposed approach is to reject the request with a
> clear error, rather than silently accept it with different storage behavior.
> *Acceptance criteria*
> - With the option disabled, existing request handling remains unchanged.
> - With it enabled, an EC bucket retains EC when a client sends STANDARD.
> - Conflicting archive requests follow the agreed behavior and have focused
> tests.
> - Regular uploads, copies, and multipart upload initialization apply the same
> rules. Multipart uploads retain the configuration selected at initialization.
> - Tests cover missing bucket settings, invalid storage-class headers, and
> linked buckets.
> - Changing the gateway setting does not modify existing objects. Responses
> describe stored object metadata.
> - Documentation explains the default, precedence rules, conflict handling,
> and restart requirements.
> cc: [~SaketaChalamchala] [~greenwich]
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
