devmadhuu commented on PR #11229: URL: https://github.com/apache/ozone/pull/11229#issuecomment-6467065893
> Question on the resolver design: should there be an operator switch that lets the bucket's settings take precedence over the client's storage class? > > Without it, we get situations like #11679: since [HDDS-10979](https://issues.apache.org/jira/browse/HDDS-10979) a `STANDARD` header overrides an EC bucket, and a copy tool silently wrote RATIS/THREE into our EC cluster. With storage policies, the same precedence (request > bucket > default) would let a client's `GLACIER` move data to COLD in a bucket set to HOT. `S3StorageClassResolver` seems like the natural place for such a switch, so it might be worth deciding now whether it should be possible, even if it's implemented later. @greenwich Thanks for raising this. An optional gateway setting to give bucket settings priority makes sense. For example, a copy tool may send `STANDARD` when writing to an EC bucket. Today, that header selects RATIS/THREE and overrides the bucket’s EC setting. With the proposed option enabled, the write would keep the bucket’s EC configuration. The same control should cover storage policy when we connect the new resolver to the write paths. Otherwise, an archive-class request could select COLD even when the bucket is configured as HOT. I propose handling this in a follow-up JIRA covering regular uploads, copies, and multipart upload initialization. The option would be disabled by default to preserve existing behavior. One detail needs agreement: when a client explicitly requests GLACIER or DEEP_ARCHIVE for a HOT bucket, should we use the bucket’s settings or reject the conflicting request? I would prefer rejecting this conflict so that the client is told its archive request could not be honored. This PR can remain focused on storage-class definitions and mapping. The follow-up can introduce the operator setting and apply it consistently across the write paths. Follow up JIRA: [HDDS-17521](https://issues.apache.org/jira/browse/HDDS-17521) cc: @SaketaChalamchala -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
