github-actions[bot] commented on code in PR #69290:
URL: https://github.com/apache/doris/pull/69290#discussion_r4608910184
##########
cloud/src/recycler/s3_accessor.cpp:
##########
@@ -314,8 +315,23 @@ std::optional<S3Conf> S3Conf::from_obj_store_info(const
ObjectStoreInfoPB& obj_i
return s3_conf;
}
+namespace {
+
+// Endpoint, bucket and prefix as configured, plus the credential identity:
gateways such as Ceph
+// RGW resolve one bucket name per tenant. The ak is hidden as in logs, the sk
is never part of it.
+// '\n' never occurs in an endpoint or a bucket name, so different confs never
collide.
+std::string s3_location(const S3Conf& conf) {
+ return fmt::format("{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}", conf.endpoint,
conf.bucket, conf.prefix,
+ static_cast<int>(conf.provider),
hide_access_key(conf.ak), conf.role_arn,
+ conf.external_id,
static_cast<int>(conf.cred_provider_type));
+}
+
+} // namespace
+
S3Accessor::S3Accessor(S3Conf conf)
- : StorageVaultAccessor(AccessorType::S3), conf_(std::move(conf)) {}
+ : StorageVaultAccessor(AccessorType::S3, s3_location(conf),
+ storage_root(conf.bucket, conf.prefix)),
Review Comment:
[P2] Treat S3 bucket aliases conservatively in storage roots
A configured bucket string is not always a distinct namespace: an [AWS
access-point
alias](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-naming.html)
can replace its underlying bucket name, and [ListObjectsV2 supports
both](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-service-api-support.html).
Configure vault A with bucket `doris-bucket` and vault B with an access-point
alias for that bucket, both using prefix `warehouse`, HTTPS, virtual-hosted
addressing and credentials authorized on both. Both list the same
`data/T/R_0.dat`, whose committed rowset has `resource_id=A`. Their locations
differ, so both are scanned, but this literal `conf.bucket` makes their roots
unequal; `stored_under()` then rejects the live segment in B even after
rereading metadata. Every such segment increments `num_inverted_check_failed`
and the check returns 1 on healthy storage. This follows from the documented
alias semantics and inspected listing/ownership cod
e; no AWS runtime test was run. Resolve a canonical bucket identity or retain
the conservative ownership fallback for access-point aliases, and cover a
bucket/alias pair with zero leaks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]