zakariya-s commented on code in PR #2932:
URL: https://github.com/apache/iceberg-rust/pull/2932#discussion_r3735905025
##########
crates/storage/opendal/src/lib.rs:
##########
@@ -384,6 +415,27 @@ impl OpenDalStorage {
}
}
+ /// Returns whether `path` uses a path-scoped dynamic credential provider.
+ ///
+ /// Such paths cannot share an OpenDAL deleter unless the provider can
expose
+ /// the credential scope that applies to each path. Process them
independently
+ /// so bulk deletion remains bounded without crossing credential
boundaries.
+ fn uses_dynamic_credentials(&self, path: &str) -> bool {
+ match self {
+ #[cfg(feature = "opendal-s3")]
+ OpenDalStorage::S3 {
+ credential_provider: Some(provider),
+ ..
+ } => provider.supports_path(path),
+ #[cfg(feature = "opendal-gcs")]
+ OpenDalStorage::Gcs {
Review Comment:
Yeah I left this in the initial commit for sake of simplicity, we now batch
by `DeleteBatchKey` which includes the credential scope so batching behaviour
is now restored
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
