jimmystewpot opened a new pull request, #3714:
URL: https://github.com/apache/iceberg-rust/pull/3714

   ## Which issue does this PR close?
   
   - Part of #2394 (PR 1 of 3 split from #2396 per [review 
feedback](https://github.com/apache/iceberg-rust/pull/2396#pullrequestreview-4099865518)).
   
   ## What changes are included in this PR?
   
   AWS-backed catalogs (`iceberg-catalog-glue` and `iceberg-catalog-s3tables`) 
share common AWS SDK configuration, STS `AssumeRole` authentication, and S3 
`FileIO` credential-bridging behaviour.
   
   This PR introduces the `iceberg-aws` integration crate 
(`crates/integrations/aws`), organised into three focused modules:
   - `properties`: `AwsProperties` (`properties_view!`, modelled on 
`TableProperties`) providing typed accessors, S3 property mapping, and 
`prepare_s3_file_io` to bridge assumed-role or profile-resolved SDK credentials 
into S3 `FileIO` without triggering duplicate STS calls in OpenDAL.
   - `credentials`: `RefreshingCredentialsProvider` (caching STS credentials 
behind a `tokio::sync::RwLock` with single-flight refresh, fallback to 
unexpired cached credentials on transient STS errors, and a 30s 
proactive-refresh backoff) and `AwsSdkCredentialProvider` (adapting AWS SDK 
credential providers to `reqsign_core::ProvideCredential`).
   - `config`: AWS `SdkConfig` construction that isolates STS `AssumeRole` 
requests from catalog-specific endpoint overrides.
   
   > **Note for committers:** Because `iceberg-aws` sets `publish = true`, CI's 
`Check publishable crates exist on crates.io` step requires `iceberg-aws` to be 
reserved on `crates.io` (`0.0.0` placeholder) before merging; see *Adding a new 
crate* in `website/src/release.md`.
   
   ## Are these changes tested?
   
   Yes — 22 unit and `mockito`-backed STS tests in `iceberg-aws` covering 
property precedence, secret redaction in `Debug`, STS endpoint isolation, 
concurrent single-flight refresh, transient refresh fallback and 30s backoff, 
and S3 `FileIO` credential bridging for both `AssumeRole` and named profiles.
   
   ## AI Disclosure
   
   - **AI use:** AI assistance (Claude, Codex, and Gemini) was used to split 
this crate out of #2396, organise it around `AwsProperties` 
(`properties_view!`), and add unit tests.
   - **Verification:** Reviewed the changes and ran `cargo test -p 
iceberg-aws`, `make check-fmt`, `make check-clippy`, `make check-doc`, `make 
check-public-api`, `make check-license-notice`, `make 
check-dependency-licenses`, `taplo fmt --check`, and `cargo machete` locally.
   - **Areas for reviewers to focus on:**
     - `AwsProperties` API surface (`public-api.txt`) and whether any 
methods/constants should be adjusted before `glue` (PR 2) and `s3tables` (PR 3) 
consume it.
     - Concurrency and backoff behaviour in `RefreshingCredentialsProvider`: 
the 30-second `retry_not_before` cooldown intentionally applies only during the 
5-minute proactive refresh window while unexpired cached credentials remain 
usable, so a transient startup blip does not negative-cache failures for 30 
seconds.
     - Credential bridging in `AwsProperties::prepare_s3_file_io`: shares 
`AwsSdkCredentialProvider` when `client.assume-role.arn` or `profile_name` is 
configured and no explicit `s3.*` static credentials are set.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to