Hi Dmitri, > SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag represents legacy use cases that > are not well-defined in the current Polaris codebase.
Thanks for confirming, that was my impression as well. I tried to set this flag on, but Polaris started failing in many surprising ways. > with subscoping "on", but STS disabled [...] the static S3 credentials will > actually be used for FileIO instances inside Polaris servers. That's not the behavior I'm observing. Common properties like region, endpoint and path-style access do get included in the AccessConfig, but no credential is present [1]. Thanks, Alex [1]: https://github.com/apache/polaris/blob/f33646b564c813979a8a7f0d96d09dd12fd569f5/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L124 On Tue, Nov 4, 2025 at 7:38 PM Dmitri Bourlatchkov <[email protected]> wrote: > > Hi Alex, > > Thanks for starting this discussion! > > From my POV, the SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag represents > legacy use cases that are not well-defined in the current Polaris > codebase. The flag can certainly be used if you know the code well, but > from the end user's perspective, I believe the related behaviour changes > are very obscure. I'd propose to avoid using this flag for main use cases, > but keep it for backward compatibility for a few releases, aiming to > eventually remove it. > > That said, even with subscoping "on", but STS disabled, the behaviour is > still quite obscure. However, IIRC the static S3 credentials will actually > be used for FileIO instances inside Polaris servers. > > I think it's a good idea to use StorageAccessConfig on all call paths, both > internal and when vending credentials to clients. > > Having distinct methods like getClientAccessConfig() > and getServerAccessConfig() in AccessConfigProvider sounds like a > reasonable approach to implementing this. > > Cheers, > Dmitri. > > On Tue, Nov 4, 2025 at 8:11 AM Alexandre Dutra <[email protected]> wrote: > > > Hi all, > > > > I'm having trouble understanding how AWS system-wide credentials work > > when subscoping or STS is unavailable. My apologies if this has been > > asked previously, but I would greatly appreciate it if someone could > > provide an explanation. > > > > Currently, Polaris honors AWS system-wide credentials (set via > > polaris.storage.aws.access-key and polaris.storage.aws.secret-key) > > only when credentials subscoping is enabled and STS is available. > > > > In this scenario, Polaris generates subscoped credentials using these > > static credentials for every STS request [1]. > > > > However, if either credentials subscoping or STS is unavailable, these > > system-wide credentials are ignored, afaict. > > > > The core problem, in my opinion, is that the current code mixes > > credentials subscoping (for the server itself) and credentials vending > > (for clients). While it's appropriate not to vend these long-lived > > credentials directly to clients, I believe they should be used when > > the server itself needs to access the remote storage. > > > > This leads to issues in two specific S3 setups: > > > > 1) System-wide credentials with credentials subscoping OFF (STS status > > is irrelevant). > > > > 2) System-wide credentials with credentials subscoping ON, but STS > > marked unavailable. > > > > In both cases, the generated AccessConfig doesn't contain credentials, > > which means that the FileIO instance that Polaris creates will rely on > > the default provider chain for credentials. > > > > I wonder if the AccessConfig should instead contain the keys > > s3.access-key-id and s3.secret-access-key, populated with the static, > > system-wide credentials? > > > > If I were a user using an S3-like service without STS, I would have > > expected that those static credentials would be used for FileIO > > instance creation (and also, one day, for S3 remote signing). > > > > To address this, I was thinking of adding a dedicated method to > > AccessConfigProvider. Something like getClientAccessConfig() vs > > getServerAccessConfig(). The latter would return a specialized > > AccessConfig for server-side usage, ensuring that if static > > credentials are present, these are used whenever subscoping is off or > > STS is unavailable. > > > > What do you all think? > > > > Thanks, > > Alex > > > > [1]: > > https://github.com/apache/polaris/blob/f97c5eb50016489129575aab62d5efb3efb7552e/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L99-L100 > >
