Hi All, >From my POV the use case is clear and PR [5884] is ready to merge.
If there are no further concerns, I propose merging it on Nov 6. Thanks, Dmitri. [5884] https://github.com/apache/polaris/pull/5884 On Mon, Nov 2, 2026 at 7:10 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Yufei, > > > Could you share more details about the workload identity use case? > > Clients (Engines) may have ambient credentials in the execution env., > which allows them to access storage (e.g. GCS). > > In this case, Polaris does not have to vend credentials. However, with the > current GCS storage integration, Polaris cannot avoid vending credentials. > > If the admin user elects to use workload identity for clients (ambient > credentials), Polaris credential vending work is useless overhead. Moreover > it may present a security gap by exposing different credentials to clients, > which are supposed to authenticate via another (workload identity) > channel. I believe it is important for the admin user to be able to > minimize credential exposure via Polaris in this case. > > > I think we need a concrete use case to decide whether we need this [...] > > From my POV the use case I described above is self-sufficient. > > A similar practical use case relates to your reasonable and timely > proposal to limit the use of SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION [2]. > > If we are to take the next step and adapt our Integration Test to act as > examples of how to avoid relying on this flag, this results in certain > coding difficulties, which are evident [1] in [5884]. > > > [...] where the setting should go. > > The natural place for a new setting that controls access to derived > credentials seems to be in the type-specific storage config [5884], as I > proposed in my original email in this thread. > > Do you have an alternative suggestion for how to address this use case? > > Thanks, > Dmitri. > > [1] https://github.com/apache/polaris/pull/5756#discussion_r4362688654 > > [2] https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg > > [5756] https://github.com/apache/polaris/pull/5756 > > [5884] https://github.com/apache/polaris/pull/5884 > > On Mon, Nov 2, 2026 at 3:31 PM Yufei Gu <[email protected]> wrote: > >> Hi Dmitri, >> >> Thanks for clarifying. My earlier comment was about clients choosing >> not to request credential vending through X-Iceberg-Access-Delegation. >> I understand that this is a server-side admin decision, which differs >> from clients opting out. >> >> Could you share more details about the workload identity use case? I’m >> still not clear why admins would need to disable credential vending in >> that scenario. >> >> stsUnavailable is for S3-compatible storage without STS support. If >> there’s a similar limitation for Azure/GCS, these settings make sense >> to me. But if this is an admin policy, I’m not sure storage >> configuration is the right place, given that a storage config may >> apply to multiple tables or namespaces in the future (Rishi's >> proposal). >> >> I think we need a concrete use case to decide whether we need this and >> where the setting should go. WDYT? >> >> >> Yufei >> >> On Fri, Oct 30, 2026 at 3:17 PM Dmitri Bourlatchkov via dev >> <[email protected]> wrote: >> > >> > Hi Yufei, >> > >> > > [...] we could use the header to achieve the same result. >> > >> > The proposed change allows admin users to configure Polaris to avoid >> > vending credentials in Azure and GCS environments. It is a deliberate >> > server-side admin decision. >> > >> > This was possible for S3 before. PR [5884] aligns Polaris capabilities >> in >> > Azure and GCS with existing Polaris capabilities in S3 environments. >> > >> > What alternative do you suggest? I'm not sure I can see that in JB's or >> > Ayush's messages. >> > >> > Thanks, >> > Dmitri. >> > >> > [5884] https://github.com/apache/polaris/pull/5884 >> > >> > On Thu, Oct 29, 2026 at 7:30 PM Yufei Gu <[email protected]> wrote: >> > >> > > Apologies for the late reply. >> > > >> > > Thanks Dmitri for raising this, and Vignesh for working onit. As >> Ayush and >> > > JB mentioned, we could use the header to achieve the same result. Do >> we >> > > want to introduce another spec change, or am I missing some use cases? >> > > >> > > Yufei >> > > >> > > >> > > On Thu, Oct 29, 2026 at 2:05 PM Dmitri Bourlatchkov <[email protected] >> > >> > > wrote: >> > > >> > > > Hi All, >> > > > >> > > > PR [5884] seems to be in a good shape. As far as I can tell, all >> existing >> > > > points have been addressed. >> > > > >> > > > If there are no objections, I propose merging it on Oct 30. >> > > > >> > > > Thanks, >> > > > Dmitri. >> > > > >> > > > [5884] https://github.com/apache/polaris/pull/5884 >> > > > >> > > > On Tue, Oct 27, 2026 at 5:46 PM Dmitri Bourlatchkov < >> > > > [email protected]> wrote: >> > > > >> > > > > Hi JB, Ayush, >> > > > > >> > > > > Good points re: DefaultAccessDelegationModeResolver! >> > > > > >> > > > > Vignesh contributed PR [5884] to implement the new properties. >> This PR >> > > > > adjusts the logic in DefaultAccessDelegationModeResolver too. >> Kudos to >> > > > > Vugnesh! >> > > > > >> > > > > As far as I can tell, after [5884] if a client requests >> > > > > "vended-credentials" but the server is configured >> > > > > with skipCredentialDownscoping = true (for example), then the >> client >> > > will >> > > > > receive a clear error message indicating that credential vending >> is not >> > > > > available. >> > > > > >> > > > > This is related to the Iceberg API clarification vote [1]. If >> that spec >> > > > > change passes in Iceberg, Polaris will be able to support "none" >> as a >> > > > > vending mode for a better client config experience, IMHO. >> > > > > >> > > > > In any case, please review [5884]. I think it still makes sense in >> > > > Polaris >> > > > > even without the Iceberg spec change. >> > > > > >> > > > > In Polaris code the main use case may be in tests, but some users >> may >> > > > want >> > > > > to consider the new options for workload identity-based >> deployments >> > > too. >> > > > > >> > > > > [1] >> https://lists.apache.org/thread/hc2764vozz7ygx5fjj64104hotrof0lk >> > > > > >> > > > > [5884] https://github.com/apache/polaris/pull/5884 >> > > > > >> > > > > Cheers, >> > > > > Dmitri. >> > > > > >> > > > > On Sun, Oct 25, 2026 at 5:53 AM Jean-Baptiste Onofré < >> [email protected]> >> > > > > wrote: >> > > > > >> > > > >> Hi all, >> > > > >> >> > > > >> +1 on the catalog-level >> AzureStorageConfigInfo.skipSasTokenGeneration >> > > > and >> > > > >> GcpStorageConfigInfo.skipCredentialDownscoping properties. A >> typed, >> > > well >> > > > >> defined setting is a better home for "don't vend" than the >> > > > >> SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag, which only applies >> at the >> > > > >> realm level. It also matches what stsUnavailable already does >> for S3. >> > > > >> >> > > > >> I agree with Ayush on the access delegation mode. >> > > > >> >> > > > >> I think it would be great to clarify two points before working >> on the >> > > > >> implementation PR: >> > > > >> - If a catalog opts out of vending and the client asks only for >> > > > >> vend-credentials, we should reject the request up front with a >> clear >> > > > >> error. >> > > > >> Failing later inside the vending path is harder to diagnose >> imho. This >> > > > >> should apply to all three providers, so the S3 stsUnavailable >> case >> > > > behaves >> > > > >> the same way. If changing the S3 behavior is out of the scope >> for the >> > > > PR, >> > > > >> we can track as a follow-up, but we need consistency here imho. >> > > > >> - Both properties should default to false, so existing catalogs >> keep >> > > > >> vending credentials. They should be optional in the Management >> API, so >> > > > >> this >> > > > >> is a backward compatible addition. We should also document them >> next >> > > to >> > > > >> stsUnavailable (in site/content/in-dev/unreleased/), and say >> there >> > > that >> > > > >> they are meant for non-cloud or emulator deployments. >> > > > >> >> > > > >> Happy to review the PRs ;) >> > > > >> >> > > > >> Regards >> > > > >> JB >> > > > >> >> > > > >> On Fri, Oct 23, 2026 at 8:15 PM Dmitri Bourlatchkov < >> [email protected] >> > > > >> > > > >> wrote: >> > > > >> >> > > > >> > Hi All, >> > > > >> > >> > > > >> > As discovered during the review of PR 5756 [1], now >> > > > >> > that SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION is not recommended >> for >> > > > >> > mainstream use [3], deployments (e.g. tests) using non-cloud >> options >> > > > for >> > > > >> > ADLS and GS (GCS) storage need a mechanism for opting out of >> minting >> > > > >> > session tokens and resolving to local (server and/or client) >> > > > credentials >> > > > >> > for storage access. >> > > > >> > >> > > > >> > I propose adding the following properties at the Management API >> > > level >> > > > >> [1]. >> > > > >> > >> > > > >> > * AzureStorageConfigInfo.skipSasTokenGeneration >> > > > >> > >> > > > >> > * GcpStorageConfigInfo.skipCredentialDownscoping >> > > > >> > >> > > > >> > Both names relate to the corresponding native APIs in each >> > > technology. >> > > > >> > >> > > > >> > Setting these properties will effectively disable credential >> > > vending. >> > > > >> This >> > > > >> > is similar to stsUnavailable in AwsStorageConfigInfo. >> > > > >> > >> > > > >> > The main purpose of the new properties is establishing a >> > > catalog-level >> > > > >> opt >> > > > >> > in/out mechanism and providing well-defined user-level >> settings. >> > > > >> > >> > > > >> > PolarisStorageIntegration implementations still retain value >> without >> > > > >> > credential vending because they potentially define >> non-credential >> > > > >> > properties in StorageAccessConfig. The latter case currently >> applies >> > > > >> only >> > > > >> > to S3, but downstream customizations may utilize it too. >> > > > >> > >> > > > >> > [1] >> > > > https://github.com/apache/polaris/pull/5756#discussion_r4362688654 >> > > > >> > >> > > > >> > [2] >> > > > >> > >> > > > >> > >> > > > >> >> > > > >> > > >> https://github.com/apache/polaris/blob/5fafc67a832a8900fefb399abc00549d864470d6/spec/polaris-management-service.yml#L21 >> > > > >> > >> > > > >> > [3] >> > > https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg >> > > > >> > >> > > > >> > Thoughts? >> > > > >> > >> > > > >> > Thanks, >> > > > >> > Dmitri. >> > > > >> > >> > > > >> >> > > > > >> > > > > >> > > > > -- >> > > > > Dmitri Bourlatchkov >> > > > > Senior Staff Software Engineer, Dremio >> > > > > Dremio.com >> > > > > < >> > > > >> > > >> https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature >> > > > >> > > > / >> > > > > Follow Us on LinkedIn <https://www.linkedin.com/company/dremio> >> / Get >> > > > > Started <https://www.dremio.com/get-started/> >> > > > > >> > > > > >> > > > > The Agentic Lakehouse >> > > > > The only lakehouse built for agents, managed by agents >> > > > > >> > > > > >> > > > > >> > > > >> > > >> >
