Hi All,

>From my POV the use case is clear and PR [5884] is ready to merge.

If there are no further concerns, I propose merging it on Nov 6.

Thanks,
Dmitri.

[5884] https://github.com/apache/polaris/pull/5884

On Mon, Nov 2, 2026 at 7:10 PM Dmitri Bourlatchkov <[email protected]> wrote:

> Hi Yufei,
>
> > Could you share more details about the workload identity use case?
>
> Clients (Engines) may have ambient credentials in the execution env.,
> which allows them to access storage (e.g. GCS).
>
> In this case, Polaris does not have to vend credentials. However, with the
> current GCS storage integration, Polaris cannot avoid vending credentials.
>
> If the admin user elects to use workload identity for clients (ambient
> credentials), Polaris credential vending work is useless overhead. Moreover
> it may present a security gap by exposing different credentials to clients,
> which are supposed to authenticate via another (workload identity)
> channel. I believe it is important for the admin user to be able to
> minimize credential exposure via Polaris in this case.
>
> > I think we need a concrete use case to decide whether we need this [...]
>
> From my POV the use case I described above is self-sufficient.
>
> A similar practical use case relates to your reasonable and timely
> proposal to limit the use of SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION [2].
>
> If we are to take the next step and adapt our Integration Test to act as
> examples of how to avoid relying on this flag, this results in certain
> coding difficulties, which are evident [1] in [5884].
>
> > [...] where the setting should go.
>
> The natural place for a new setting that controls access to derived
> credentials seems to be in the type-specific storage config [5884], as I
> proposed in my original email in this thread.
>
> Do you have an alternative suggestion for how to address this use case?
>
> Thanks,
> Dmitri.
>
> [1] https://github.com/apache/polaris/pull/5756#discussion_r4362688654
>
> [2] https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg
>
> [5756] https://github.com/apache/polaris/pull/5756
>
> [5884] https://github.com/apache/polaris/pull/5884
>
> On Mon, Nov 2, 2026 at 3:31 PM Yufei Gu <[email protected]> wrote:
>
>> Hi Dmitri,
>>
>> Thanks for clarifying. My earlier comment was about clients choosing
>> not to request credential vending through X-Iceberg-Access-Delegation.
>> I understand that this is a server-side admin decision, which differs
>> from clients opting out.
>>
>> Could you share more details about the workload identity use case? I’m
>> still not clear why admins would need to disable credential vending in
>> that scenario.
>>
>> stsUnavailable is for S3-compatible storage without STS support. If
>> there’s a similar limitation for Azure/GCS, these settings make sense
>> to me. But if this is an admin policy, I’m not sure storage
>> configuration is the right place, given that a storage config may
>> apply to multiple tables or namespaces in the future (Rishi's
>> proposal).
>>
>> I think we need a concrete use case to decide whether we need this and
>> where the setting should go. WDYT?
>>
>>
>> Yufei
>>
>> On Fri, Oct 30, 2026 at 3:17 PM Dmitri Bourlatchkov via dev
>> <[email protected]> wrote:
>> >
>> > Hi Yufei,
>> >
>> > > [...] we could use the header to achieve the same result.
>> >
>> > The proposed change allows admin users to configure Polaris to avoid
>> > vending credentials in Azure and GCS environments. It is a deliberate
>> > server-side admin decision.
>> >
>> > This was possible for S3 before. PR [5884] aligns Polaris capabilities
>> in
>> > Azure and GCS with existing Polaris capabilities in S3 environments.
>> >
>> > What alternative do you suggest? I'm not sure I can see that in JB's or
>> > Ayush's messages.
>> >
>> > Thanks,
>> > Dmitri.
>> >
>> > [5884] https://github.com/apache/polaris/pull/5884
>> >
>> > On Thu, Oct 29, 2026 at 7:30 PM Yufei Gu <[email protected]> wrote:
>> >
>> > > Apologies for the late reply.
>> > >
>> > > Thanks Dmitri for raising this, and Vignesh for working onit. As
>> Ayush and
>> > > JB mentioned, we could use the header to achieve the same result. Do
>> we
>> > > want to introduce another spec change, or am I missing some use cases?
>> > >
>> > > Yufei
>> > >
>> > >
>> > > On Thu, Oct 29, 2026 at 2:05 PM Dmitri Bourlatchkov <[email protected]
>> >
>> > > wrote:
>> > >
>> > > > Hi All,
>> > > >
>> > > > PR [5884] seems to be in a good shape. As far as I can tell, all
>> existing
>> > > > points have been addressed.
>> > > >
>> > > > If there are no objections, I propose merging it on Oct 30.
>> > > >
>> > > > Thanks,
>> > > > Dmitri.
>> > > >
>> > > > [5884] https://github.com/apache/polaris/pull/5884
>> > > >
>> > > > On Tue, Oct 27, 2026 at 5:46 PM Dmitri Bourlatchkov <
>> > > > [email protected]> wrote:
>> > > >
>> > > > > Hi JB, Ayush,
>> > > > >
>> > > > > Good points re: DefaultAccessDelegationModeResolver!
>> > > > >
>> > > > > Vignesh contributed PR [5884] to implement the new properties.
>> This PR
>> > > > > adjusts the logic in DefaultAccessDelegationModeResolver too.
>> Kudos to
>> > > > > Vugnesh!
>> > > > >
>> > > > > As far as I can tell, after [5884] if a client requests
>> > > > > "vended-credentials" but the server is configured
>> > > > > with skipCredentialDownscoping = true (for example), then the
>> client
>> > > will
>> > > > > receive a clear error message indicating that credential vending
>> is not
>> > > > > available.
>> > > > >
>> > > > > This is related to the Iceberg API clarification vote [1]. If
>> that spec
>> > > > > change passes in Iceberg, Polaris will be able to support "none"
>> as a
>> > > > > vending mode for a better client config experience, IMHO.
>> > > > >
>> > > > > In any case, please review [5884]. I think it still makes sense in
>> > > > Polaris
>> > > > > even without the Iceberg spec change.
>> > > > >
>> > > > > In Polaris code the main use case may be in tests, but some users
>> may
>> > > > want
>> > > > > to consider the new options for workload identity-based
>> deployments
>> > > too.
>> > > > >
>> > > > > [1]
>> https://lists.apache.org/thread/hc2764vozz7ygx5fjj64104hotrof0lk
>> > > > >
>> > > > > [5884] https://github.com/apache/polaris/pull/5884
>> > > > >
>> > > > > Cheers,
>> > > > > Dmitri.
>> > > > >
>> > > > > On Sun, Oct 25, 2026 at 5:53 AM Jean-Baptiste Onofré <
>> [email protected]>
>> > > > > wrote:
>> > > > >
>> > > > >> Hi all,
>> > > > >>
>> > > > >> +1 on the catalog-level
>> AzureStorageConfigInfo.skipSasTokenGeneration
>> > > > and
>> > > > >> GcpStorageConfigInfo.skipCredentialDownscoping properties. A
>> typed,
>> > > well
>> > > > >> defined setting is a better home for "don't vend" than the
>> > > > >> SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag, which only applies
>> at the
>> > > > >> realm level. It also matches what stsUnavailable already does
>> for S3.
>> > > > >>
>> > > > >> I agree with Ayush on the access delegation mode.
>> > > > >>
>> > > > >> I think it would be great to clarify two points before working
>> on the
>> > > > >> implementation PR:
>> > > > >> - If a catalog opts out of vending and the client asks only for
>> > > > >> vend-credentials, we should reject the request up front with a
>> clear
>> > > > >> error.
>> > > > >> Failing later inside the vending path is harder to diagnose
>> imho. This
>> > > > >> should apply to all three providers, so the S3 stsUnavailable
>> case
>> > > > behaves
>> > > > >> the same way. If changing the S3 behavior is out of the scope
>> for the
>> > > > PR,
>> > > > >> we can track as a follow-up, but we need consistency here imho.
>> > > > >> - Both properties should default to false, so existing catalogs
>> keep
>> > > > >> vending credentials. They should be optional in the Management
>> API, so
>> > > > >> this
>> > > > >> is a backward compatible addition. We should also document them
>> next
>> > > to
>> > > > >> stsUnavailable (in site/content/in-dev/unreleased/), and say
>> there
>> > > that
>> > > > >> they are meant for non-cloud or emulator deployments.
>> > > > >>
>> > > > >> Happy to review the PRs ;)
>> > > > >>
>> > > > >> Regards
>> > > > >> JB
>> > > > >>
>> > > > >> On Fri, Oct 23, 2026 at 8:15 PM Dmitri Bourlatchkov <
>> [email protected]
>> > > >
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Hi All,
>> > > > >> >
>> > > > >> > As discovered during the review of PR 5756 [1], now
>> > > > >> > that SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION is not recommended
>> for
>> > > > >> > mainstream use [3], deployments (e.g. tests) using non-cloud
>> options
>> > > > for
>> > > > >> > ADLS and GS (GCS) storage need a mechanism for opting out of
>> minting
>> > > > >> > session tokens and resolving to local (server and/or client)
>> > > > credentials
>> > > > >> > for storage access.
>> > > > >> >
>> > > > >> > I propose adding the following properties at the Management API
>> > > level
>> > > > >> [1].
>> > > > >> >
>> > > > >> > * AzureStorageConfigInfo.skipSasTokenGeneration
>> > > > >> >
>> > > > >> > * GcpStorageConfigInfo.skipCredentialDownscoping
>> > > > >> >
>> > > > >> > Both names relate to the corresponding native APIs in each
>> > > technology.
>> > > > >> >
>> > > > >> > Setting these properties will effectively disable credential
>> > > vending.
>> > > > >> This
>> > > > >> > is similar to stsUnavailable in AwsStorageConfigInfo.
>> > > > >> >
>> > > > >> > The main purpose of the new properties is establishing a
>> > > catalog-level
>> > > > >> opt
>> > > > >> > in/out mechanism and providing well-defined user-level
>> settings.
>> > > > >> >
>> > > > >> > PolarisStorageIntegration implementations still retain value
>> without
>> > > > >> > credential vending because they potentially define
>> non-credential
>> > > > >> > properties in StorageAccessConfig. The latter case currently
>> applies
>> > > > >> only
>> > > > >> > to S3, but downstream customizations may utilize it too.
>> > > > >> >
>> > > > >> > [1]
>> > > > https://github.com/apache/polaris/pull/5756#discussion_r4362688654
>> > > > >> >
>> > > > >> > [2]
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> https://github.com/apache/polaris/blob/5fafc67a832a8900fefb399abc00549d864470d6/spec/polaris-management-service.yml#L21
>> > > > >> >
>> > > > >> > [3]
>> > > https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg
>> > > > >> >
>> > > > >> > Thoughts?
>> > > > >> >
>> > > > >> > Thanks,
>> > > > >> > Dmitri.
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Dmitri Bourlatchkov
>> > > > > Senior Staff Software Engineer, Dremio
>> > > > > Dremio.com
>> > > > > <
>> > > >
>> > >
>> https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature
>> > > >
>> > > > /
>> > > > > Follow Us on LinkedIn <https://www.linkedin.com/company/dremio>
>> / Get
>> > > > > Started <https://www.dremio.com/get-started/>
>> > > > >
>> > > > >
>> > > > > The Agentic Lakehouse
>> > > > > The only lakehouse built for agents, managed by agents
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>>
>

Reply via email to