If engines have ambient credentials, they don't need to do anything. For example, the default behavior of Spark[2] and Trino[1] does not send the header `X-Iceberg-Access-Delegation`. Polaris won't vend any credentials. This is a more flexible way for users, supporting mixed use cases where some engines need credential vending and others don't.
1. Trino, explicitly documented as disabled by default: - [REST catalog documentation](https://trino.io/docs/current/object-storage/metastores.html#rest-catalog): `iceberg.rest-catalog.vended-credentials-enabled` “Defaults to `false`.” - [Header-setting code](https://github.com/trinodb/trino/blob/master/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/catalog/rest/IcebergRestCatalogPropertiesProvider.java#L61-L67): adds `X-Iceberg-Access-Delegation: vended-credentials` only when enabled; custom headers can also be configured. 2. Spark with Iceberg REST - [SparkCatalog](https://github.com/apache/iceberg/blob/main/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java#L156-L162): forwards the configured catalog options to Iceberg. - [RESTSessionCatalog](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L181-L187): constructs HTTP headers using `RESTUtil.configHeaders(config)`. - [RESTUtil.configHeaders](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/RESTUtil.java#L433-L435): extracts explicitly configured `header.*` properties; it supplies no default delegation value. Yufei On Wed, Nov 4, 2026 at 10:20 AM Dmitri Bourlatchkov via dev <[email protected]> wrote: > > Hi All, > > From my POV the use case is clear and PR [5884] is ready to merge. > > If there are no further concerns, I propose merging it on Nov 6. > > Thanks, > Dmitri. > > [5884] https://github.com/apache/polaris/pull/5884 > > On Mon, Nov 2, 2026 at 7:10 PM Dmitri Bourlatchkov <[email protected]> wrote: > > > Hi Yufei, > > > > > Could you share more details about the workload identity use case? > > > > Clients (Engines) may have ambient credentials in the execution env., > > which allows them to access storage (e.g. GCS). > > > > In this case, Polaris does not have to vend credentials. However, with the > > current GCS storage integration, Polaris cannot avoid vending credentials. > > > > If the admin user elects to use workload identity for clients (ambient > > credentials), Polaris credential vending work is useless overhead. Moreover > > it may present a security gap by exposing different credentials to clients, > > which are supposed to authenticate via another (workload identity) > > channel. I believe it is important for the admin user to be able to > > minimize credential exposure via Polaris in this case. > > > > > I think we need a concrete use case to decide whether we need this [...] > > > > From my POV the use case I described above is self-sufficient. > > > > A similar practical use case relates to your reasonable and timely > > proposal to limit the use of SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION [2]. > > > > If we are to take the next step and adapt our Integration Test to act as > > examples of how to avoid relying on this flag, this results in certain > > coding difficulties, which are evident [1] in [5884]. > > > > > [...] where the setting should go. > > > > The natural place for a new setting that controls access to derived > > credentials seems to be in the type-specific storage config [5884], as I > > proposed in my original email in this thread. > > > > Do you have an alternative suggestion for how to address this use case? > > > > Thanks, > > Dmitri. > > > > [1] https://github.com/apache/polaris/pull/5756#discussion_r4362688654 > > > > [2] https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg > > > > [5756] https://github.com/apache/polaris/pull/5756 > > > > [5884] https://github.com/apache/polaris/pull/5884 > > > > On Mon, Nov 2, 2026 at 3:31 PM Yufei Gu <[email protected]> wrote: > > > >> Hi Dmitri, > >> > >> Thanks for clarifying. My earlier comment was about clients choosing > >> not to request credential vending through X-Iceberg-Access-Delegation. > >> I understand that this is a server-side admin decision, which differs > >> from clients opting out. > >> > >> Could you share more details about the workload identity use case? I’m > >> still not clear why admins would need to disable credential vending in > >> that scenario. > >> > >> stsUnavailable is for S3-compatible storage without STS support. If > >> there’s a similar limitation for Azure/GCS, these settings make sense > >> to me. But if this is an admin policy, I’m not sure storage > >> configuration is the right place, given that a storage config may > >> apply to multiple tables or namespaces in the future (Rishi's > >> proposal). > >> > >> I think we need a concrete use case to decide whether we need this and > >> where the setting should go. WDYT? > >> > >> > >> Yufei > >> > >> On Fri, Oct 30, 2026 at 3:17 PM Dmitri Bourlatchkov via dev > >> <[email protected]> wrote: > >> > > >> > Hi Yufei, > >> > > >> > > [...] we could use the header to achieve the same result. > >> > > >> > The proposed change allows admin users to configure Polaris to avoid > >> > vending credentials in Azure and GCS environments. It is a deliberate > >> > server-side admin decision. > >> > > >> > This was possible for S3 before. PR [5884] aligns Polaris capabilities > >> in > >> > Azure and GCS with existing Polaris capabilities in S3 environments. > >> > > >> > What alternative do you suggest? I'm not sure I can see that in JB's or > >> > Ayush's messages. > >> > > >> > Thanks, > >> > Dmitri. > >> > > >> > [5884] https://github.com/apache/polaris/pull/5884 > >> > > >> > On Thu, Oct 29, 2026 at 7:30 PM Yufei Gu <[email protected]> wrote: > >> > > >> > > Apologies for the late reply. > >> > > > >> > > Thanks Dmitri for raising this, and Vignesh for working onit. As > >> Ayush and > >> > > JB mentioned, we could use the header to achieve the same result. Do > >> we > >> > > want to introduce another spec change, or am I missing some use cases? > >> > > > >> > > Yufei > >> > > > >> > > > >> > > On Thu, Oct 29, 2026 at 2:05 PM Dmitri Bourlatchkov <[email protected] > >> > > >> > > wrote: > >> > > > >> > > > Hi All, > >> > > > > >> > > > PR [5884] seems to be in a good shape. As far as I can tell, all > >> existing > >> > > > points have been addressed. > >> > > > > >> > > > If there are no objections, I propose merging it on Oct 30. > >> > > > > >> > > > Thanks, > >> > > > Dmitri. > >> > > > > >> > > > [5884] https://github.com/apache/polaris/pull/5884 > >> > > > > >> > > > On Tue, Oct 27, 2026 at 5:46 PM Dmitri Bourlatchkov < > >> > > > [email protected]> wrote: > >> > > > > >> > > > > Hi JB, Ayush, > >> > > > > > >> > > > > Good points re: DefaultAccessDelegationModeResolver! > >> > > > > > >> > > > > Vignesh contributed PR [5884] to implement the new properties. > >> This PR > >> > > > > adjusts the logic in DefaultAccessDelegationModeResolver too. > >> Kudos to > >> > > > > Vugnesh! > >> > > > > > >> > > > > As far as I can tell, after [5884] if a client requests > >> > > > > "vended-credentials" but the server is configured > >> > > > > with skipCredentialDownscoping = true (for example), then the > >> client > >> > > will > >> > > > > receive a clear error message indicating that credential vending > >> is not > >> > > > > available. > >> > > > > > >> > > > > This is related to the Iceberg API clarification vote [1]. If > >> that spec > >> > > > > change passes in Iceberg, Polaris will be able to support "none" > >> as a > >> > > > > vending mode for a better client config experience, IMHO. > >> > > > > > >> > > > > In any case, please review [5884]. I think it still makes sense in > >> > > > Polaris > >> > > > > even without the Iceberg spec change. > >> > > > > > >> > > > > In Polaris code the main use case may be in tests, but some users > >> may > >> > > > want > >> > > > > to consider the new options for workload identity-based > >> deployments > >> > > too. > >> > > > > > >> > > > > [1] > >> https://lists.apache.org/thread/hc2764vozz7ygx5fjj64104hotrof0lk > >> > > > > > >> > > > > [5884] https://github.com/apache/polaris/pull/5884 > >> > > > > > >> > > > > Cheers, > >> > > > > Dmitri. > >> > > > > > >> > > > > On Sun, Oct 25, 2026 at 5:53 AM Jean-Baptiste Onofré < > >> [email protected]> > >> > > > > wrote: > >> > > > > > >> > > > >> Hi all, > >> > > > >> > >> > > > >> +1 on the catalog-level > >> AzureStorageConfigInfo.skipSasTokenGeneration > >> > > > and > >> > > > >> GcpStorageConfigInfo.skipCredentialDownscoping properties. A > >> typed, > >> > > well > >> > > > >> defined setting is a better home for "don't vend" than the > >> > > > >> SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag, which only applies > >> at the > >> > > > >> realm level. It also matches what stsUnavailable already does > >> for S3. > >> > > > >> > >> > > > >> I agree with Ayush on the access delegation mode. > >> > > > >> > >> > > > >> I think it would be great to clarify two points before working > >> on the > >> > > > >> implementation PR: > >> > > > >> - If a catalog opts out of vending and the client asks only for > >> > > > >> vend-credentials, we should reject the request up front with a > >> clear > >> > > > >> error. > >> > > > >> Failing later inside the vending path is harder to diagnose > >> imho. This > >> > > > >> should apply to all three providers, so the S3 stsUnavailable > >> case > >> > > > behaves > >> > > > >> the same way. If changing the S3 behavior is out of the scope > >> for the > >> > > > PR, > >> > > > >> we can track as a follow-up, but we need consistency here imho. > >> > > > >> - Both properties should default to false, so existing catalogs > >> keep > >> > > > >> vending credentials. They should be optional in the Management > >> API, so > >> > > > >> this > >> > > > >> is a backward compatible addition. We should also document them > >> next > >> > > to > >> > > > >> stsUnavailable (in site/content/in-dev/unreleased/), and say > >> there > >> > > that > >> > > > >> they are meant for non-cloud or emulator deployments. > >> > > > >> > >> > > > >> Happy to review the PRs ;) > >> > > > >> > >> > > > >> Regards > >> > > > >> JB > >> > > > >> > >> > > > >> On Fri, Oct 23, 2026 at 8:15 PM Dmitri Bourlatchkov < > >> [email protected] > >> > > > > >> > > > >> wrote: > >> > > > >> > >> > > > >> > Hi All, > >> > > > >> > > >> > > > >> > As discovered during the review of PR 5756 [1], now > >> > > > >> > that SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION is not recommended > >> for > >> > > > >> > mainstream use [3], deployments (e.g. tests) using non-cloud > >> options > >> > > > for > >> > > > >> > ADLS and GS (GCS) storage need a mechanism for opting out of > >> minting > >> > > > >> > session tokens and resolving to local (server and/or client) > >> > > > credentials > >> > > > >> > for storage access. > >> > > > >> > > >> > > > >> > I propose adding the following properties at the Management API > >> > > level > >> > > > >> [1]. > >> > > > >> > > >> > > > >> > * AzureStorageConfigInfo.skipSasTokenGeneration > >> > > > >> > > >> > > > >> > * GcpStorageConfigInfo.skipCredentialDownscoping > >> > > > >> > > >> > > > >> > Both names relate to the corresponding native APIs in each > >> > > technology. > >> > > > >> > > >> > > > >> > Setting these properties will effectively disable credential > >> > > vending. > >> > > > >> This > >> > > > >> > is similar to stsUnavailable in AwsStorageConfigInfo. > >> > > > >> > > >> > > > >> > The main purpose of the new properties is establishing a > >> > > catalog-level > >> > > > >> opt > >> > > > >> > in/out mechanism and providing well-defined user-level > >> settings. > >> > > > >> > > >> > > > >> > PolarisStorageIntegration implementations still retain value > >> without > >> > > > >> > credential vending because they potentially define > >> non-credential > >> > > > >> > properties in StorageAccessConfig. The latter case currently > >> applies > >> > > > >> only > >> > > > >> > to S3, but downstream customizations may utilize it too. > >> > > > >> > > >> > > > >> > [1] > >> > > > https://github.com/apache/polaris/pull/5756#discussion_r4362688654 > >> > > > >> > > >> > > > >> > [2] > >> > > > >> > > >> > > > >> > > >> > > > >> > >> > > > > >> > > > >> https://github.com/apache/polaris/blob/5fafc67a832a8900fefb399abc00549d864470d6/spec/polaris-management-service.yml#L21 > >> > > > >> > > >> > > > >> > [3] > >> > > https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg > >> > > > >> > > >> > > > >> > Thoughts? > >> > > > >> > > >> > > > >> > Thanks, > >> > > > >> > Dmitri. > >> > > > >> > > >> > > > >> > >> > > > > > >> > > > > > >> > > > > -- > >> > > > > Dmitri Bourlatchkov > >> > > > > Senior Staff Software Engineer, Dremio > >> > > > > Dremio.com > >> > > > > < > >> > > > > >> > > > >> https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature > >> > > > > >> > > > / > >> > > > > Follow Us on LinkedIn <https://www.linkedin.com/company/dremio> > >> / Get > >> > > > > Started <https://www.dremio.com/get-started/> > >> > > > > > >> > > > > > >> > > > > The Agentic Lakehouse > >> > > > > The only lakehouse built for agents, managed by agents > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > >
