Hi Yufei, Polaris will not vend credentials to clients if X-Iceberg-Access-Delegation is not specified. However, it will vend if a client uses X-Iceberg-Access-Delegation.
Polaris Servers cannot assume that all clients are well-behaving. I believe it is valuable for Polaris to provide a configuration option for admin users to have better control in this situation. Secondary to that that, I think it should be possible to configure the Polaris Server itself to use ambient credentials for its own storage access. This follows from your earlier proposal (to which I agree) to relegate SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION to test-only use cases [1]. [1] https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg Cheers, Dmitri. On Wed, Nov 4, 2026 at 1:53 PM Yufei Gu <[email protected]> wrote: > If engines have ambient credentials, they don't need to do anything. > For example, the default behavior of Spark[2] and Trino[1] does not > send the header `X-Iceberg-Access-Delegation`. Polaris won't vend any > credentials. This is a more flexible way for users, supporting mixed > use cases where some engines need credential vending and others don't. > > 1. Trino, explicitly documented as disabled by default: > - [REST catalog > documentation]( > https://trino.io/docs/current/object-storage/metastores.html#rest-catalog > ): > `iceberg.rest-catalog.vended-credentials-enabled` “Defaults to > `false`.” > - [Header-setting > code]( > https://github.com/trinodb/trino/blob/master/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/catalog/rest/IcebergRestCatalogPropertiesProvider.java#L61-L67 > ): > adds `X-Iceberg-Access-Delegation: vended-credentials` only when > enabled; custom headers can also be configured. > > 2. Spark with Iceberg REST > - [SparkCatalog]( > https://github.com/apache/iceberg/blob/main/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java#L156-L162 > ): > forwards the configured catalog options to Iceberg. > - [RESTSessionCatalog]( > https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L181-L187 > ): > constructs HTTP headers using `RESTUtil.configHeaders(config)`. > - [RESTUtil.configHeaders]( > https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/RESTUtil.java#L433-L435 > ): > extracts explicitly configured `header.*` properties; it supplies no > default delegation value. > > Yufei > > On Wed, Nov 4, 2026 at 10:20 AM Dmitri Bourlatchkov via dev > <[email protected]> wrote: > > > > Hi All, > > > > From my POV the use case is clear and PR [5884] is ready to merge. > > > > If there are no further concerns, I propose merging it on Nov 6. > > > > Thanks, > > Dmitri. > > > > [5884] https://github.com/apache/polaris/pull/5884 > > > > On Mon, Nov 2, 2026 at 7:10 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > > > Hi Yufei, > > > > > > > Could you share more details about the workload identity use case? > > > > > > Clients (Engines) may have ambient credentials in the execution env., > > > which allows them to access storage (e.g. GCS). > > > > > > In this case, Polaris does not have to vend credentials. However, with > the > > > current GCS storage integration, Polaris cannot avoid vending > credentials. > > > > > > If the admin user elects to use workload identity for clients (ambient > > > credentials), Polaris credential vending work is useless overhead. > Moreover > > > it may present a security gap by exposing different credentials to > clients, > > > which are supposed to authenticate via another (workload identity) > > > channel. I believe it is important for the admin user to be able to > > > minimize credential exposure via Polaris in this case. > > > > > > > I think we need a concrete use case to decide whether we need this > [...] > > > > > > From my POV the use case I described above is self-sufficient. > > > > > > A similar practical use case relates to your reasonable and timely > > > proposal to limit the use of SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION > [2]. > > > > > > If we are to take the next step and adapt our Integration Test to act > as > > > examples of how to avoid relying on this flag, this results in certain > > > coding difficulties, which are evident [1] in [5884]. > > > > > > > [...] where the setting should go. > > > > > > The natural place for a new setting that controls access to derived > > > credentials seems to be in the type-specific storage config [5884], as > I > > > proposed in my original email in this thread. > > > > > > Do you have an alternative suggestion for how to address this use case? > > > > > > Thanks, > > > Dmitri. > > > > > > [1] https://github.com/apache/polaris/pull/5756#discussion_r4362688654 > > > > > > [2] https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg > > > > > > [5756] https://github.com/apache/polaris/pull/5756 > > > > > > [5884] https://github.com/apache/polaris/pull/5884 > > > > > > On Mon, Nov 2, 2026 at 3:31 PM Yufei Gu <[email protected]> wrote: > > > > > >> Hi Dmitri, > > >> > > >> Thanks for clarifying. My earlier comment was about clients choosing > > >> not to request credential vending through X-Iceberg-Access-Delegation. > > >> I understand that this is a server-side admin decision, which differs > > >> from clients opting out. > > >> > > >> Could you share more details about the workload identity use case? I’m > > >> still not clear why admins would need to disable credential vending in > > >> that scenario. > > >> > > >> stsUnavailable is for S3-compatible storage without STS support. If > > >> there’s a similar limitation for Azure/GCS, these settings make sense > > >> to me. But if this is an admin policy, I’m not sure storage > > >> configuration is the right place, given that a storage config may > > >> apply to multiple tables or namespaces in the future (Rishi's > > >> proposal). > > >> > > >> I think we need a concrete use case to decide whether we need this and > > >> where the setting should go. WDYT? > > >> > > >> > > >> Yufei > > >> > > >> On Fri, Oct 30, 2026 at 3:17 PM Dmitri Bourlatchkov via dev > > >> <[email protected]> wrote: > > >> > > > >> > Hi Yufei, > > >> > > > >> > > [...] we could use the header to achieve the same result. > > >> > > > >> > The proposed change allows admin users to configure Polaris to avoid > > >> > vending credentials in Azure and GCS environments. It is a > deliberate > > >> > server-side admin decision. > > >> > > > >> > This was possible for S3 before. PR [5884] aligns Polaris > capabilities > > >> in > > >> > Azure and GCS with existing Polaris capabilities in S3 environments. > > >> > > > >> > What alternative do you suggest? I'm not sure I can see that in > JB's or > > >> > Ayush's messages. > > >> > > > >> > Thanks, > > >> > Dmitri. > > >> > > > >> > [5884] https://github.com/apache/polaris/pull/5884 > > >> > > > >> > On Thu, Oct 29, 2026 at 7:30 PM Yufei Gu <[email protected]> > wrote: > > >> > > > >> > > Apologies for the late reply. > > >> > > > > >> > > Thanks Dmitri for raising this, and Vignesh for working onit. As > > >> Ayush and > > >> > > JB mentioned, we could use the header to achieve the same result. > Do > > >> we > > >> > > want to introduce another spec change, or am I missing some use > cases? > > >> > > > > >> > > Yufei > > >> > > > > >> > > > > >> > > On Thu, Oct 29, 2026 at 2:05 PM Dmitri Bourlatchkov < > [email protected] > > >> > > > >> > > wrote: > > >> > > > > >> > > > Hi All, > > >> > > > > > >> > > > PR [5884] seems to be in a good shape. As far as I can tell, all > > >> existing > > >> > > > points have been addressed. > > >> > > > > > >> > > > If there are no objections, I propose merging it on Oct 30. > > >> > > > > > >> > > > Thanks, > > >> > > > Dmitri. > > >> > > > > > >> > > > [5884] https://github.com/apache/polaris/pull/5884 > > >> > > > > > >> > > > On Tue, Oct 27, 2026 at 5:46 PM Dmitri Bourlatchkov < > > >> > > > [email protected]> wrote: > > >> > > > > > >> > > > > Hi JB, Ayush, > > >> > > > > > > >> > > > > Good points re: DefaultAccessDelegationModeResolver! > > >> > > > > > > >> > > > > Vignesh contributed PR [5884] to implement the new properties. > > >> This PR > > >> > > > > adjusts the logic in DefaultAccessDelegationModeResolver too. > > >> Kudos to > > >> > > > > Vugnesh! > > >> > > > > > > >> > > > > As far as I can tell, after [5884] if a client requests > > >> > > > > "vended-credentials" but the server is configured > > >> > > > > with skipCredentialDownscoping = true (for example), then the > > >> client > > >> > > will > > >> > > > > receive a clear error message indicating that credential > vending > > >> is not > > >> > > > > available. > > >> > > > > > > >> > > > > This is related to the Iceberg API clarification vote [1]. If > > >> that spec > > >> > > > > change passes in Iceberg, Polaris will be able to support > "none" > > >> as a > > >> > > > > vending mode for a better client config experience, IMHO. > > >> > > > > > > >> > > > > In any case, please review [5884]. I think it still makes > sense in > > >> > > > Polaris > > >> > > > > even without the Iceberg spec change. > > >> > > > > > > >> > > > > In Polaris code the main use case may be in tests, but some > users > > >> may > > >> > > > want > > >> > > > > to consider the new options for workload identity-based > > >> deployments > > >> > > too. > > >> > > > > > > >> > > > > [1] > > >> https://lists.apache.org/thread/hc2764vozz7ygx5fjj64104hotrof0lk > > >> > > > > > > >> > > > > [5884] https://github.com/apache/polaris/pull/5884 > > >> > > > > > > >> > > > > Cheers, > > >> > > > > Dmitri. > > >> > > > > > > >> > > > > On Sun, Oct 25, 2026 at 5:53 AM Jean-Baptiste Onofré < > > >> [email protected]> > > >> > > > > wrote: > > >> > > > > > > >> > > > >> Hi all, > > >> > > > >> > > >> > > > >> +1 on the catalog-level > > >> AzureStorageConfigInfo.skipSasTokenGeneration > > >> > > > and > > >> > > > >> GcpStorageConfigInfo.skipCredentialDownscoping properties. A > > >> typed, > > >> > > well > > >> > > > >> defined setting is a better home for "don't vend" than the > > >> > > > >> SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag, which only > applies > > >> at the > > >> > > > >> realm level. It also matches what stsUnavailable already does > > >> for S3. > > >> > > > >> > > >> > > > >> I agree with Ayush on the access delegation mode. > > >> > > > >> > > >> > > > >> I think it would be great to clarify two points before > working > > >> on the > > >> > > > >> implementation PR: > > >> > > > >> - If a catalog opts out of vending and the client asks only > for > > >> > > > >> vend-credentials, we should reject the request up front with > a > > >> clear > > >> > > > >> error. > > >> > > > >> Failing later inside the vending path is harder to diagnose > > >> imho. This > > >> > > > >> should apply to all three providers, so the S3 stsUnavailable > > >> case > > >> > > > behaves > > >> > > > >> the same way. If changing the S3 behavior is out of the scope > > >> for the > > >> > > > PR, > > >> > > > >> we can track as a follow-up, but we need consistency here > imho. > > >> > > > >> - Both properties should default to false, so existing > catalogs > > >> keep > > >> > > > >> vending credentials. They should be optional in the > Management > > >> API, so > > >> > > > >> this > > >> > > > >> is a backward compatible addition. We should also document > them > > >> next > > >> > > to > > >> > > > >> stsUnavailable (in site/content/in-dev/unreleased/), and say > > >> there > > >> > > that > > >> > > > >> they are meant for non-cloud or emulator deployments. > > >> > > > >> > > >> > > > >> Happy to review the PRs ;) > > >> > > > >> > > >> > > > >> Regards > > >> > > > >> JB > > >> > > > >> > > >> > > > >> On Fri, Oct 23, 2026 at 8:15 PM Dmitri Bourlatchkov < > > >> [email protected] > > >> > > > > > >> > > > >> wrote: > > >> > > > >> > > >> > > > >> > Hi All, > > >> > > > >> > > > >> > > > >> > As discovered during the review of PR 5756 [1], now > > >> > > > >> > that SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION is not > recommended > > >> for > > >> > > > >> > mainstream use [3], deployments (e.g. tests) using > non-cloud > > >> options > > >> > > > for > > >> > > > >> > ADLS and GS (GCS) storage need a mechanism for opting out > of > > >> minting > > >> > > > >> > session tokens and resolving to local (server and/or > client) > > >> > > > credentials > > >> > > > >> > for storage access. > > >> > > > >> > > > >> > > > >> > I propose adding the following properties at the > Management API > > >> > > level > > >> > > > >> [1]. > > >> > > > >> > > > >> > > > >> > * AzureStorageConfigInfo.skipSasTokenGeneration > > >> > > > >> > > > >> > > > >> > * GcpStorageConfigInfo.skipCredentialDownscoping > > >> > > > >> > > > >> > > > >> > Both names relate to the corresponding native APIs in each > > >> > > technology. > > >> > > > >> > > > >> > > > >> > Setting these properties will effectively disable > credential > > >> > > vending. > > >> > > > >> This > > >> > > > >> > is similar to stsUnavailable in AwsStorageConfigInfo. > > >> > > > >> > > > >> > > > >> > The main purpose of the new properties is establishing a > > >> > > catalog-level > > >> > > > >> opt > > >> > > > >> > in/out mechanism and providing well-defined user-level > > >> settings. > > >> > > > >> > > > >> > > > >> > PolarisStorageIntegration implementations still retain > value > > >> without > > >> > > > >> > credential vending because they potentially define > > >> non-credential > > >> > > > >> > properties in StorageAccessConfig. The latter case > currently > > >> applies > > >> > > > >> only > > >> > > > >> > to S3, but downstream customizations may utilize it too. > > >> > > > >> > > > >> > > > >> > [1] > > >> > > > > https://github.com/apache/polaris/pull/5756#discussion_r4362688654 > > >> > > > >> > > > >> > > > >> > [2] > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > >> > > > > > >> > > > > >> > https://github.com/apache/polaris/blob/5fafc67a832a8900fefb399abc00549d864470d6/spec/polaris-management-service.yml#L21 > > >> > > > >> > > > >> > > > >> > [3] > > >> > > https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg > > >> > > > >> > > > >> > > > >> > Thoughts? > > >> > > > >> > > > >> > > > >> > Thanks, > > >> > > > >> > Dmitri. > > >> > > > >> > > > >> > > > >> > > >> > > > > > > >> > > > > > > >> > > > > -- > > >> > > > > Dmitri Bourlatchkov > > >> > > > > Senior Staff Software Engineer, Dremio > > >> > > > > Dremio.com > > >> > > > > < > > >> > > > > > >> > > > > >> > https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature > > >> > > > > > >> > > > / > > >> > > > > Follow Us on LinkedIn < > https://www.linkedin.com/company/dremio> > > >> / Get > > >> > > > > Started <https://www.dremio.com/get-started/> > > >> > > > > > > >> > > > > > > >> > > > > The Agentic Lakehouse > > >> > > > > The only lakehouse built for agents, managed by agents > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >
