bharos commented on issue #10978:
URL: https://github.com/apache/gravitino/issues/10978#issuecomment-4394478252
I debugged similar issue internally with Azure AD, I can provide my findings
here if it can help with this issue (assuming this has similar cause)
TLDR: Use
spark.sql.catalog.my_catalog.rest.auth.oauth2.exchange-enabled=false with
iceberg runtime JAR version ≥1.10.1.
**Details:**
When Spark or Flink connects to Gravitino's Iceberg REST server using the
native Iceberg REST catalog client with credential=client_id:secret and Azure
AD as the IDP, any job longer than the access token lifetime (~1 hour) will
fail.
**Root Cause**
Iceberg v1's OAuth2Util.AuthSession uses token exchange (RFC 8693) as the
only refresh mechanism — even when client credentials are available.
Initial auth: credential → client_credentials grant → Azure AD →
access_token (1hr) ✅
Token refresh: access_token → token exchange grant → Azure AD → ERROR
❌
Azure AD does not support RFC 8693 token exchange. Azure's On-Behalf-Of flow
uses a different grant type (jwt-bearer). The token exchange request is
rejected.
Code Trace (Iceberg 1.6.1)
OAuth2Util.AuthSession.refresh():
refresh()
└→ refreshCurrentToken()
├─ token NOT expired → refreshToken() → token exchange request ❌
└─ token expired → refreshExpiredToken()
└→ credential != null?
→ refreshToken() → token exchange request ❌
Both paths call refreshToken() which builds a tokenExchangeRequest
(grant_type=urn:ietf:params:oauth:grant-type:token-exchange). There is no
fallback to client_credentials re-authentication in v1.
**Solution:**
Iceberg 1.10+ with exchange-enabled=false
PR [apache/iceberg#13809](https://github.com/apache/iceberg/pull/13809)
(merged Aug 2025) adds a flag to disable token exchange as the refresh path.
When disabled, Iceberg re-runs client_credentials on expiry.
spark.sql.catalog.my_catalog.oauth2-server-uri=https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token
spark.sql.catalog.my_catalog.rest.auth.oauth2.exchange-enabled=false
Requires upgrading iceberg-spark-runtime to ≥1.10.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
