lasdf1234 opened a new pull request, #11294:
URL: https://github.com/apache/gravitino/pull/11294
### What changes were proposed in this pull request?
This PR adds **Iceberg 1.11 vended credential refresh** for object storage
in the Iceberg REST Catalog (IRC), and includes the **server-side Iceberg
1.11.0 upgrade** required for the feature.
**Vended credential refresh (IRC)**
- Add `IcebergVendedCredentials` to map expiring Gravitino storage tokens to
Iceberg 1.11 client properties and table-scoped refresh paths (`GET
.../tables/{table}/credentials`).
- Extend `CredentialPropertyUtils.toIcebergProperties` with token expiry
metadata for S3/IRSA, OSS, and ADLS (GCS expiry already supported).
- Wire credential refresh into IRC load-table, credentials API, and
scan-plan responses in `CatalogWrapperForREST`.
- Set Iceberg REST credential `prefix` from table location so Iceberg 1.11
clients can match credentials by storage scheme during refresh.
**Iceberg 1.11 upgrade (included in this branch)**
- Upgrade server-side Iceberg to **1.11.0** for IRC and lakehouse catalog.
- Adapt IRC REST handlers, plan-scan response building, and related
Spark/Flink IT classpath expectations.
Related upgrade PR: #11219
### Why are the changes needed?
Iceberg 1.11 clients can refresh vended object-storage credentials before
they expire during long-running jobs. Gravitino IRC must return expiry fields
and table-scoped refresh endpoints for S3, GCS, OSS, and ADLS when temporary
credentials are enabled.
The server must run Iceberg 1.11 to align with the REST spec and client
refresh behavior.
### Does this PR introduce _any_ user-facing change?
- When `vended-credentials` is enabled, load-table / credentials / scan-plan
responses may include cloud-specific expiry fields and refresh-endpoint
properties (for example `client.refresh-credentials-endpoint`,
`gcs.oauth2.refresh-credentials-endpoint`, `adls.refresh-credentials-endpoint`).
- Refresh paths use the relative form without a leading slash:
`v1/{catalog}/namespaces/.../tables/.../credentials`.
- IRC and lakehouse Iceberg catalog documentation reflect **Iceberg 1.11.0**
server support.
### How was this patch tested?
- [x] `./gradlew :common:test --tests
org.apache.gravitino.credential.TestCredentialPropertiesUtils -PskipITs`
- [x] `./gradlew :iceberg:iceberg-rest-server:test --tests
org.apache.gravitino.iceberg.service.TestIcebergVendedCredentials --tests
org.apache.gravitino.iceberg.service.TestCatalogWrapperForREST -PskipITs`
- [x] `./gradlew :iceberg:iceberg-rest-server:compileJava
:iceberg:iceberg-rest-server:compileTestJava -PskipITs`
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
