Hi folks, I wrote up a spec proposal for the KMS credential vending part of this discussion: https://docs.google.com/document/d/1VSewbVmjukU5eTiZruCJVmUZcRZvcAsd4JOeC6Lt3fo
The intent is to separate this from the current key-metadata / encryption-metadata work, so that the existing PR can hopefully move forward independently while we discuss the REST credential-vending shape here. At a high level, the proposal mirrors storage credential vending: the REST catalog can return short-lived, scoped key-management-credentials, and clients match those credentials to KMS key IDs referenced by Iceberg encryption metadata. Any feedback welcome, especially on whether this is the right REST-level direction and if the proposed response shape is enough for current table-level encryption while also leaving room for future column-level encryption. Cheers, Adam On Thu, 7 May 2026 at 13:56, Ádám Szita <[email protected]> wrote: > Hi all, > > I think there are two related but separate topics here: > > 1. supporting Iceberg table encryption with a REST catalog client, and > 2. defining a REST mechanism for vending/refreshing KMS or Vault > credentials. > > For the current PR, I don’t think we necessarily need to solve #2. > Yes, REST catalog deployments may use credential vending for storage > access, > but that is still a catalog feature/configuration choice rather than a > requirement > for REST catalog usage. So, similarly, I don’t think supporting table > encryption > with a REST catalog should require the catalog to also implement KMS > credential vending. > > I do think it's a good idea to have the KMS creds provided by the catalog > side, > especially for catalogs that want to enforce least privilege at table > scope. > But I see that as a follow-up design/API discussion. I also think it > should be > general enough to avoid coupling certain cloud storage types with their > respective KMS offerings. (e.g. S3 storage doesn't necessarily imply AWS > KMS > usage). > > We have KeyManagementClient interface that users can implement > for their own KMS needs. Large enterprise deployments may already use an > internal KMS, I don’t think we should require them to implement both > this interface and KMS credential vending before they can use table > encryption with a REST catalog. > > Let me know what you all think. > > Cheers, > Adam > > On 2026/05/06 04:59:23 Prashant Singh wrote: > > Thanks everyone for chiming in on this, > > > > *Regarding the PR : * > > The approach we are taking here where we pick encryption creds from the > > client side (catalog level) and storage creds from server vended creds, > is > > fairly concerning to me. > > if the REST server is giving one access because someone has *SELECT* > > grant, *SERVER* should mint them everything that required for the client > to > > successfully read the table, sending storage creds but not KMS creds > > feels incorrect to me, how is the server gonna know the client has creds > > for *decryption* for this already ? This is entirely different from what > > other catalogs expect i.e entirely a client side configuration for > example > > Hive Catalog / Glue catalog etc.... > > > > Regarding backward compatibility, if we do this, server vending KMS > > credentials eventually, these clients will simply not work since they > > always expect the credentials to come from the client side (Note : I am > > strictly talking of table level) > > > > *Regarding the Spec Change : * > > It makes sense to me as well, to decouple KMS & storage creds ! > > > > All in all I believe we should conclude this discussion and not rush into > > things (we can make incremental progress for sure), if we have open > > questions and an awaiting spec change. > > > > Best, > > Prashant Singh > > > > On Tue, May 5, 2026 at 12:11 AM Gidon Gershinsky <[email protected]> > wrote: > > > > > Hi all, > > > > > > The built-in encryption is designed to protect tables in any storage > > > backend, including untrusted storage. So I also think the storage > > > credentials and KMS credentials are best treated separately, as they > don't > > > have the same purpose/scope. > > > > > > Regarding the current PR (13225) - it doesn't handle credentials > directly; > > > instead, it focuses on building the scaffolding for table encryption > in the > > > REST catalog client - similarly to the Hive catalog client (and other > > > catalogs in the future). > > > > > > Cheers, Gidon > > > > > > > > > On Tue, May 5, 2026 at 3:40 AM Chris Lu <[email protected]> wrote: > > > > > >> Hi Prashant, > > >> > > >> My own preference would be to keep storage credentials and > encryption/KMS > > >> credentials separate in the REST contract. > > >> > > >> Storage credentials and KMS/Vault credentials have different scopes, > > >> lifetimes, providers, and failure modes. > > >> > > >> > > >> - Storage credentials authorize access to object locations. > > >> - > > >> > > >> KMS/Vault credentials authorize key wrap/unwrap operations. > > >> > > >> Object-store SSE-KMS is a storage-layer feature, but Iceberg table > > >> encryption is not necessarily tied to the object store. Vault or > > >> external KMS support would not naturally fit inside a storage > credential > > >> object. So mixing KMS credentials into the storage credentials object > > >> may work for some S3/SSE cases, but it makes the REST model harder to > > >> extend. > > >> > > >> A cleaner model may be to have separate sections for storage > credentials > > >> and encryption credentials. Another option is a generic typed > credential > > >> model, where each credential has a type such as storage or > encryption, and > > >> a provider such as S3, Vault, or a cloud KMS. > > >> > > >> The typed model may be easier to extend later if we expect more > > >> credential types. > > >> > > >> For backward compatibility, I think the current PR can still move > > >> forward, but the client should not assume that storage credentials > are the > > >> permanent place for KMS credentials. If we need an interim > implementation, > > >> maybe it should be documented as catalog-level KMS configuration > support, > > >> not table-level encryption credential vending. > > >> > > >> I also think the catalog should have a clear capability or failure > > >> signal. If a table requires Iceberg encryption and the catalog > supports > > >> vended credentials, then the catalog should either return the required > > >> encryption credentials or fail early during table load or credential > > >> refresh with a clear error. Failing later during file IO or key unwrap > > >> would be much harder to debug. > > >> > > >> So my suggested direction would be: > > >> > > >> > > >> - Keep object-store SSE credentials/config separate from Iceberg > > >> table encryption credentials. > > >> - > > >> > > >> Add a dedicated encryption credential model, or a generic typed > > >> credential model. > > >> - > > >> > > >> Add a capability or requirement signal so clients can fail early > when > > >> required credentials are missing. > > >> > > >> Avoid baking in the assumption that KMS credentials always live under > > >> storage credentials. This should leave room for catalogs that vend > > >> table-scoped KMS/Vault credentials in the future, instead of only > > >> supporting catalog-level KMS client configuration. > > >> > > >> Thanks, > > >> Chris > > >> > > > > > > On Tue, May 5, 2026 at 3:22 AM Sreesh Maheshwar < > [email protected]> > > > wrote: > > > > > >> Hey Prashant, > > >> > > >> Thank you for kicking off this discussion! Some thoughts: > > >> > > >> > The current PR advocates for including KMS credentials within the > > >> storage credentials object. > > >> > > >> This isn't true - the KMS client is currently configured with just the > > >> catalog config client-side (though that includes the server's > `getConfig` > > >> response) [1]; there is no per-table vending of KMS credentials as > such. > > >> > > >> > Separation of KMS/Vault Creds from Storage Creds > > >> > > >> I agree, if/when it supports this, the AWS KMS client could choose to > > >> introduce `kms.access-key-id` and co. instead of reusing > `s3.access-key-id` > > >> and co. (similar to how we have `rest.access-key-id` and co. [2]) to > > >> differentiate from storage credentials. And similarly for vending: if > we'd > > >> like to support the vending of KMS credentials [3], we can discuss its > > >> format to enable separation. > > >> > > >> (Specifically on the current PR [4], I don't think that the concerns > > >> described affect it due to my first point above. I think > > >> catalog-initialised KMS clients, as implemented, are useful to > support now > > >> and in the future.) > > >> > > >> Thanks, > > >> Sreesh Maheshwar > > >> > > >> [1] > > >> > https://github.com/apache/iceberg/pull/13225/files#diff-86450612dbe323d6d06cbc3846aa1913f042eaedadc0ca027c36bfbe08d3a46cR284 > > >> [2] > > >> > https://github.com/apache/iceberg/blob/2d54125734ddc9b9fb87db147ff255918108fa2c/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L193 > > >> [3] https://github.com/apache/iceberg/issues/16194 > > >> > > > [4] https://github.com/apache/iceberg/pull/13225 > > > > > >> > > >> On Monday, May 4th, 2026 at 9:03 AM, Prashant Singh < > > >> [email protected]> wrote: > > >> > > >> Hi everyone, > > >> > > >> I’d like to start a discussion regarding how we handle credentials for > > >> encryption (like KMS or Vault (recently being discussed)) in the REST > > >> catalog. > > >> > > >> As we know, unlike other catalogs, the REST catalog mints credentials > at > > >> the table level for the client to use in subsequent operations, and we > > >> already have the dedicated /credentials endpoint in place to handle > > >> refreshing these. > > >> > > >> While reviewing the recent encryption PR, a few architectural concerns > > >> came up that I believe we need to conclude on before we mark the REST > > >> catalog as "ready" for supporting encryption: > > >> > > >> - > > >> > > >> *Separation of KMS/Vault Creds from Storage Creds:* How should we > > >> handle external key managers like Vault? The current PR [1] > advocates for > > >> including KMS credentials within the storage credentials object. > If we end > > >> up supporting Vault, this can't be mixed with storage cred. *(Side > > >> note: catalogs have historically mixed KMS creds with this object > for > > >> things like SSE, but that is entirely an object-store-level > concept).* > > >> We need a clear path forward for how REST will return per-table > credentials > > >> specifically for Vault/KMS stores. > > >> - > > >> > > >> *Catalog Awareness & Client-Side Assertions:* If the catalog > returns > > >> credentials, it overrides the client-side credentials. This means > a naive > > >> catalog that is unaware of encryption and just treats metadata > as-is will > > >> have no clue it needs to vend these specific KMS credentials and > if it > > >> forgerts to do that there is no way for the client to know this > (for object > > >> store cases) except to fail during runtime ? Should the catalog > fail such > > >> requests *as part of their contract of supporting v3* (we don't > need > > >> this in spec), should the catalog send some signal, hey i send you > creds > > >> for encryption too, and if the client doesn't find it fails early ? > > >> - > > >> > > >> *Backward Compatibility Risks:* If we release the client now with > the > > >> expectation that "storage credentials" will always contain KMS > credentials. > > >> If we later introduce a dedicated field for encryption credentials > in the > > >> loadTable response, we will be forced to maintain backward > > >> compatibility to support both ways of returning credentials. > > >> > > >> To be clear, I do not want to block the progress on the current PR. I > > >> really appreciate all the hard work that has gone into it! However, I > think > > >> it is crucial that we align on these design points for the REST > catalog's > > >> encryption architecture before finalizing it. > > >> > > >> I would appreciate any thoughts or feedback on how we should structure > > >> this. > > >> > > >> [1] https://github.com/apache/iceberg/pull/13225 > > >> > > >> Thanks, > > >> > > >> Prashant Singh > > >> > > >> > > >> > > >
