Hi all,
I'm Nevin. I'm building a governed table encryption feature based on
Apache Gravitino. I've been following the encryption-over-REST work
and would love to help move it forward, whether through reviews,
implementation, tests, or a short write-up of the options.
Reading the three PRs [1][2][3], the #17155 design doc [4] and
the earlier dev@ thread [5], I think the discussion comes down to two
design questions the spec hasn't settled yet:
1. May a client get storage and KMS credentials from different
sources (e.g. vended storage, its own KMS)?
See [4], [5], the review on [1], and [3].
2. Should naming the KMS be separate from supplying KMS credentials
(e.g. the catalog names it, the client uses its own credentials)?
See [1] and [2].
My current leaning is toward landing [1] first, narrowed as its
review proposed:
- This is the simple case: the client uses its own credentials for
both storage and KMS.
- The catalog passes KMS configuration (which KMS, e.g.
encryption.kms-type) as ordinary catalog properties, and each
table carries its encryption.key-id. These are pointers, not
credentials; they grant nothing.
- The REST client builds its KMS client from catalog and client
properties, as the Hive catalog client does today, and calls
the KMS with whatever credentials its environment provides.
- No new REST endpoints, fields or credential types. A source-
compatibility question about the protected newTableOps hooks was
raised on [1] and is still open.
On the mixed case from question 1, vended storage credentials with
the client's own KMS credentials ("hybrid" in [3]): it has real
value, as argued on [1], but needs more design, such as how a client
opts in and whether it fits [4]. I'm not opposed, and an explicit
opt-in flag as in [3] seems fine to me. Either way, I'd suggest the
simple case not wait on it.
I see a few benefits:
- Safe: the catalog needs neither key material nor KMS credentials;
key access stays with the client's own identity.
- Deployers: keys stay governed by their existing IAM.
- Catalog implementers: nothing to build beyond storing and
returning properties; no per-cloud KMS credential minting.
- Engines: the same encryption path as the Hive catalog.
- A flexible starting point: KMS credential vending [2] and other
access-delegation modes can be added later as opt-ins without
breaking it.
How this relates to the open proposals:
- [1] #13225: the simple case above; its review proposed this scope.
- [2] #17155: complementary. It covers the catalog vending KMS
credentials; the simple case covers clients using their own.
The two can coexist, and both use encryption.kms-type.
- [3] #18081: explores hybrid behind an opt-in flag; it can follow
once that design settles.
- [4] Design doc: consistent. The simple case vends nothing;
whether hybrid fits [4] is question 1.
I'd welcome any views on this and on the two questions above.
Thanks!
Warmly,
Nevin
[1] #13225 (Sreesh Maheshwar), Core: Support client-configured
encryption in RESTCatalog. The REST client builds a KMS client
from catalog and client properties.
https://www.google.com/url?q=https://github.com/apache/iceberg/pull/13225&source=gmail&ust=1791870345063000&sa=E
[2] #17155 (Adam Szita), Spec: REST KMS credential vending for table
encryption. Adds the kms-vended-credentials access-delegation
mode, encryption.kms-type as a catalog property, and
key-management-credentials keyed by KMS key ID.
https://www.google.com/url?q=https://github.com/apache/iceberg/pull/17155&source=gmail&ust=1791870345063000&sa=E
[3] #18081 (Prashant Singh, draft), Core: [Demonstration only]
handle hybrid mode encryption with REST Catalog. Catalog-vended
storage credentials with the client's own KMS credentials,
behind an opt-in flag.
https://www.google.com/url?q=https://github.com/apache/iceberg/pull/18081&source=gmail&ust=1791870345063000&sa=E
[4] "KMS credential vending for Iceberg table encryption" (Adam
Szita), design doc for #17155. Says a client must not be
required to combine delegated storage credentials with
client-local/static KMS credentials; a comment adds that they
"need to be vended either together or not at all".
https://www.google.com/url?q=https://docs.google.com/document/d/1VSewbVmjukU5eTiZruCJVmUZcRZvcAsd4JOeC6Lt3fo&source=gmail&ust=1791870345063000&sa=E
[5] dev@ thread, "[DISCUSS] REST Catalog: Credential management for
KMS/Vault and Table level Encryption" (Prashant Singh, May to
July 2026). The discussion that led to [4] and #17155.
https://www.google.com/url?q=https://www.mail-archive.com/[email protected]/msg13452.html&source=gmail&ust=1791870345063000&sa=E
--
Nevin Zheng
๐จ๐ปโ๐ป Software Engineer ยท ๐ข๏ธ Databaseologist
๐ช Datastrato
โ๏ธ [email protected]
๐ https://www.linkedin.com/in/nevinzheng/
Robert'); DROP TABLE students;--
Sent from my ๐ค (Opus 5.5)
