Hi Adam, Thanks for the proposal.
I share Dmitri's question; my understanding is that this pertains to client-side encryption. I can confirm that KMS should work, as I recall an issue regarding this being fixed in the past. Adam, could you please clarify the scope of this work? Regards, JB On Tue, May 26, 2026 at 8:01 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Adam, > > Thanks for this proposal! > > Polaris should already support storage-side KMS in AWS (and compatible > systems) via [2802] (cf. [1]). > > I guess the new features you mention relate to client-side encryption, > right? > > [1] > https://polaris.apache.org/blog/2025/12/24/securing-s3-data-with-aws-kms/ > > [2802] https://github.com/apache/polaris/pull/2802 > > Cheers, > Dmitri. > > On Tue, May 26, 2026 at 11:06 AM Adam Szita <[email protected]> wrote: > > > Hi all, > > > > Iceberg 1.11 shipped the base implementation for table encryption, > > including KMS-based key wrapping/unwrapping and encrypted data/delete, > > manifest, and manifest-list files. REST catalog support is also being > > worked on in Iceberg (see https://github.com/apache/iceberg/pull/13225). > > > > I have been testing Polaris with Iceberg REST client-side encryption > > enabled. Basic catalog operations such as loadTable, commit/drop without > > purge, list, etc. work without Polaris changes because Polaris only needs > > the table metadata JSON for those paths, and metadata.json is not > > encrypted. > > > > The places where Polaris does need encryption awareness are the > server-side > > paths that read encrypted Iceberg artifacts. The first concrete example > is > > drop table with purge: TableCleanupTask reads snapshot manifest lists and > > manifests to enumerate files for deletion, so it needs to use an > > EncryptingFileIO. The same would apply to any Polaris-side table > > maintenance/optimization, orphan/snapshot cleanup logic, or any future > > remote scan/planning capability that reads manifests or data/delete > files. > > > > There is also a related but separate topic around vending KMS credentials > > to clients. That likely needs Iceberg REST spec work first, similar in > > spirit to current storage credential vending, so I think it should be > > designed for but not required as the first Polaris step. > > > > The first Polaris-side building block I would propose is to allow Iceberg > > catalogs to carry KMS configuration, similarly to how catalogs currently > > carry StorageConfigurationInfo. This should be separate from storage > > configuration because the storage backend and KMS provider may differ, > for > > example GCS storage with AWS KMS. AWS KMS would be a reasonable first > > implementation target, using Iceberg’s existing KeyManagementClient/AWS > KMS > > support, while leaving the model extensible for Azure and GCP. > > > > I have already been experimenting with this locally and would be happy to > > work on the Polaris changes. A possible first PR could be limited to: > > > > 1. Add catalog-level KMS configuration model/API support. > > 2. Add AWS KMS server-side configuration wiring. > > > > Any feedback is welcome. > > > > Cheers, > > Adam > > >
