Hi JB / MK / Yufei / Dmitri Previously there was an alignment to support KMS support at multiple levels
1. Default with no key configuration 2. Catalog level with key configuration 3. Table level key configuration It seems the PR is leaning towards handling #2. I initially wanted to address #1 as part of this PR <https://github.com/apache/polaris/pull/1424/files>, but unfortunately got delayed. We didn't want to create friction with users for data lakes in case of hard rotation(disable and create new) of keys. I wanted to understand the current stance on KMS.Could you let us know your thoughts on #1? On Tue, Oct 14, 2025 at 11:22 PM Michael Collado <[email protected]> wrote: > Sorry, my ask re: 2 was to include this support in the PR - right now in > > https://github.com/apache/polaris/pull/2802/files#diff-d305f7a426a7690c576722c114257792b3fcee726624655d15893b71499827f8R181 > , the KMS key is specified by > reading storageConfigurationInfo.getKmsKeyArn() in the storage integration. > If someone creates a table with a different key, as in your example, that > key will be ignored in the policy generation. > > Mike > > On Tue, Oct 14, 2025 at 7:02 AM Rizzo Cascio, Fabio > <[email protected]> wrote: > > > Hi Mike, > > > > 2. Yes, that is correct. The override should be already doable in > Polaris > > during a table creation, something like this: > > > > { > > "name": "my_table", > > "location": "s3://your-bucket/path/to/table", > > "schema": { > > "type": "struct", > > "fields": [ > > { "id": 1, "name": "id", "type": "long", "required": true }, > > { "id": 2, "name": "data", "type": "string", "required": false } > > ] > > }, > > "properties": { > > "s3.encryption.key": > > "arn:aws:kms:us-west-2:123456789012:key/your-key-id" > > } > > } > > > > The options are: > > 1. use the kms key when you create the catalog so that all data in the > > bucket is encrypted with the same key > > 2. Don't use the kms key when you create the catalog and use different > > keys when you create tables > > > > We use number one cause number 2 gets expensive when you have a lot of > > data. > > > > > > 1. > > > > > > From: Michael Collado <[email protected]> > > Date: Monday, 13 October 2025 at 18:07 > > To: [email protected] <[email protected]> > > Subject: [EXTERNAL]Re: KMS Key addition for s3 > > > > Very exciting change. A couple of questions/suggestions: > > > > 1. Ideally, the IAM policy statement should have the encryption context > set > > so that the key can only be used to en/decrypt files under the specific > > table path. > > 2. The structure suggests that all tables under the catalog will use the > > same KMS key for encryption. Should we support overriding the key at the > > table level? The S3FileIOProperties class in Iceberg defines the > s3.sse.key > > property at the table-level so each table can specify its own KMS key. > Can > > we allow for that override, if present? > > > > Mike > > > > On Mon, Oct 13, 2025 at 7:27 AM Jean-Baptiste Onofré <[email protected]> > > wrote: > > > > > Hi Fabio > > > > > > Thanks for the PR ! We will take a look on it. > > > > > > Regards > > > JB > > > > > > On Mon, Oct 13, 2025 at 9:50 AM Rizzo Cascio, Fabio > > > <[email protected]> wrote: > > > > > > > > Hi guys, > > > > > > > > I have created a new PR to be able to use a kms key for the S3 > bucket, > > > it is mandatory for me to use any S3 storage and hopefully a good > > addition > > > for other people that want to use it. > > > > > > > > PR link: https://github.com/apache/polaris/pull/2802 > > > > > > > > Thanks > > > > > > > > Fabio > > > > > > > > This message is confidential and subject to terms at: > > > https://www.jpmorgan.com/emaildisclaimer including on confidential, > > > privileged or legal entity information, malicious content and > monitoring > > of > > > electronic messages. If you are not the intended recipient, please > delete > > > this message and notify the sender immediately. Any unauthorized use is > > > strictly prohibited. > > > > > > > This message is confidential and subject to terms at: > > https://www.jpmorgan.com/emaildisclaimer including on confidential, > > privileged or legal entity information, malicious content and monitoring > of > > electronic messages. If you are not the intended recipient, please delete > > this message and notify the sender immediately. Any unauthorized use is > > strictly prohibited. > > >
