I feel like there’s something I’m overlooking. Setting the kms property at the catalog level works, but it doesn’t support the s3 iceberg properties, right?
To support those, the best approach is to let clients use the s3 iceberg properties directly. Then, in AwsCredentialsStorageIntegration, we can simply add a policy that allows any key within that account. This way, we don’t need to rely on metadata properties to retrieve the key value. Does that make sense? Thanks Fabio From: Dmitri Bourlatchkov <[email protected]> Date: Wednesday, 22 October 2025 at 15:42 To: [email protected] <[email protected]> Subject: Re: [EXTERNAL]Re: KMS Key addition for s3 Hi Fabio, Yes, that would be the preferable solution for KMS from my POV. I wonder what other people think about this too :) Cheers, Dmitri. On Wed, Oct 22, 2025 at 9:45 AM Rizzo Cascio, Fabio <[email protected]> wrote: > Hi Dmitri, > > If I add the kms props to AccessConfig in AWSCredentialStorage I can see > it in the LoadTable response coming from Polaris. > > Fabio > > From: Dmitri Bourlatchkov <[email protected]> > Date: Tuesday, 21 October 2025 at 15:23 > To: [email protected] <[email protected]> > Subject: Re: [EXTERNAL]Re: KMS Key addition for s3 > > Hi Fabio, > > Yes, I glimpsed that from your email. Sorry if my post caused confusion. I > just wanted to reply to the top email as what I'm proposing seems to be a > key feature for KMS support. > > Would you be able to validate whether sending KMS FileIO properties to > clients from LoadTable responses work in practice (e.g. in Spark)? > > I believe this can be done by adding KMS properties as "extra" properties > to AccessConfig. > > Thanks, > Dmitri. > > On Tue, Oct 21, 2025 at 4:15 AM Rizzo Cascio, Fabio > <[email protected]> wrote: > > > Hi Dmitri, > > > > This is what I was saying in my other email. Anyway I’m gonna update my > PR > > with the changes I have made to get it working, the project won’t build > > because I haven’t update the tests etc, I just want to show my changes > and > > see if we can agree on a direction before I make all the changes. > > > > Thanks > > > > Fabio > > > > > > > > > > From: Dmitri Bourlatchkov <[email protected]> > > Date: Monday, 20 October 2025 at 17:38 > > To: [email protected] <[email protected]> > > Subject: [EXTERNAL]Re: KMS Key addition for s3 > > > > Hi Fabio, Ashok and All, > > > > Apologies if I'm missing something obvious, but the two WIP KMS PRs > [1424] > > [2802] appear to be dealing only with AWS policies on the vended > credential > > session. They do not appear to deal with client configuration (in > LoadTable > > responses). > > > > As far as I understand, Iceberg clients need certain FileIO properties to > > be set in order to utilize KMS. > > > > I'd imagine that Polaris ought to provide these FileIO properties in > > LoadTable responses in addition to granting privileges for KMS access to > > the vended (session) credentials. > > > > In other words, the decision whether to use KMS rests with Polaris (we > can > > discuss how to configure that). If that is enabled, clients should not > need > > any extra configuration, they should get complete and usable > > configuration + credentials from Polaris. > > > > WDYT? > > > > [1424] https://github.com/apache/polaris/pull/1424 > > [2802] https://github.com/apache/polaris/pull/2802 > > > > Thanks, > > Dmitri. > > > > > > On Mon, Oct 13, 2025 at 3:50 AM Rizzo Cascio, Fabio > > <[email protected]> wrote: > > > > > Hi guys, > > > > > > I have created a new PR to be able to use a kms key for the S3 bucket, > it > > > is mandatory for me to use any S3 storage and hopefully a good addition > > for > > > other people that want to use it. > > > > > > PR link: https://github.com/apache/polaris/pull/2802 > > > > > > Thanks > > > > > > Fabio > > > > > > This message is confidential and subject to terms at: > > > https://www.jpmorgan.com/emaildisclaimer including on confidential, > > > privileged or legal entity information, malicious content and > monitoring > > of > > > electronic messages. If you are not the intended recipient, please > delete > > > this message and notify the sender immediately. Any unauthorized use is > > > strictly prohibited. > > > > > > > This message is confidential and subject to terms at: > > https://www.jpmorgan.com/emaildisclaimer including on confidential, > > privileged or legal entity information, malicious content and monitoring > of > > electronic messages. If you are not the intended recipient, please delete > > this message and notify the sender immediately. Any unauthorized use is > > strictly prohibited. > > > > This message is confidential and subject to terms at: > https://www.jpmorgan.com/emaildisclaimer including on confidential, > privileged or legal entity information, malicious content and monitoring of > electronic messages. If you are not the intended recipient, please delete > this message and notify the sender immediately. Any unauthorized use is > strictly prohibited. > This message is confidential and subject to terms at: https://www.jpmorgan.com/emaildisclaimer including on confidential, privileged or legal entity information, malicious content and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
