ggershinsky commented on pull request #2638: URL: https://github.com/apache/iceberg/pull/2638#issuecomment-852118456
I've dug in the code a bit, there seems to be a practical solution to this. The workers (data/delete writers) can generate random DEKs for each file - like we do today in PME - and pack them in the `ContentFile` objects for the delivery to the driver (manifest writer). The manifest writer will interact with a KMS to wrap the DEKs (per the single/double wrap design), and will store the result in the `key_material` field. I'll change the PR accordingly. As for the "native"-vs-general encryption - there are some interesting trade-offs. Both subjects TBD. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
