rambleraptor opened a new issue, #1289: URL: https://github.com/apache/iceberg-go/issues/1289
### Feature Request / Improvement iceberg-go will read the `encryption-keys` metadata, but it doesn't support encrypting / decrypting data using that metadata. I think the biggest part of this will be figuring out the proper interfaces. There will many different implementations of these interfaces over time. I'd love to hear everyone's idea on the basic design here: # Interfaces - [ ] Encryption interfaces + Plaintext manager — EncryptionManager, EncryptedInputFile/EncryptedOutputFile, EncryptionKeyMetadata, NativeEncryption*, and the no-op PlaintextEncryptionManager. - [ ] KMS client interface — KeyManagementClient (wrapKey/unwrapKey/supportsKeyGeneration/generateKey), plus a test/in-memory impl and encryption.kms-type/encryption.kms-impl catalog-property wiring. - [ ] StandardEncryptionManager — envelope KEK/DEK logic, local DEK generation, key cache, rotation/timestamp tracking, encryption.key-id + encryption.data-key-length table properties. Consumes the existing encryption-keys metadata array. # Integration - [ ] EncryptingFileIO wrapper — slot encryption into the io.IO layer: decrypt on Open of Data/Manifest/ManifestList/Statistics files based on key_metadata, encrypt on write. - [ ] Parquet native encryption for data/delete files — wire FileEncryptionProperties/FileDecryptionProperties from NativeEncryptionOutputFile into table/internal/parquet_files.go (write) and the scan read path. - [ ] Manifest + manifest-list encryption — whole-file AES-GCM using the existing key_metadata fields, including the manifest-list double-encryption (KEK + timestamp AAD) on the write/read planners. - [ ] Puffin blob / statistics-file encryption — encrypt DV and stats blobs; populate StatisticsFile.KeyMetadata. # Testing - [ ] Format-version compatibility + end-to-end + cross-client tests — reject encryption props on v1, gate behavior, and round-trip a Java-produced encrypted table end-to-end (the equivalent of the DV cross-client compliance PR #1131). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
