Hi everyone, Based on the conversation in https://lists.apache.org/thread/4zm5jhwp42z10pfz2sths7q59r9dlf8l and on the PR https://github.com/apache/iceberg/pull/16527, I’d like to raise a vote to add a new encryption spec which documents the binary formats used by the standard encryption scheme.
Today the spec describes the key_metadata field in manifest entries and the encrypted-key-metadata field in table metadata as “implementation-specific.” In practice these formats are fixed by the Java implementation, and any other implementation has to reverse-engineer them from the Java source to read or write encrypted tables. This came out of implementing table encryption in iceberg-rust. The PR adds a new encryption spec document covering: - The standard key metadata format: a version byte followed by an Avro binary-encoded record holding the DEK, the AAD prefix, and the file length. - The two-tier key hierarchy in the encryption-keys list: KEK entries wrapped by the KMS, and manifest list key entries encrypted by a KEK. This documents existing behaviour rather than changing it, so no implementation needs to change. Vote [ ] +1 Adopt the spec change [ ] 0 No opinion [ ] -1 Do not adopt, because… Thanks, Xander
