aarushigupta132 opened a new pull request, #2701:
URL: https://github.com/apache/iceberg-rust/pull/2701
## Which issue does this PR close?
- Working towards: #2034
## What changes are included in this PR?
Adds an opt-in encryption seam to `ParquetWriterBuilder` so Iceberg can write
encrypted data files using **Parquet Modular Encryption (PME)** — not stream
encryption. There is a single plaintext writer; whether a file is encrypted
is
decided by the builder's configuration, not by a separate `build` method.
- New `FileEncryptionHandler` trait
(`crates/iceberg/src/encryption/handler.rs`):
the write-side counterpart of the read path's key resolution. It produces
the
per-file `StandardKeyMetadata` (fresh DEK + AAD prefix). `async` so schemes
that mint/wrap the DEK via a KMS can plug in later.
- `StandardFileEncryptionHandler` default impl, and `EncryptionManager` also
implements the trait (generating a fresh DEK + AAD prefix per file; the
KMS/KEK
envelope work stays one tier up at the manifest-list layer).
- `ParquetWriterBuilder::with_file_encryption_handler(...)`: when set, the
writer
builds `FileEncryptionProperties` (uniform encryption: DEK as footer key,
AAD
prefix as file AAD) and stamps the encoded `StandardKeyMetadata` onto
`DataFile::key_metadata` so readers (#2584) can decrypt.
- Validates the DEK is a usable AES length (16/24/32 bytes) before building
the
encryption properties, so a bad handler surfaces a clear `iceberg::Error`
instead of an opaque arrow-rs failure (mirrors the read-path check).
This is the data-file write counterpart to the read path in #2584 and the
manifest-list work in #2677.
## Are these changes tested?
- `test_parquet_writer_encrypted_round_trip`: writes an encrypted parquet
file
via `with_file_encryption_handler`, reads it back with
`FileDecryptionProperties` built from the emitted `key_metadata`, and
asserts
the rows match.
- `test_standard_handler_emits_distinct_keys`: each file gets a fresh DEK and
AAD prefix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
