xanderbailey opened a new pull request, #2286: URL: https://github.com/apache/iceberg-rust/pull/2286
## Which issue does this PR close? Part of https://github.com/apache/iceberg-rust/issues/2034 ## What changes are included in this PR? ### Summary This PR adds AGS1 stream encryption/decryption support for Iceberg, building on the core crypto primitives merged in #2026. It implements the block-based AES-GCM stream format used by Iceberg for encrypting manifest lists and manifest files, byte-compatible with Java's `AesGcmInputStream` / `AesGcmOutputStream`. ### Motivation The previous PR provided low-level `AesGcmCipher` encrypt/decrypt operations. This PR layers the AGS1 streaming format on top, enabling encryption of arbitrarily large files with random-access read support, this is a prerequisite for encrypting Iceberg metadata and data files. ### Changes **New Module: `encryption/stream.rs`** — AGS1 stream format implementation - `AesGcmFileRead`: Implements `FileRead` for transparent random-access decryption. Maps plaintext byte ranges to encrypted blocks, reads and decrypts them in a single I/O call, and returns the requested plaintext slice. - `AesGcmFileWrite`: Implements `FileWrite` for transparent streaming encryption. Buffers plaintext, emits encrypted AGS1 blocks when full, and finalizes on close. - `stream_block_aad()`: Constructs per-block AAD matching Java's `Ciphers.streamBlockAAD()`. - Constants: `PLAIN_BLOCK_SIZE` (1 MiB), `CIPHER_BLOCK_SIZE`, `GCM_STREAM_MAGIC`, etc. **New Module: `encryption/file_decryptor.rs`** — File-level decryption helper - `AesGcmFileDecryptor`: Holds decryption material (DEK + AAD prefix) and wraps a `FileRead` for transparent decryption. **New Module: `encryption/file_encryptor.rs`** — File-level encryption helper - `AesGcmFileEncryptor`: Write-side counterpart; wraps a `FileWrite` for transparent encryption. **AGS1 File Format:** ```text [Header: 8 bytes] "AGS1" magic (4 bytes) + plain block size u32 LE (4 bytes) [Block 0..N] Nonce (12 bytes) + Ciphertext (up to 1 MiB) + GCM Tag (16 bytes) ``` Each block's AAD: `aad_prefix || block_index (4 bytes, LE)` ### Java Compatibility | Java Class | Rust Implementation | |---|---| | `AesGcmInputStream` | `AesGcmFileRead` | | `AesGcmOutputStream` | `AesGcmFileWrite` | | `Ciphers.streamBlockAAD()` | `stream_block_aad()` | | `Ciphers.PLAIN_BLOCK_SIZE` | `PLAIN_BLOCK_SIZE` | ### Future Work Upcoming PRs will add: 1. Key management interfaces (`KeyManagementClient` trait) 2. `EncryptionManager` implementation 3. Key metadata serialization 4. Integration with `InputFile` / `OutputFile` ## Are these changes tested? Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
