xanderbailey opened a new pull request, #2026:
URL: https://github.com/apache/iceberg-rust/pull/2026
Add Core Encryption Primitives for Iceberg Encryption Support
## Summary
This PR introduces the foundational cryptographic primitives needed for
implementing encryption in iceberg-rust, providing AES-GCM encryption
operations that match the Java implementation's behavior and data format.
## Motivation
Iceberg's Java implementation supports table-level encryption to protect
sensitive data at rest. To achieve feature parity and ensure interoperability
between Java and Rust implementations, we need to build encryption support from
the ground up. This PR provides the core cryptographic operations that will
serve as the foundation for the complete encryption feature.
## Changes
New Module: encryption
Added a new encryption module with core AES-GCM cryptographic operations:
- encryption/crypto.rs - Core encryption implementation
- EncryptionAlgorithm enum supporting AES-128-GCM and AES-256-GCM
- SecureKey struct with automatic memory zeroization for security
- AesGcmEncryptor providing encrypt/decrypt operations with AAD support
Key Features
1. Java-Compatible Format: Ciphertext format matches Java's implementation
exactly:
[12-byte nonce][encrypted data][16-byte GCM authentication tag]
1. This ensures files encrypted by Java can be decrypted by Rust and vice
versa.
2. Secure Key Handling: Uses the zeroize crate to automatically clear
encryption keys from memory when dropped, preventing key material from
lingering in memory.
3. Additional Authenticated Data (AAD): Full support for AAD to ensure
integrity of associated metadata that isn't encrypted.
4. Comprehensive Testing: 8 tests covering:
- Round-trip encryption/decryption for both AES-128 and AES-256
- AAD validation
- Empty plaintext handling
- Tamper detection
- Format compatibility verification
Dependencies Added
- aes-gcm = "0.10" - Industry-standard AES-GCM implementation
- zeroize = "1.7" - Secure memory cleanup for encryption keys
Compatibility
This implementation directly corresponds to Java's
https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/encryption/Ciphers.java:
| Java Class | Rust Implementation |
|-----------------------------|------------------------------------------|
| Ciphers.AesGcmEncryptor | AesGcmEncryptor::encrypt() |
| Ciphers.AesGcmDecryptor | AesGcmEncryptor::decrypt() |
| EncryptionAlgorithm.AES_GCM | EncryptionAlgorithm::Aes128Gcm/Aes256Gcm |
Testing
Future Work
This PR is the first in a series to implement full encryption support.
Upcoming PRs will add:
1. Table properties for encryption configuration
2. Key management interfaces (KeyManagementClient trait)
3. EncryptionManager implementation
4. Native Parquet encryption integration
5. AWS KMS support
6. Integration with Table and FileIO
Review Notes
- This PR is intentionally minimal and self-contained
- No existing code paths are modified - this is purely additive
- The module is public but won't be used until future PRs wire it up
- Format compatibility with Java has been prioritized to ensure
interoperability
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases. You can
link an issue to this PR using the GitHub syntax. For example `Closes #123`
indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Yes
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example, are
they covered by existing tests)?
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
