This is an automated email from the ASF dual-hosted git repository.
liurenjie1024 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg-rust.git
The following commit(s) were added to refs/heads/main by this push:
new fea5906cb feat(encryption) [1/N] Support encryption: Add crypto for
AES-GCM (#2026)
fea5906cb is described below
commit fea5906cbceead1f6993c408d50bb2b18eea7cf0
Author: Xander <[email protected]>
AuthorDate: Wed Mar 25 00:58:29 2026 +0000
feat(encryption) [1/N] Support encryption: Add crypto for AES-GCM (#2026)
Add Core Encryption Primitives for Iceberg Encryption Support.
Part of https://github.com/apache/iceberg-rust/issues/2034
## Summary
This PR introduces the foundational cryptographic primitives needed for
implementing encryption in iceberg-rust, providing AES-GCM encryption
operations that match the Java implementation's behavior and data
format.
## Motivation
Iceberg's Java implementation supports table-level encryption to protect
sensitive data at rest. To achieve feature parity and ensure
interoperability between Java and Rust implementations, we need to build
encryption support from the ground up. This PR provides the core
cryptographic operations that will serve as the foundation for the
complete encryption feature.
## Changes
New Module: encryption
Added a new encryption module with core AES-GCM cryptographic
operations:
- encryption/crypto.rs - Core encryption implementation
- EncryptionAlgorithm enum supporting AES-128-GCM as this is the only
algorithm currently supported in arrow parquet
- SecureKey struct with automatic memory zeroization for security
- AesGcmEncryptor providing encrypt/decrypt operations with AAD support
Key Features
1. Java-Compatible Format: Ciphertext format matches Java's
implementation exactly:
[12-byte nonce][encrypted data][16-byte GCM authentication tag]
1. This ensures files encrypted by Java can be decrypted by Rust and
vice versa.
2. Secure Key Handling: Uses the zeroize crate to automatically clear
encryption keys from memory when dropped, preventing key material from
lingering in memory.
3. Additional Authenticated Data (AAD): Full support for AAD to ensure
integrity of associated metadata that isn't encrypted.
4. Comprehensive Testing: 8 tests covering:
- Round-trip encryption/decryption for both AES-128 and AES-256
- AAD validation
- Empty plaintext handling
- Tamper detection
- Format compatibility verification
Dependencies Added
- aes-gcm = "0.10" - Industry-standard AES-GCM implementation
- zeroize = "1.7" - Secure memory cleanup for encryption keys
Compatibility
This implementation directly corresponds to Java's
https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/encryption/Ciphers.java:
| Java Class | Rust Implementation |
|-----------------------------|------------------------------------------|
| Ciphers.AesGcmEncryptor | AesGcmEncryptor::encrypt() |
| Ciphers.AesGcmDecryptor | AesGcmEncryptor::decrypt() |
| EncryptionAlgorithm.AES_GCM | EncryptionAlgorithm::Aes128Gcm|
Testing
Future Work
This PR is the first in a series to implement full encryption support.
Upcoming PRs will add:
1. Table properties for encryption configuration
2. Key management interfaces (KeyManagementClient trait)
3. EncryptionManager implementation
4. Native Parquet encryption integration
5. AWS KMS support
6. Integration with Table and FileIO
Review Notes
- This PR is intentionally minimal and self-contained
- No existing code paths are modified - this is purely additive
- The module is public but won't be used until future PRs wire it up
- Format compatibility with Java has been prioritized to ensure
interoperability
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #. https://github.com/apache/iceberg-rust/issues/2035
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Yes
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
---
Cargo.lock | 356 ++++++++++++++--------
Cargo.toml | 3 +
crates/iceberg/Cargo.toml | 2 +
crates/iceberg/src/encryption/crypto.rs | 523 ++++++++++++++++++++++++++++++++
crates/iceberg/src/encryption/mod.rs | 25 ++
crates/iceberg/src/lib.rs | 1 +
6 files changed, 788 insertions(+), 122 deletions(-)
diff --git a/Cargo.lock b/Cargo.lock
index 970c4bad0..8171f2838 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -8,6 +8,16 @@ version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index";
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
+[[package]]
+name = "aead"
+version = "0.5.2"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "d122413f284cf2d62fb1b7db97e02edb8cda96d769b16e443a4f6195e35662b0"
+dependencies = [
+ "crypto-common",
+ "generic-array",
+]
+
[[package]]
name = "aes"
version = "0.8.4"
@@ -19,6 +29,20 @@ dependencies = [
"cpufeatures",
]
+[[package]]
+name = "aes-gcm"
+version = "0.10.3"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "831010a0f742e1209b3bcea8fab6a8e149051ba6099432c8cb2cc117dec3ead1"
+dependencies = [
+ "aead",
+ "aes",
+ "cipher",
+ "ctr",
+ "ghash",
+ "subtle",
+]
+
[[package]]
name = "ahash"
version = "0.8.12"
@@ -80,7 +104,22 @@ source =
"registry+https://github.com/rust-lang/crates.io-index";
checksum = "43d5b281e737544384e969a5ccad3f1cdd24b48086a0fc1b2a5262a26b8f4f4a"
dependencies = [
"anstyle",
- "anstyle-parse",
+ "anstyle-parse 0.2.7",
+ "anstyle-query",
+ "anstyle-wincon",
+ "colorchoice",
+ "is_terminal_polyfill",
+ "utf8parse",
+]
+
+[[package]]
+name = "anstream"
+version = "1.0.0"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
+dependencies = [
+ "anstyle",
+ "anstyle-parse 1.0.0",
"anstyle-query",
"anstyle-wincon",
"colorchoice",
@@ -90,9 +129,9 @@ dependencies = [
[[package]]
name = "anstyle"
-version = "1.0.13"
+version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "5192cca8006f1fd4f7237516f40fa183bb07f8fbdfedaa0036de5ea9b0b45e78"
+checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
[[package]]
name = "anstyle-parse"
@@ -103,6 +142,15 @@ dependencies = [
"utf8parse",
]
+[[package]]
+name = "anstyle-parse"
+version = "1.0.0"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
+dependencies = [
+ "utf8parse",
+]
+
[[package]]
name = "anstyle-query"
version = "1.1.5"
@@ -504,9 +552,9 @@ checksum =
"c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "aws-config"
-version = "1.8.14"
+version = "1.8.15"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "8a8fc176d53d6fe85017f230405e3255cedb4a02221cb55ed6d76dccbbb099b2"
+checksum = "11493b0bad143270fb8ad284a096dd529ba91924c5409adeac856cc1bf047dbc"
dependencies = [
"aws-credential-types",
"aws-runtime",
@@ -524,7 +572,7 @@ dependencies = [
"fastrand",
"hex",
"http 1.4.0",
- "ring",
+ "sha1",
"time",
"tokio",
"tracing",
@@ -534,9 +582,9 @@ dependencies = [
[[package]]
name = "aws-credential-types"
-version = "1.2.13"
+version = "1.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "6d203b0bf2626dcba8665f5cd0871d7c2c0930223d6b6be9097592fea21242d0"
+checksum = "8f20799b373a1be121fe3005fba0c2090af9411573878f224df44b42727fcaf7"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -568,9 +616,9 @@ dependencies = [
[[package]]
name = "aws-runtime"
-version = "1.7.1"
+version = "1.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "ede2ddc593e6c8acc6ce3358c28d6677a6dc49b65ba4b37a2befe14a11297e75"
+checksum = "5fc0651c57e384202e47153c1260b84a9936e19803d747615edf199dc3b98d17"
dependencies = [
"aws-credential-types",
"aws-sigv4",
@@ -593,9 +641,9 @@ dependencies = [
[[package]]
name = "aws-sdk-glue"
-version = "1.139.0"
+version = "1.142.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "af3da2f5cf74983a60a7d5a182d76db1609ee4401057c98732ed8be973cb30ee"
+checksum = "3962675ec1f2012ae6439814e784557550fa239a4a291bd4f33d8f514d4fdb5b"
dependencies = [
"aws-credential-types",
"aws-runtime",
@@ -617,9 +665,9 @@ dependencies = [
[[package]]
name = "aws-sdk-s3tables"
-version = "1.51.0"
+version = "1.53.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "14c7f1b4eb404522622f5489fc649ba193c1e3ce4416bfcfbbcb008ad0cbfe4f"
+checksum = "c91febb29f5287a7b723dbacca6d81b1086b8ac0af6b35b873539ee19c74827f"
dependencies = [
"aws-credential-types",
"aws-runtime",
@@ -641,9 +689,9 @@ dependencies = [
[[package]]
name = "aws-sdk-sso"
-version = "1.95.0"
+version = "1.97.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "00c5ff27c6ba2cbd95e6e26e2e736676fdf6bcf96495b187733f521cfe4ce448"
+checksum = "9aadc669e184501caaa6beafb28c6267fc1baef0810fb58f9b205485ca3f2567"
dependencies = [
"aws-credential-types",
"aws-runtime",
@@ -665,9 +713,9 @@ dependencies = [
[[package]]
name = "aws-sdk-ssooidc"
-version = "1.97.0"
+version = "1.99.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "4d186f1e5a3694a188e5a0640b3115ccc6e084d104e16fd6ba968dca072ffef8"
+checksum = "1342a7db8f358d3de0aed2007a0b54e875458e39848d54cc1d46700b2bfcb0a8"
dependencies = [
"aws-credential-types",
"aws-runtime",
@@ -689,9 +737,9 @@ dependencies = [
[[package]]
name = "aws-sdk-sts"
-version = "1.99.0"
+version = "1.101.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "9acba7c62f3d4e2408fa998a3a8caacd8b9a5b5549cf36e2372fbdae329d5449"
+checksum = "ab41ad64e4051ecabeea802d6a17845a91e83287e1dd249e6963ea1ba78c428a"
dependencies = [
"aws-credential-types",
"aws-runtime",
@@ -714,9 +762,9 @@ dependencies = [
[[package]]
name = "aws-sigv4"
-version = "1.4.1"
+version = "1.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "37411f8e0f4bea0c3ca0958ce7f18f6439db24d555dbd809787262cd00926aa9"
+checksum = "b0b660013a6683ab23797778e21f1f854744fdf05f68204b4cca4c8c04b5d1f4"
dependencies = [
"aws-credential-types",
"aws-smithy-http",
@@ -862,9 +910,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types"
-version = "1.4.6"
+version = "1.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "d2b1117b3b2bbe166d11199b540ceed0d0f7676e36e7b962b5a437a9971eac75"
+checksum = "9d73dbfbaa8e4bc57b9045137680b958d274823509a360abfd8e1d514d40c95c"
dependencies = [
"base64-simd",
"bytes",
@@ -897,9 +945,9 @@ dependencies = [
[[package]]
name = "aws-types"
-version = "1.3.13"
+version = "1.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "0470cc047657c6e286346bdf10a8719d26efd6a91626992e0e64481e44323e96"
+checksum = "47c8323699dd9b3c8d5b3c13051ae9cdef58fd179957c882f8374dd8725962d9"
dependencies = [
"aws-credential-types",
"aws-smithy-async",
@@ -1024,9 +1072,9 @@ dependencies = [
[[package]]
name = "bon"
-version = "3.9.0"
+version = "3.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "2d13a61f2963b88eef9c1be03df65d42f6996dfeac1054870d950fcf66686f83"
+checksum = "f47dbe92550676ee653353c310dfb9cf6ba17ee70396e1f7cf0a2020ad49b2fe"
dependencies = [
"bon-macros",
"rustversion",
@@ -1034,9 +1082,9 @@ dependencies = [
[[package]]
name = "bon-macros"
-version = "3.9.0"
+version = "3.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "d314cc62af2b6b0c65780555abb4d02a03dd3b799cd42419044f0c38d99738c0"
+checksum = "519bd3116aeeb42d5372c29d982d16d0170d3d4a5ed85fc7dd91642ffff3c67c"
dependencies = [
"darling 0.23.0",
"ident_case",
@@ -1125,9 +1173,9 @@ dependencies = [
[[package]]
name = "cc"
-version = "1.2.56"
+version = "1.2.57"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "aebf35691d1bfb0ac386a69bac2fde4dd276fb618cf8bf4f5318fe285e821bb2"
+checksum = "7a0dd1ca384932ff3641c8718a02769f1698e7563dc6974ffd03346116310423"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -1183,9 +1231,9 @@ dependencies = [
[[package]]
name = "clap"
-version = "4.5.60"
+version = "4.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "2797f34da339ce31042b27d23607e051786132987f595b02ba4f6a6dffb7030a"
+checksum = "b193af5b67834b676abd72466a96c1024e6a6ad978a1f484bd90b85c94041351"
dependencies = [
"clap_builder",
"clap_derive",
@@ -1193,11 +1241,11 @@ dependencies = [
[[package]]
name = "clap_builder"
-version = "4.5.60"
+version = "4.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "24a241312cea5059b13574bb9b3861cabf758b879c15190b37b6d6fd63ab6876"
+checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f"
dependencies = [
- "anstream",
+ "anstream 1.0.0",
"anstyle",
"clap_lex",
"strsim",
@@ -1205,9 +1253,9 @@ dependencies = [
[[package]]
name = "clap_derive"
-version = "4.5.55"
+version = "4.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "a92793da1a46a5f2a02a6f4c46c6496b28c43638adea8306fcb0caa1634f24e5"
+checksum = "1110bd8a634a1ab8cb04345d8d878267d57c3cf1b38d91b71af6686408bbca6a"
dependencies = [
"heck",
"proc-macro2",
@@ -1217,9 +1265,9 @@ dependencies = [
[[package]]
name = "clap_lex"
-version = "1.0.0"
+version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "3a822ea5bc7590f9d40f1ba12c0dc3c2760f3482c6984db1573ad11031420831"
+checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
[[package]]
name = "clipboard-win"
@@ -1241,9 +1289,9 @@ dependencies = [
[[package]]
name = "colorchoice"
-version = "1.0.4"
+version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "b05b61dc5112cbb17e4b6cd61790d9845d13888356391624cbe7e41efeac1e75"
+checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "colored"
@@ -1296,13 +1344,12 @@ dependencies = [
[[package]]
name = "console"
-version = "0.16.2"
+version = "0.16.3"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "03e45a4a8926227e4197636ba97a9fc9b00477e9f4bd711395687c5f0734bec4"
+checksum = "d64e8af5551369d19cf50138de61f1c42074ab970f74e99be916646777f8fc87"
dependencies = [
"encode_unicode",
"libc",
- "once_cell",
"unicode-width 0.2.2",
"windows-sys 0.61.2",
]
@@ -1443,6 +1490,7 @@ source =
"registry+https://github.com/rust-lang/crates.io-index";
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
+ "rand_core 0.6.4",
"typenum",
]
@@ -1467,6 +1515,15 @@ dependencies = [
"memchr",
]
+[[package]]
+name = "ctr"
+version = "0.9.2"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "0369ee1ad671834580515889b80f2ea915f23b8be8d0daa4bbaf2ac5c7590835"
+dependencies = [
+ "cipher",
+]
+
[[package]]
name = "darling"
version = "0.20.11"
@@ -1656,9 +1713,9 @@ dependencies = [
[[package]]
name = "datafusion-cli"
-version = "52.3.0"
+version = "52.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "d6cc57c2a8889e722be7913bb3c053c554f23abafa2e99005ad6fe84c765f7ce"
+checksum = "46a0b3ed9bfda5f234c62e179bbc1258fc89452a89cd3d652da73efcb994ecf5"
dependencies = [
"arrow",
"async-trait",
@@ -2239,9 +2296,9 @@ dependencies = [
[[package]]
name = "datafusion-spark"
-version = "52.3.0"
+version = "52.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "25f2e5519037772210eee5bb87a95dc953e1bd94bc2f9c9d6bb14b0c7fb9ab0a"
+checksum = "8e53604bca77d4544426a425e2a50d7b911bbe35d3c8193de24093b445f23856"
dependencies = [
"arrow",
"bigdecimal",
@@ -2280,9 +2337,9 @@ dependencies = [
[[package]]
name = "datafusion-sqllogictest"
-version = "52.3.0"
+version = "52.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "74e697441492ce35353b07842181f0f92765c5d6ac1daaead4974ecf20058247"
+checksum = "3929b7067193345bc345a5ea5f231cccde36fe58fb055d8caef7247ad7566fd5"
dependencies = [
"arrow",
"async-trait",
@@ -2306,9 +2363,9 @@ dependencies = [
[[package]]
name = "datafusion-substrait"
-version = "52.3.0"
+version = "52.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "fe00df31ca03a167d3e40054120930fe5fb689e66bc625b602fac7153b222aea"
+checksum = "2379388ecab67079eeb1185c953fb9c5ed4b283fa3cb81417538378a30545957"
dependencies = [
"async-recursion",
"async-trait",
@@ -2428,9 +2485,9 @@ dependencies = [
[[package]]
name = "dissimilar"
-version = "1.0.10"
+version = "1.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "8975ffdaa0ef3661bfe02dbdcc06c9f829dfafe6a3c474de366a8d5e44276921"
+checksum = "aeda16ab4059c5fd2a83f2b9c9e9c981327b18aa8e3b313f7e6563799d4f093e"
[[package]]
name = "dlv-list"
@@ -2534,7 +2591,7 @@ version = "0.11.9"
source = "registry+https://github.com/rust-lang/crates.io-index";
checksum = "b2daee4ea451f429a58296525ddf28b45a3b64f1acf6587e2067437bb11e218d"
dependencies = [
- "anstream",
+ "anstream 0.6.21",
"anstyle",
"env_filter",
"jiff",
@@ -2892,24 +2949,34 @@ dependencies = [
"cfg-if",
"js-sys",
"libc",
- "r-efi",
+ "r-efi 5.3.0",
"wasip2",
"wasm-bindgen",
]
[[package]]
name = "getrandom"
-version = "0.4.1"
+version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "139ef39800118c7683f2fd3c98c1b23c09ae076556b435f8e9064ae108aaeeec"
+checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555"
dependencies = [
"cfg-if",
"libc",
- "r-efi",
+ "r-efi 6.0.0",
"wasip2",
"wasip3",
]
+[[package]]
+name = "ghash"
+version = "0.5.1"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "f0d8a4362ccb29cb0b265253fb0a2728f592895ee6854fd9bc13f2ffda266ff1"
+dependencies = [
+ "opaque-debug",
+ "polyval",
+]
+
[[package]]
name = "glob"
version = "0.3.3"
@@ -3184,7 +3251,7 @@ dependencies = [
"libc",
"percent-encoding",
"pin-project-lite",
- "socket2 0.6.2",
+ "socket2 0.6.3",
"tokio",
"tower-service",
"tracing",
@@ -3218,6 +3285,7 @@ dependencies = [
name = "iceberg"
version = "0.9.0"
dependencies = [
+ "aes-gcm",
"anyhow",
"apache-avro",
"array-init",
@@ -3269,6 +3337,7 @@ dependencies = [
"typetag",
"url",
"uuid",
+ "zeroize",
"zstd",
]
@@ -3697,9 +3766,9 @@ checksum =
"d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
[[package]]
name = "iri-string"
-version = "0.7.10"
+version = "0.7.11"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "c91338f0783edbd6195decb37bae672fd3b165faffb89bf7b9e6942f8b1a731a"
+checksum = "d8e7418f59cc01c88316161279a7f665217ae316b388e58a0d10e29f54f1e5eb"
dependencies = [
"memchr",
"serde",
@@ -3731,15 +3800,15 @@ dependencies = [
[[package]]
name = "itoa"
-version = "1.0.17"
+version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "92ecc6618181def0457392ccd0ee51198e065e016d1d527a7ac1b6dc7c1f09d2"
+checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "jiff"
-version = "0.2.22"
+version = "0.2.23"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "819b44bc7c87d9117eb522f14d46e918add69ff12713c475946b0a29363ed1c2"
+checksum = "1a3546dc96b6d42c5f24902af9e2538e82e39ad350b0c766eb3fbf2d8f3d8359"
dependencies = [
"jiff-static",
"jiff-tzdb-platform",
@@ -3752,9 +3821,9 @@ dependencies = [
[[package]]
name = "jiff-static"
-version = "0.2.22"
+version = "0.2.23"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "470252db18ecc35fd766c0891b1e3ec6cbbcd62507e85276c01bf75d8e94d4a1"
+checksum = "2a8c8b344124222efd714b73bb41f8b5120b27a7cc1c75593a6ff768d9d05aa4"
dependencies = [
"proc-macro2",
"quote",
@@ -3763,9 +3832,9 @@ dependencies = [
[[package]]
name = "jiff-tzdb"
-version = "0.1.5"
+version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "68971ebff725b9e2ca27a601c5eb38a4c5d64422c4cbab0c535f248087eda5c2"
+checksum = "c900ef84826f1338a557697dc8fc601df9ca9af4ac137c7fb61d4c6f2dfd3076"
[[package]]
name = "jiff-tzdb-platform"
@@ -3891,9 +3960,9 @@ checksum =
"2c4a545a15244c7d945065b5d392b2d2d7f21526fba56ce51467b06ed445e8f7"
[[package]]
name = "libc"
-version = "0.2.182"
+version = "0.2.183"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "6800badb6cb2082ffd7b6a67e6125bb39f18782f793520caee8cb8846be06112"
+checksum = "b5b646652bf6661599e1da8901b3b9522896f01e736bad5f723fe7a3a27f899d"
[[package]]
name = "liblzma"
@@ -3956,11 +4025,11 @@ dependencies = [
[[package]]
name = "libtest-mimic"
-version = "0.8.1"
+version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "5297962ef19edda4ce33aaa484386e0a5b3d7f2f4e037cbeee00503ef6b29d33"
+checksum = "14e6ba06f0ade6e504aff834d7c34298e5155c6baca353cc6a4aaff2f9fd7f33"
dependencies = [
- "anstream",
+ "anstream 1.0.0",
"anstyle",
"clap",
"escape8259",
@@ -4162,9 +4231,9 @@ dependencies = [
[[package]]
name = "moka"
-version = "0.12.14"
+version = "0.12.15"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "85f8024e1c8e71c778968af91d43700ce1d11b219d127d79fb2934153b82b42b"
+checksum = "957228ad12042ee839f93c8f257b62b4c0ab5eaae1d4fa60de53b27c9d7c5046"
dependencies = [
"async-lock",
"crossbeam-channel",
@@ -4358,9 +4427,9 @@ dependencies = [
[[package]]
name = "num_enum"
-version = "0.7.5"
+version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "b1207a7e20ad57b847bbddc6776b968420d38292bbfe2089accff5e19e82454c"
+checksum = "5d0bca838442ec211fa11de3a8b0e0e8f3a4522575b5c4c06ed722e005036f26"
dependencies = [
"num_enum_derive",
"rustversion",
@@ -4368,9 +4437,9 @@ dependencies = [
[[package]]
name = "num_enum_derive"
-version = "0.7.5"
+version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "ff32365de1b6743cb203b710788263c44a03de03802daf96092f2da4fe6ba4d7"
+checksum = "680998035259dcfcafe653688bf2aa6d3e2dc05e98be6ab46afb089dc84f1df8"
dependencies = [
"proc-macro-crate",
"proc-macro2",
@@ -4426,9 +4495,9 @@ dependencies = [
[[package]]
name = "once_cell"
-version = "1.21.3"
+version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
+checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "once_cell_polyfill"
@@ -4436,6 +4505,12 @@ version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index";
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
+[[package]]
+name = "opaque-debug"
+version = "0.3.1"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
+
[[package]]
name = "opendal"
version = "0.55.0"
@@ -4798,6 +4873,18 @@ version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index";
checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
+[[package]]
+name = "polyval"
+version = "0.6.2"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "9d1fe60d06143b2430aa532c94cfe9e29783047f06c0d7fd359a9a51b729fa25"
+dependencies = [
+ "cfg-if",
+ "cpufeatures",
+ "opaque-debug",
+ "universal-hash",
+]
+
[[package]]
name = "portable-atomic"
version = "1.13.1"
@@ -4806,9 +4893,9 @@ checksum =
"c33a9471896f1c69cecef8d20cbe2f7accd12527ce60845ff44c153bb2a21b49"
[[package]]
name = "portable-atomic-util"
-version = "0.2.5"
+version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "7a9db96d7fa8782dd8c15ce32ffe8680bbd1e978a43bf51a34d39483540495f5"
+checksum = "091397be61a01d4be58e7841595bd4bfedb15f1cd54977d79b8271e94ed799a3"
dependencies = [
"portable-atomic",
]
@@ -4885,11 +4972,11 @@ dependencies = [
[[package]]
name = "proc-macro-crate"
-version = "3.4.0"
+version = "3.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "219cb19e96be00ab2e37d6e299658a0cfa83e52429179969b0f0121b4ac46983"
+checksum = "e67ba7e9b2b56446f1d419b1d807906278ffa1a658a8a5d8a39dcb1f5a78614f"
dependencies = [
- "toml_edit 0.23.10+spec-1.0.0",
+ "toml_edit 0.25.5+spec-1.1.0",
]
[[package]]
@@ -5021,7 +5108,7 @@ dependencies = [
"quinn-udp",
"rustc-hash",
"rustls",
- "socket2 0.6.2",
+ "socket2 0.6.3",
"thiserror 2.0.18",
"tokio",
"tracing",
@@ -5058,16 +5145,16 @@ dependencies = [
"cfg_aliases",
"libc",
"once_cell",
- "socket2 0.6.2",
+ "socket2 0.6.3",
"tracing",
"windows-sys 0.60.2",
]
[[package]]
name = "quote"
-version = "1.0.44"
+version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
+checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
@@ -5078,6 +5165,12 @@ version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
+[[package]]
+name = "r-efi"
+version = "6.0.0"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
+
[[package]]
name = "radix_trie"
version = "0.2.1"
@@ -5615,9 +5708,9 @@ dependencies = [
[[package]]
name = "schannel"
-version = "0.1.28"
+version = "0.1.29"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "891d81b926048e76efe18581bf793546b4c0eaf8448d72be8de2bbee5fd166e1"
+checksum = "91c1b7e4904c873ef0710c1f407dde2e6287de2bebc1bbbf7d430bb7cbffd939"
dependencies = [
"windows-sys 0.61.2",
]
@@ -6013,12 +6106,12 @@ dependencies = [
[[package]]
name = "socket2"
-version = "0.6.2"
+version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "86f4aa3ad99f2088c990dfa82d367e19cb29268ed67c574d10d0a4bfe71f07e0"
+checksum = "3a766e1110788c36f4fa1c2b71b387a7815aa65f88ce0229841826633d93723e"
dependencies = [
"libc",
- "windows-sys 0.60.2",
+ "windows-sys 0.61.2",
]
[[package]]
@@ -6454,7 +6547,7 @@ source =
"registry+https://github.com/rust-lang/crates.io-index";
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
- "getrandom 0.4.1",
+ "getrandom 0.4.2",
"once_cell",
"rustix",
"windows-sys 0.61.2",
@@ -6578,9 +6671,9 @@ dependencies = [
[[package]]
name = "tinyvec"
-version = "1.10.0"
+version = "1.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "bfa5fdc3bce6191a1dbc8c02d5c8bffcf557bafa17c124c5264a458f1b0613fa"
+checksum = "3e61e67053d25a4e82c844e8424039d9745781b3fc4f32b8d55ed50f5f667ef3"
dependencies = [
"tinyvec_macros",
]
@@ -6603,7 +6696,7 @@ dependencies = [
"parking_lot",
"pin-project-lite",
"signal-hook-registry",
- "socket2 0.6.2",
+ "socket2 0.6.3",
"tokio-macros",
"windows-sys 0.61.2",
]
@@ -6676,9 +6769,9 @@ dependencies = [
[[package]]
name = "toml_datetime"
-version = "0.7.5+spec-1.1.0"
+version = "1.0.1+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "92e1cfed4a3038bc5a127e35a2d360f145e1f4b971b551a2ba5fd7aedf7e1347"
+checksum = "9b320e741db58cac564e26c607d3cc1fdc4a88fd36c879568c07856ed83ff3e9"
dependencies = [
"serde_core",
]
@@ -6694,28 +6787,28 @@ dependencies = [
"serde_spanned",
"toml_datetime 0.6.11",
"toml_write",
- "winnow",
+ "winnow 0.7.15",
]
[[package]]
name = "toml_edit"
-version = "0.23.10+spec-1.0.0"
+version = "0.25.5+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "84c8b9f757e028cee9fa244aea147aab2a9ec09d5325a9b01e0a49730c2b5269"
+checksum = "8ca1a40644a28bce036923f6a431df0b34236949d111cc07cb6dca830c9ef2e1"
dependencies = [
"indexmap 2.13.0",
- "toml_datetime 0.7.5+spec-1.1.0",
+ "toml_datetime 1.0.1+spec-1.1.0",
"toml_parser",
- "winnow",
+ "winnow 1.0.0",
]
[[package]]
name = "toml_parser"
-version = "1.0.9+spec-1.1.0"
+version = "1.0.10+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "702d4415e08923e7e1ef96cd5727c0dfed80b4d2fa25db9647fe5eb6f7c5a4c4"
+checksum = "7df25b4befd31c4816df190124375d5a20c6b6921e2cad937316de3fccd63420"
dependencies = [
- "winnow",
+ "winnow 1.0.0",
]
[[package]]
@@ -6815,9 +6908,9 @@ dependencies = [
[[package]]
name = "tracing-subscriber"
-version = "0.3.22"
+version = "0.3.23"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "2f30143827ddab0d256fd843b7a66d164e9f271cfa0dde49142c5ca0ca291f1e"
+checksum = "cb7f578e5945fb242538965c2d0b04418d38ec25c79d160cd279bf0731c8d319"
dependencies = [
"nu-ansi-term",
"sharded-slab",
@@ -6999,6 +7092,16 @@ version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index";
checksum = "81e544489bf3d8ef66c953931f56617f423cd4b5494be343d9b9d3dda037b9a3"
+[[package]]
+name = "universal-hash"
+version = "0.5.1"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "fc1de2c688dc15305988b563c3854064043356019f97a4b46276fe734c4f07ea"
+dependencies = [
+ "crypto-common",
+ "subtle",
+]
+
[[package]]
name = "unsafe-libyaml"
version = "0.2.11"
@@ -7047,7 +7150,7 @@ version = "1.22.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
checksum = "a68d3c8f01c0cfa54a75291d83601161799e4a89a39e0929f4b0354d88757a37"
dependencies = [
- "getrandom 0.4.1",
+ "getrandom 0.4.2",
"js-sys",
"serde_core",
"wasm-bindgen",
@@ -7637,9 +7740,18 @@ checksum =
"d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650"
[[package]]
name = "winnow"
-version = "0.7.14"
+version = "0.7.15"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
+dependencies = [
+ "memchr",
+]
+
+[[package]]
+name = "winnow"
+version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "5a5364e9d77fcdeeaa6062ced926ee3381faa2ee02d3eb83a5c27a8825540829"
+checksum = "a90e88e4667264a994d34e6d1ab2d26d398dcdca8b7f52bec8668957517fc7d8"
dependencies = [
"memchr",
]
@@ -7775,18 +7887,18 @@ dependencies = [
[[package]]
name = "zerocopy"
-version = "0.8.40"
+version = "0.8.47"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "a789c6e490b576db9f7e6b6d661bcc9799f7c0ac8352f56ea20193b2681532e5"
+checksum = "efbb2a062be311f2ba113ce66f697a4dc589f85e78a4aea276200804cea0ed87"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
-version = "0.8.40"
+version = "0.8.47"
source = "registry+https://github.com/rust-lang/crates.io-index";
-checksum = "f65c489a7071a749c849713807783f70672b28094011623e200cb86dcb835953"
+checksum = "0e8bc7269b54418e7aeeef514aa68f8690b8c0489a06b0136e5f57c4c5ccab89"
dependencies = [
"proc-macro2",
"quote",
diff --git a/Cargo.toml b/Cargo.toml
index eee1e6dc7..1f3eec4ac 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -40,6 +40,8 @@ repository = "https://github.com/apache/iceberg-rust";
rust-version = "1.92"
[workspace.dependencies]
+aes = { version = "0.8", features = ["zeroize"] }
+aes-gcm = "0.10"
anyhow = "1.0.72"
apache-avro = { version = "0.21", features = ["zstandard"] }
array-init = "2"
@@ -134,4 +136,5 @@ url = "2.5.7"
uuid = { version = "1.18", features = ["v7"] }
volo = "0.10.6"
volo-thrift = "0.10.8"
+zeroize = "1.7"
zstd = "0.13.3"
diff --git a/crates/iceberg/Cargo.toml b/crates/iceberg/Cargo.toml
index 41ee77161..aa1d0cd4a 100644
--- a/crates/iceberg/Cargo.toml
+++ b/crates/iceberg/Cargo.toml
@@ -33,6 +33,7 @@ default = []
[dependencies]
+aes-gcm = { workspace = true }
anyhow = { workspace = true }
apache-avro = { workspace = true }
array-init = { workspace = true }
@@ -78,6 +79,7 @@ typed-builder = { workspace = true }
typetag = { workspace = true }
url = { workspace = true }
uuid = { workspace = true }
+zeroize = { workspace = true }
zstd = { workspace = true }
[dev-dependencies]
diff --git a/crates/iceberg/src/encryption/crypto.rs
b/crates/iceberg/src/encryption/crypto.rs
new file mode 100644
index 000000000..0b34580db
--- /dev/null
+++ b/crates/iceberg/src/encryption/crypto.rs
@@ -0,0 +1,523 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Core cryptographic operations for Iceberg encryption.
+
+use std::fmt;
+use std::str::FromStr;
+
+use aes_gcm::aead::generic_array::typenum::U12;
+use aes_gcm::aead::rand_core::RngCore;
+use aes_gcm::aead::{Aead, AeadCore, KeyInit, OsRng, Payload};
+use aes_gcm::{Aes128Gcm, Aes256Gcm, AesGcm, Nonce};
+use zeroize::Zeroizing;
+
+/// AES-192-GCM with 96-bit nonce. Not provided by `aes-gcm` but constructible
+/// from the underlying primitives, same as `Aes128Gcm` and `Aes256Gcm`.
+type Aes192Gcm = AesGcm<aes_gcm::aes::Aes192, U12>;
+
+use crate::{Error, ErrorKind, Result};
+
+/// Wrapper for sensitive byte data (encryption keys, DEKs, etc.) that:
+/// - Zeroizes memory on drop
+/// - Redacts content in [`Debug`] and [`Display`] output
+/// - Provides only `&[u8]` access via [`as_bytes()`](Self::as_bytes)
+/// - Uses `Box<[u8]>` (immutable boxed slice) since key bytes never grow
+///
+/// Use this type for any struct field that holds plaintext key material.
+/// Because its [`Debug`] impl always prints `[N bytes REDACTED]`, structs
+/// containing `SensitiveBytes` can safely derive or implement `Debug`
+/// without risk of leaking key material.
+#[derive(Clone, PartialEq, Eq)]
+struct SensitiveBytes(Zeroizing<Box<[u8]>>);
+
+impl SensitiveBytes {
+ /// Wraps the given bytes as sensitive material.
+ pub fn new(bytes: impl Into<Box<[u8]>>) -> Self {
+ Self(Zeroizing::new(bytes.into()))
+ }
+
+ /// Returns the underlying bytes.
+ pub fn as_bytes(&self) -> &[u8] {
+ &self.0
+ }
+
+ /// Returns the number of bytes.
+ #[allow(dead_code)] // Encryption work is ongoing so currently unused
+ pub fn len(&self) -> usize {
+ self.0.len()
+ }
+
+ /// Returns `true` if the byte slice is empty.
+ #[allow(dead_code)] // Encryption work is ongoing so currently unused
+ pub fn is_empty(&self) -> bool {
+ self.0.is_empty()
+ }
+}
+
+impl fmt::Debug for SensitiveBytes {
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+ write!(f, "[{} bytes REDACTED]", self.0.len())
+ }
+}
+
+impl fmt::Display for SensitiveBytes {
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+ write!(f, "[{} bytes REDACTED]", self.0.len())
+ }
+}
+
+/// Supported AES key sizes for AES-GCM encryption.
+///
+/// The Iceberg spec supports 128, 192, and 256-bit keys for AES-GCM.
+/// See: <https://iceberg.apache.org/gcm-stream-spec/#goals>
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum AesKeySize {
+ /// 128-bit AES key (16 bytes)
+ Bits128 = 128,
+ /// 192-bit AES key (24 bytes)
+ Bits192 = 192,
+ /// 256-bit AES key (32 bytes)
+ Bits256 = 256,
+}
+
+impl AesKeySize {
+ /// Returns the key length in bytes for this key size.
+ pub fn key_length(&self) -> usize {
+ match self {
+ Self::Bits128 => 16,
+ Self::Bits192 => 24,
+ Self::Bits256 => 32,
+ }
+ }
+
+ /// Returns the key size for a given DEK length in bytes.
+ ///
+ /// Matches Java's `encryption.data-key-length` property semantics:
+ /// 16 → 128-bit, 24 → 192-bit, 32 → 256-bit.
+ pub fn from_key_length(len: usize) -> Result<Self> {
+ match len {
+ 16 => Ok(Self::Bits128),
+ 24 => Ok(Self::Bits192),
+ 32 => Ok(Self::Bits256),
+ _ => Err(Error::new(
+ ErrorKind::FeatureUnsupported,
+ format!("Unsupported data key length: {len} (must be 16, 24,
or 32)"),
+ )),
+ }
+ }
+}
+
+impl FromStr for AesKeySize {
+ type Err = Error;
+
+ fn from_str(s: &str) -> Result<Self> {
+ match s {
+ "128" | "AES_GCM_128" | "AES128_GCM" => Ok(Self::Bits128),
+ "192" | "AES_GCM_192" | "AES192_GCM" => Ok(Self::Bits192),
+ "256" | "AES_GCM_256" | "AES256_GCM" => Ok(Self::Bits256),
+ _ => Err(Error::new(
+ ErrorKind::FeatureUnsupported,
+ format!("Unsupported AES key size: {s}"),
+ )),
+ }
+ }
+}
+
+/// A secure encryption key that zeroes its memory on drop.
+pub struct SecureKey {
+ key: SensitiveBytes,
+ key_size: AesKeySize,
+}
+
+impl SecureKey {
+ /// Creates a new secure key with the specified key size.
+ ///
+ /// # Errors
+ /// Returns an error if the key length doesn't match the key size
requirements.
+ pub fn new(key: &[u8]) -> Result<Self> {
+ let key_size = AesKeySize::from_key_length(key.len())?;
+ Ok(Self {
+ key: SensitiveBytes::new(key),
+ key_size,
+ })
+ }
+
+ /// Generates a new random key for the specified key size.
+ pub fn generate(key_size: AesKeySize) -> Self {
+ let mut key = vec![0u8; key_size.key_length()];
+ OsRng.fill_bytes(&mut key);
+ Self {
+ key: SensitiveBytes::new(key),
+ key_size,
+ }
+ }
+
+ /// Returns the AES key size.
+ pub fn key_size(&self) -> AesKeySize {
+ self.key_size
+ }
+
+ /// Returns the key bytes.
+ pub fn as_bytes(&self) -> &[u8] {
+ self.key.as_bytes()
+ }
+}
+
+/// AES-GCM cipher for encrypting and decrypting data.
+pub struct AesGcmCipher {
+ key: SensitiveBytes,
+ key_size: AesKeySize,
+}
+
+impl AesGcmCipher {
+ /// AES-GCM nonce length in bytes (96 bits).
+ pub const NONCE_LEN: usize = 12;
+ /// AES-GCM authentication tag length in bytes (128 bits).
+ pub const TAG_LEN: usize = 16;
+
+ /// Creates a new cipher with the specified key.
+ pub fn new(key: SecureKey) -> Self {
+ Self {
+ key: SensitiveBytes::new(key.as_bytes()),
+ key_size: key.key_size(),
+ }
+ }
+
+ /// Encrypts data using AES-GCM.
+ ///
+ /// # Arguments
+ /// * `plaintext` - The data to encrypt
+ /// * `aad` - Additional authenticated data (optional)
+ ///
+ /// # Returns
+ /// The encrypted data in the format: [12-byte nonce][ciphertext][16-byte
auth tag]
+ /// This matches the Java implementation format for compatibility.
+ pub fn encrypt(&self, plaintext: &[u8], aad: Option<&[u8]>) ->
Result<Vec<u8>> {
+ match self.key_size {
+ AesKeySize::Bits128 => {
+ encrypt_aes_gcm::<Aes128Gcm>(self.key.as_bytes(), plaintext,
aad)
+ }
+ AesKeySize::Bits192 => {
+ encrypt_aes_gcm::<Aes192Gcm>(self.key.as_bytes(), plaintext,
aad)
+ }
+ AesKeySize::Bits256 => {
+ encrypt_aes_gcm::<Aes256Gcm>(self.key.as_bytes(), plaintext,
aad)
+ }
+ }
+ }
+
+ /// Decrypts data using AES-GCM.
+ ///
+ /// # Arguments
+ /// * `ciphertext` - The encrypted data with format: [12-byte
nonce][encrypted data][16-byte auth tag]
+ /// * `aad` - Additional authenticated data (must match encryption)
+ ///
+ /// # Returns
+ /// The decrypted plaintext.
+ pub fn decrypt(&self, ciphertext: &[u8], aad: Option<&[u8]>) ->
Result<Vec<u8>> {
+ if ciphertext.len() < Self::NONCE_LEN + Self::TAG_LEN {
+ return Err(Error::new(
+ ErrorKind::DataInvalid,
+ format!(
+ "Ciphertext too short: expected at least {} bytes, got {}",
+ Self::NONCE_LEN + Self::TAG_LEN,
+ ciphertext.len()
+ ),
+ ));
+ }
+
+ match self.key_size {
+ AesKeySize::Bits128 => {
+ decrypt_aes_gcm::<Aes128Gcm>(self.key.as_bytes(), ciphertext,
aad)
+ }
+ AesKeySize::Bits192 => {
+ decrypt_aes_gcm::<Aes192Gcm>(self.key.as_bytes(), ciphertext,
aad)
+ }
+ AesKeySize::Bits256 => {
+ decrypt_aes_gcm::<Aes256Gcm>(self.key.as_bytes(), ciphertext,
aad)
+ }
+ }
+ }
+}
+
+fn encrypt_aes_gcm<C>(key_bytes: &[u8], plaintext: &[u8], aad: Option<&[u8]>)
-> Result<Vec<u8>>
+where C: Aead + AeadCore + KeyInit {
+ let cipher = C::new_from_slice(key_bytes).map_err(|e| {
+ Error::new(ErrorKind::DataInvalid, "Invalid AES
key").with_source(anyhow::anyhow!(e))
+ })?;
+ let nonce = C::generate_nonce(&mut OsRng);
+
+ let ciphertext = if let Some(aad) = aad {
+ cipher.encrypt(&nonce, Payload {
+ msg: plaintext,
+ aad,
+ })
+ } else {
+ cipher.encrypt(&nonce, plaintext.as_ref())
+ }
+ .map_err(|e| {
+ Error::new(ErrorKind::Unexpected, "AES-GCM encryption failed")
+ .with_source(anyhow::anyhow!(e))
+ })?;
+
+ // Prepend nonce to ciphertext (Java compatible format)
+ let mut result = Vec::with_capacity(nonce.len() + ciphertext.len());
+ result.extend_from_slice(&nonce);
+ result.extend_from_slice(&ciphertext);
+ Ok(result)
+}
+
+fn decrypt_aes_gcm<C>(key_bytes: &[u8], ciphertext: &[u8], aad: Option<&[u8]>)
-> Result<Vec<u8>>
+where C: Aead + AeadCore + KeyInit {
+ let cipher = C::new_from_slice(key_bytes).map_err(|e| {
+ Error::new(ErrorKind::DataInvalid, "Invalid AES
key").with_source(anyhow::anyhow!(e))
+ })?;
+
+ let nonce = Nonce::from_slice(&ciphertext[..AesGcmCipher::NONCE_LEN]);
+ let encrypted_data = &ciphertext[AesGcmCipher::NONCE_LEN..];
+
+ let plaintext = if let Some(aad) = aad {
+ cipher.decrypt(nonce, Payload {
+ msg: encrypted_data,
+ aad,
+ })
+ } else {
+ cipher.decrypt(nonce, encrypted_data)
+ }
+ .map_err(|e| {
+ Error::new(ErrorKind::Unexpected, "AES-GCM decryption failed")
+ .with_source(anyhow::anyhow!(e))
+ })?;
+
+ Ok(plaintext)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn test_aes_key_size() {
+ assert_eq!(AesKeySize::Bits128.key_length(), 16);
+ assert_eq!(AesKeySize::Bits192.key_length(), 24);
+ assert_eq!(AesKeySize::Bits256.key_length(), 32);
+
+ assert_eq!(
+ AesKeySize::from_key_length(16).unwrap(),
+ AesKeySize::Bits128
+ );
+ assert_eq!(
+ AesKeySize::from_key_length(24).unwrap(),
+ AesKeySize::Bits192
+ );
+ assert_eq!(
+ AesKeySize::from_key_length(32).unwrap(),
+ AesKeySize::Bits256
+ );
+ assert!(AesKeySize::from_key_length(8).is_err());
+
+ assert_eq!(AesKeySize::from_str("128").unwrap(), AesKeySize::Bits128);
+ assert_eq!(
+ AesKeySize::from_str("AES_GCM_128").unwrap(),
+ AesKeySize::Bits128
+ );
+ assert_eq!(
+ AesKeySize::from_str("AES_GCM_256").unwrap(),
+ AesKeySize::Bits256
+ );
+ assert!(AesKeySize::from_str("INVALID").is_err());
+ }
+
+ #[test]
+ fn test_secure_key() {
+ // Test key generation
+ let key1 = SecureKey::generate(AesKeySize::Bits128);
+ assert_eq!(key1.as_bytes().len(), 16);
+ assert_eq!(key1.key_size(), AesKeySize::Bits128);
+
+ // Test key creation with validation
+ let valid_key = [0u8; 16];
+ assert!(SecureKey::new(valid_key.as_slice()).is_ok());
+
+ let invalid_key = [0u8; 33];
+ assert!(SecureKey::new(invalid_key.as_slice()).is_err());
+ }
+
+ #[test]
+ fn test_aes128_gcm_encryption_roundtrip() {
+ let key = SecureKey::generate(AesKeySize::Bits128);
+ let cipher = AesGcmCipher::new(key);
+
+ let plaintext = b"Hello, Iceberg encryption!";
+ let aad = b"additional authenticated data";
+
+ // Test without AAD
+ let ciphertext = cipher.encrypt(plaintext, None).unwrap();
+ assert!(ciphertext.len() > plaintext.len() + 12); // nonce + tag
+ assert_ne!(&ciphertext[12..], plaintext); // encrypted portion differs
+
+ let decrypted = cipher.decrypt(&ciphertext, None).unwrap();
+ assert_eq!(decrypted, plaintext);
+
+ // Test with AAD
+ let ciphertext = cipher.encrypt(plaintext, Some(aad)).unwrap();
+ let decrypted = cipher.decrypt(&ciphertext, Some(aad)).unwrap();
+ assert_eq!(decrypted, plaintext);
+
+ // Test with wrong AAD fails
+ assert!(cipher.decrypt(&ciphertext, Some(b"wrong aad")).is_err());
+ }
+
+ #[test]
+ fn test_aes192_gcm_encryption_roundtrip() {
+ let key = SecureKey::generate(AesKeySize::Bits192);
+ let cipher = AesGcmCipher::new(key);
+
+ let plaintext = b"Hello, Iceberg encryption!";
+ let aad = b"additional authenticated data";
+
+ // Test without AAD
+ let ciphertext = cipher.encrypt(plaintext, None).unwrap();
+ let decrypted = cipher.decrypt(&ciphertext, None).unwrap();
+ assert_eq!(decrypted, plaintext);
+
+ // Test with AAD
+ let ciphertext = cipher.encrypt(plaintext, Some(aad)).unwrap();
+ let decrypted = cipher.decrypt(&ciphertext, Some(aad)).unwrap();
+ assert_eq!(decrypted, plaintext);
+
+ // Test with wrong AAD fails
+ assert!(cipher.decrypt(&ciphertext, Some(b"wrong aad")).is_err());
+ }
+
+ #[test]
+ fn test_aes256_gcm_encryption_roundtrip() {
+ let key = SecureKey::generate(AesKeySize::Bits256);
+ let cipher = AesGcmCipher::new(key);
+
+ let plaintext = b"Hello, Iceberg encryption!";
+ let aad = b"additional authenticated data";
+
+ // Test without AAD
+ let ciphertext = cipher.encrypt(plaintext, None).unwrap();
+ let decrypted = cipher.decrypt(&ciphertext, None).unwrap();
+ assert_eq!(decrypted, plaintext);
+
+ // Test with AAD
+ let ciphertext = cipher.encrypt(plaintext, Some(aad)).unwrap();
+ let decrypted = cipher.decrypt(&ciphertext, Some(aad)).unwrap();
+ assert_eq!(decrypted, plaintext);
+
+ // Test with wrong AAD fails
+ assert!(cipher.decrypt(&ciphertext, Some(b"wrong aad")).is_err());
+ }
+
+ #[test]
+ fn test_cross_key_size_incompatibility() {
+ let plaintext = b"Cross-key test";
+
+ let key128 = SecureKey::generate(AesKeySize::Bits128);
+ let key256 = SecureKey::generate(AesKeySize::Bits256);
+
+ let cipher128 = AesGcmCipher::new(key128);
+ let cipher256 = AesGcmCipher::new(key256);
+
+ // Ciphertext from 128-bit key should not decrypt with 256-bit key
+ let ciphertext = cipher128.encrypt(plaintext, None).unwrap();
+ assert!(cipher256.decrypt(&ciphertext, None).is_err());
+ }
+
+ #[test]
+ fn test_encryption_with_empty_plaintext() {
+ let key = SecureKey::generate(AesKeySize::Bits128);
+ let cipher = AesGcmCipher::new(key);
+
+ let plaintext = b"";
+ let ciphertext = cipher.encrypt(plaintext, None).unwrap();
+
+ // Even empty plaintext produces nonce + tag
+ assert_eq!(ciphertext.len(), 12 + 16); // 12-byte nonce + 16-byte tag
+
+ let decrypted = cipher.decrypt(&ciphertext, None).unwrap();
+ assert_eq!(decrypted, plaintext);
+ }
+
+ #[test]
+ fn test_decryption_with_tampered_ciphertext() {
+ let key = SecureKey::generate(AesKeySize::Bits128);
+ let cipher = AesGcmCipher::new(key);
+
+ let plaintext = b"Sensitive data";
+ let mut ciphertext = cipher.encrypt(plaintext, None).unwrap();
+
+ // Tamper with the encrypted portion (after the nonce)
+ if ciphertext.len() > 12 {
+ ciphertext[12] ^= 0xFF;
+ }
+
+ // Decryption should fail due to authentication tag mismatch
+ assert!(cipher.decrypt(&ciphertext, None).is_err());
+ }
+
+ #[test]
+ fn test_different_keys_produce_different_ciphertexts() {
+ let key1 = SecureKey::generate(AesKeySize::Bits128);
+ let key2 = SecureKey::generate(AesKeySize::Bits128);
+
+ let cipher1 = AesGcmCipher::new(key1);
+ let cipher2 = AesGcmCipher::new(key2);
+
+ let plaintext = b"Same plaintext";
+
+ let ciphertext1 = cipher1.encrypt(plaintext, None).unwrap();
+ let ciphertext2 = cipher2.encrypt(plaintext, None).unwrap();
+
+ // Different keys should produce different ciphertexts (comparing the
encrypted portion)
+ // Note: The nonces will also be different, but we're mainly
interested in the encrypted data
+ assert_ne!(&ciphertext1[12..], &ciphertext2[12..]);
+ }
+
+ #[test]
+ fn test_ciphertext_format_java_compatible() {
+ // Test that our ciphertext format matches Java's: [12-byte
nonce][ciphertext][16-byte tag]
+ let key = SecureKey::generate(AesKeySize::Bits128);
+ let cipher = AesGcmCipher::new(key);
+
+ let plaintext = b"Test data";
+ let ciphertext = cipher.encrypt(plaintext, None).unwrap();
+
+ // Format should be: [12-byte nonce][encrypted_data + 16-byte GCM tag]
+ assert_eq!(
+ ciphertext.len(),
+ 12 + plaintext.len() + 16,
+ "Ciphertext should be nonce + plaintext + tag length"
+ );
+
+ // Verify we can decrypt by extracting nonce from the beginning
+ let nonce = &ciphertext[..12];
+ assert_eq!(nonce.len(), 12, "Nonce should be 12 bytes");
+
+ // The rest is encrypted data + tag
+ let encrypted_with_tag = &ciphertext[12..];
+ assert_eq!(
+ encrypted_with_tag.len(),
+ plaintext.len() + 16,
+ "Encrypted portion should be plaintext length + 16-byte tag"
+ );
+ }
+}
diff --git a/crates/iceberg/src/encryption/mod.rs
b/crates/iceberg/src/encryption/mod.rs
new file mode 100644
index 000000000..097f4f24e
--- /dev/null
+++ b/crates/iceberg/src/encryption/mod.rs
@@ -0,0 +1,25 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Encryption module for Apache Iceberg.
+//!
+//! This module provides core cryptographic primitives for encrypting
+//! and decrypting data in Iceberg tables.
+
+mod crypto;
+
+pub use crypto::{AesGcmCipher, AesKeySize, SecureKey};
diff --git a/crates/iceberg/src/lib.rs b/crates/iceberg/src/lib.rs
index 8b345deb6..0b138d281 100644
--- a/crates/iceberg/src/lib.rs
+++ b/crates/iceberg/src/lib.rs
@@ -92,6 +92,7 @@ mod runtime;
pub mod arrow;
pub(crate) mod delete_file_index;
+pub mod encryption;
pub mod test_utils;
mod utils;
pub mod writer;
