shangxinli opened a new pull request, #1030: URL: https://github.com/apache/iceberg-cpp/pull/1030
## Summary Step 4 of #988. Adds the crypto primitives of table encryption to the bundle library, built when `ICEBERG_ENCRYPTION` is ON. No behavior change for existing builds: everything here is new, and the default (`ICEBERG_ENCRYPTION=OFF`) compiles none of it. - `AesGcmCipher` (OpenSSL EVP): AES-GCM with a random 12-byte nonce, in the `nonce | ciphertext | tag` layout of Java's `Ciphers`. Key lengths 16/24/32, AAD optional, every OpenSSL return code checked including `EVP_DecryptFinal_ex`, so a failed tag check is always an error rather than unauthenticated plaintext. - `AesGcmInputFile` / `AesGcmOutputFile`: AES GCM Stream files per [`format/gcm-stream-spec.md`](https://github.com/apache/iceberg/blob/main/format/gcm-stream-spec.md), the format used for manifests, manifest lists, Avro data files and Puffin files. 1 MiB plaintext blocks, header `AGS1` + little-endian block size, block AAD = file AAD prefix followed by the block index as a little-endian int32, and only the last block may be partial. The reader is seekable and takes the encrypted length from the key metadata rather than the file, so truncation is detected; the writer reports the plaintext `Position()` and the encrypted `StoredLength()`. `aes_gcm_stream.h` is the first public header that requires `ICEBERG_ENCRYPTION` while sharing a directory with unconditional ones (`encrypted_key.h`, `standard_key_metadata.h`). `ci/scripts/generate_public_headers.py` only classifies whole directories, so it grows a small per-header override table, and `cpp20_compatibility_test` gains a matching `ICEBERG_PUBLIC_HEADERS_ENCRYPTION` define. ## Java parity Ported against `core/src/main/java/org/apache/iceberg/encryption/{Ciphers,AesGcmInputStream,AesGcmOutputStream,AesGcmInputFile,AesGcmOutputFile}.java`. Two places where the Java semantics are not obvious from the C++ base classes and are deliberately matched: - `AesGcmOutputStream::Position()` keeps reporting the plaintext length after `Close()`. Closing flushes the final block, which resets the in-block position, so the value is captured first — Java does the same with its `finalPosition` field. - An empty stream is written as one empty block, while a stream whose length is an exact multiple of the block size gets no trailing empty block. `StandardKeyMetadata` records the stored length for every encrypted file because Java needs it for AGS1 reads; that part landed in #991. ## Test plan - [x] `crypto_test` passes. The tests decrypt AGS1 streams of 0, 1 and 100 bytes and a KEK-encrypted manifest-list key metadata record, all produced by Iceberg Java, so a wire-format divergence fails rather than round-tripping against itself. Following #991, the fixtures are inlined byte arrays rather than checked-in binaries. - [x] Round trips across block boundaries (0, 1, `block-1`, `block`, `block+1`, `2*block+17`) with write chunks that straddle blocks, plus random access across a boundary. - [x] Wrong key, wrong AAD prefix, tampered ciphertext, truncated ciphertext, bad magic and a too-short stream all return errors. - [x] `cpp20_compatibility_test` passes with `ICEBERG_ENCRYPTION=ON`. - [x] `pre-commit` clean, including the generated public-header check. Verified that the `Position()`-after-`Close()` assertion fails without its fix (reports `1048576` for a 100-byte stream). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
