raviston commented on PR #3935: URL: https://github.com/apache/parquet-java/pull/3935#issuecomment-6847360679
I ran an independent interop check against this branch (head bb677884e, JDK 17) using a separate harness, not the fixtures in this PR. It writes with `PropertiesDrivenCryptoFactory` + `InMemoryKMS` (AES_GCM_V1), V2 pages, no dictionary. The schema is `optional int32 x` plus an optional 3-level `LIST<STRING>`, and the rows include null lists, empty lists and null elements, so both levels are non-trivial. On the PyArrow side (18.1.0) the KMS client wraps keys the same way. | Writer | Reader | Result | |---|---|---| | this PR | PyArrow 18.1.0 | ✅ all 5 rows match | | this PR | this PR | ✅ | | PyArrow 18.1.0 (snappy) | this PR | ✅ | | parquet-java 1.18.1 (plaintext levels) | this PR | ✅ legacy layout detected | | parquet-java 1.18.1 | PyArrow 18.1.0 | ❌ `Serialized ciphertext length 2495751 is greater than the provided ciphertext buffer length 44` (the original bug, for reference) | `TestDataPageV2EncryptionInterop` (61), `TestSingleModuleDataPageV2` (25), `TestPropertiesDrivenEncryption` (32), `TestParquetFileWriter` (39), `TestColumnIndexFiltering` (24) and `TestEncryptionOptions` (1) all pass locally. I had opened #3947 for the same fix without seeing this PR. I'm closing it in favour of this one. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
