EdwardShen125 opened a new pull request, #1478: URL: https://github.com/apache/arrow-go/pull/1478
### Rationale for this change #566 reported that files written by pyarrow's CryptoFactory cannot be read by arrow-go: the footer and column key metadata carry PKMT1 key material with a wrapped DEK, and arrow-go had no parser or unwrapping for it, so footer decryption failed without a hand-written `KeyRetriever`. As agreed in #566, the KMS support lives in the library as an optional component. ### What changes are included in this PR? A new optional `parquet/encryption` package: - `KmsClient` interface mirroring the parquet-cpp/pyarrow contract (`WrapKey`/`UnwrapKey`) - `KeyMaterial`/`ParseKeyMetadata` for PKMT1 key metadata (internal key storage) - `KmsKeyRetriever` implementing `parquet.DecryptionKeyRetriever`: in single wrapping mode the DEK is unwrapped by the `KmsClient`; in double wrapping mode the KEK is unwrapped through the client and the DEK is then decrypted locally with AES-GCM (`nonce || ciphertext || tag`, AAD = the base64 decoded KEK ID bytes), byte-compatible with parquet-cpp's `EncryptKeyLocally` It plugs into the existing `parquet.WithKeyRetriever` option; code that does not import the package is unaffected. The package doc and a godoc example (`ExampleKmsKeyRetriever`) show the wiring and state the optional nature explicitly. The interop harness from #1439 gains a `gen-kms` mode: pyarrow's CryptoFactory writes double-wrapped (the default) and single-wrapped files using a test KMS client whose wrapping is plain base64, and Go reads both through the retriever — the footer key and a column key both take that path. No new python dependencies, so the existing cgo python CI job runs them unchanged. The review nits from #1439 are addressed (flag rename, unused footer key metadata dropped, doc comment dedupe). The write side (Go producing PKMT1 key material via a CryptoFactory) is a natural follow-up; this PR is read-only. ### Are these changes tested? Unit tests cover PKMT1 parsing (valid and error cases), single and double unwrap round-trips, and tamper detection. Against pyarrow 26.0.0 locally, with `-race` and `PARQUET_TEST_REQUIRE_PYARROW=1`, both interop modes pass; without pyarrow everything skips. ### Are there any user-facing changes? New optional public package `parquet/encryption`. No changes to existing APIs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
