daniel-adam-tfs commented on PR #596: URL: https://github.com/apache/arrow-go/pull/596#issuecomment-3626978794
@zeroshade OK, so I've generated tools/pyarrow_encrypted_uniform.parquet using tools/write_encrypted_parquet.py and tools/arrowgo_encrypted_uniform.parquet using TestEncryptFile. Very likely that I'm doing something wrong, but let me walk you through what I have. In the python code I need an instance of FileEncryptionProperties which I use in the call to write_table, but the only API I was able to find in the PyArrow lib is [CryptoFactory.file_encryption_properties](https://arrow.apache.org/docs/python/generated/pyarrow.parquet.encryption.CryptoFactory.html#pyarrow.parquet.encryption.CryptoFactory.file_encryption_properties), but for that I need a KMS. I've created a Mock implementation, which just base64 encodes and decodes the input and use this to generate the pyarrow_encrypted_uniform.parquet. I try to read this generated file in TestDecryptFile, it panics in StringKeyIDRetriever.GetKey call. ```go encryption.StringKeyIDRetriever=["footer_key": "0123456789012345", ] ``` ```go func (s StringKeyIDRetriever) GetKey(keyMetadata []byte) string { k, ok := s[*(*string)(unsafe.Pointer(&keyMetadata))] if !ok { panic(fmt.Errorf("parquet: key missing for id %s", keyMetadata)) } return k } ``` ```go keymetadata={\"keyMaterialType\":\"PKMT1\",\"internalStorage\":true,\"isFooterKey\":true,\"kmsInstanceID\":\"DEFAULT\",\"kmsInstanceURL\":\"DEFAULT\",\"masterKeyID\":\"footer_key\",\"wrappedDEK\":\"tHPE5PlN58jGE1soVo/arMTVu8C8oezum3vSnNdEcEdIn5ImAcv9rtpfZow=\",\"doubleWrapping\":true,\"keyEncryptionKeyID\":\"7pmHfFBvnjd2Wbf218WOMQ==\",\"wrappedKEK\":\"1eX3O2IHHTkAnuIXbbIQRA==\"}" ``` I'm guessing that I should implement a custom KeyIDRetriever, and json decode the retrieved metadata and return the `masterKeyID` value? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
