anirudhsr opened a new issue, #35678:
URL: https://github.com/apache/arrow/issues/35678
### Describe the usage question you have. Please include as many useful
details as possible.
Hi Team,
I have a use case where I'm trying to encrypt my parquet files and save the
keys externally so it's easier to rotate the master keys and re-encrypt the DEK
without reading the original file.
However, I'm unable to save the footer files separately with just specifying
the `internal_key_material=False` in `EncryptionConfiguration`. Is there
something else missing in the documentation or is this is a bug ?
Here's what I'm doing:
`
encryption_config = pe.EncryptionConfiguration(
footer_key=key_name,
column_keys={
key_name: ["a", "b"],
},
plaintext_footer=True,
internal_key_material=False,
data_key_length_bits=256,
)
`
`
# Begin En-cryption
file_encryption_properties = crypto_factory.file_encryption_properties(
kms_connection_config, encryption_config)
`
`
with pq.ParquetWriter(path,
table.schema,
encryption_properties=file_encryption_properties,
use_dictionary=False,
compression='none',) as writer:
writer.write_table(table)
`
This is the error that I keep getting
`
File "/Users/sayalaso/aidata/dataflow/VaultClient.py", line 149, in
parquet_write_read_with_vault
file_encryption_properties = crypto_factory.file_encryption_properties(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/_parquet_encryption.pyx", line 423, in
pyarrow._parquet_encryption.CryptoFactory.file_encryption_properties
File "pyarrow/error.pxi", line 144, in
pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 115, in pyarrow.lib.check_status
OSError: Failed to get key material store.
The Parquet file path must be specified when using external key material
`
although as per the Doc it says that: `If set to false, key material is
stored in separate files in the same folder`
### Component(s)
Parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
