tolleybot commented on code in PR #34616:
URL: https://github.com/apache/arrow/pull/34616#discussion_r1259707663
##########
cpp/src/arrow/dataset/file_parquet.h:
##########
@@ -136,6 +138,20 @@ class ARROW_DS_EXPORT ParquetFileFormat : public
FileFormat {
fs::FileLocator destination_locator) const override;
std::shared_ptr<FileWriteOptions> DefaultWriteOptions() override;
+
+ /// \brief A getter function to retrieve the dataset encryption configuration
+ std::shared_ptr<DatasetEncryptionConfiguration> GetDatasetEncryptionConfig()
const {
+ return dataset_encryption_config_;
+ }
+ /// \brief A setter for DatasetEncryptionConfiguration
+ void SetDatasetEncryptionConfig(
+ std::shared_ptr<DatasetEncryptionConfiguration>
dataset_encryption_config) {
+ dataset_encryption_config_ = std::move(dataset_encryption_config);
+ }
+
+ private:
+ // A configuration structure that provides per file encryption properties
for a dataset
+ std::shared_ptr<DatasetEncryptionConfiguration> dataset_encryption_config_ =
NULLPTR;
Review Comment:
@wgtmac @westonpace @wjones127
I need to get some idea of how the flow would go if I move the
DatasetEncryptionConfiguration into ParquetFragmentScanOptions and
ParquetFragmeentScanOptions. Here is a sample of the current implementation in
python being used
https://github.com/tolleybot/arrow/blob/f3f73d86c28551f89b5aab08bfd120dc85dfca80/python/examples/dataset/write_dataset_encrypted.py#L43-L74
My questions are:
1. How would the DatasetEncryptionConfiguration be propagated to those
classes? Through ParquetFileFormat?
2. When would the propagation take place.
3. Looking at the python sample, how would this look with this change?
Thanks for any help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
