wgtmac commented on code in PR #34616:
URL: https://github.com/apache/arrow/pull/34616#discussion_r1257284061
##########
cpp/src/arrow/dataset/file_parquet.h:
##########
@@ -136,6 +138,20 @@ class ARROW_DS_EXPORT ParquetFileFormat : public
FileFormat {
fs::FileLocator destination_locator) const override;
std::shared_ptr<FileWriteOptions> DefaultWriteOptions() override;
+
+ /// \brief A getter function to retrieve the dataset encryption configuration
+ std::shared_ptr<DatasetEncryptionConfiguration> GetDatasetEncryptionConfig()
const {
+ return dataset_encryption_config_;
+ }
+ /// \brief A setter for DatasetEncryptionConfiguration
+ void SetDatasetEncryptionConfig(
+ std::shared_ptr<DatasetEncryptionConfiguration>
dataset_encryption_config) {
+ dataset_encryption_config_ = std::move(dataset_encryption_config);
+ }
+
+ private:
+ // A configuration structure that provides per file encryption properties
for a dataset
+ std::shared_ptr<DatasetEncryptionConfiguration> dataset_encryption_config_ =
NULLPTR;
Review Comment:
I am not that familiar the design principle of dataset api. Is it the best
place to put the new config here? Or should we move it into
`ParquetFragmentScanOptions` and `ParquetFileWriteOptions` respectively?
@westonpace @wjones127
##########
cpp/src/arrow/dataset/file_parquet.h:
##########
@@ -136,6 +138,20 @@ class ARROW_DS_EXPORT ParquetFileFormat : public
FileFormat {
fs::FileLocator destination_locator) const override;
std::shared_ptr<FileWriteOptions> DefaultWriteOptions() override;
+
+ /// \brief A getter function to retrieve the dataset encryption configuration
+ std::shared_ptr<DatasetEncryptionConfiguration> GetDatasetEncryptionConfig()
const {
+ return dataset_encryption_config_;
+ }
+ /// \brief A setter for DatasetEncryptionConfiguration
+ void SetDatasetEncryptionConfig(
+ std::shared_ptr<DatasetEncryptionConfiguration>
dataset_encryption_config) {
+ dataset_encryption_config_ = std::move(dataset_encryption_config);
+ }
+
+ private:
+ // A configuration structure that provides per file encryption properties
for a dataset
+ std::shared_ptr<DatasetEncryptionConfiguration> dataset_encryption_config_ =
NULLPTR;
Review Comment:
After double check the design doc
(https://docs.google.com/document/d/13EysCNC6-Nu9wnJ8YpdzmD-aMLn4i2KXUJTNqIihy7A),
I think the original design to add two separate classes (i.e. EncryptionConfig
and DecryptionConfig) was considered to add those configs to
`ParquetFileWriteOptions` and `ParquetFragmentScanOptions` separately. In the
implementation of `MakeReaderProperties` and `MakeWriter` calls, we need to
translate these config to the current writer/reader properties. @tolleybot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
