[
https://issues.apache.org/jira/browse/PARQUET-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547866#comment-16547866
]
ASF GitHub Bot commented on PARQUET-1227:
-----------------------------------------
zivanfi closed pull request #94: PARQUET-1227: Thrift crypto metadata structures
URL: https://github.com/apache/parquet-format/pull/94
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/src/main/thrift/parquet.thrift b/src/main/thrift/parquet.thrift
index 3b15cfef..3a265796 100644
--- a/src/main/thrift/parquet.thrift
+++ b/src/main/thrift/parquet.thrift
@@ -660,6 +660,22 @@ struct ColumnMetaData {
13: optional list<PageEncodingStats> encoding_stats;
}
+struct EncryptionWithFooterKey {
+}
+
+struct EncryptionWithColumnKey {
+ /** Column path in schema **/
+ 1: required list<string> path_in_schema
+
+ /** Retrieval metadata of the column-specific key **/
+ 2: optional binary column_key_metadata
+}
+
+union ColumnCryptoMetaData {
+ 1: EncryptionWithFooterKey ENCRYPTION_WITH_FOOTER_KEY
+ 2: EncryptionWithColumnKey ENCRYPTION_WITH_COLUMN_KEY
+}
+
struct ColumnChunk {
/** File where column data is stored. If not set, assumed to be same file as
* metadata. This path is relative to the current file.
@@ -686,6 +702,9 @@ struct ColumnChunk {
/** Size of ColumnChunk's ColumnIndex, in bytes **/
7: optional i32 column_index_length
+
+ /** Crypto metadata of encrypted columns **/
+ 8: optional ColumnCryptoMetaData crypto_meta_data
}
struct RowGroup {
@@ -877,3 +896,37 @@ struct FileMetaData {
7: optional list<ColumnOrder> column_orders;
}
+struct AesGcmV1 {
+ /** Retrieval metadata of AAD used for encryption of pages and structures **/
+ 1: optional binary aad_metadata
+}
+
+struct AesGcmCtrV1 {
+ /** Retrieval metadata of AAD used for encryption of structures **/
+ 1: optional binary aad_metadata
+}
+
+union EncryptionAlgorithm {
+ 1: AesGcmV1 AES_GCM_V1
+ 2: AesGcmCtrV1 AES_GCM_CTR_V1
+}
+
+struct FileCryptoMetaData {
+ 1: required EncryptionAlgorithm encryption_algorithm
+
+ /** Parquet footer can be encrypted, or left as plaintext **/
+ 2: required bool encrypted_footer
+
+ /** Retrieval metadata of key used for encryption of footer,
+ * and (possibly) columns **/
+ 3: optional binary footer_key_metadata
+
+ /** Offset of Parquet footer (encrypted, or plaintext) **/
+ 4: required i64 footer_offset
+
+ /** If file IVs are comprised of a fixed part,
+ * and variable parts (random or counter), keep the fixed
+ * part here **/
+ 5: optional binary iv_prefix
+}
+
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Thrift crypto metadata structures
> ---------------------------------
>
> Key: PARQUET-1227
> URL: https://issues.apache.org/jira/browse/PARQUET-1227
> Project: Parquet
> Issue Type: Sub-task
> Components: parquet-cpp, parquet-format
> Reporter: Gidon Gershinsky
> Assignee: Gidon Gershinsky
> Priority: Major
> Labels: pull-request-available
>
> New Thrift structures for Parquet modular encryption
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
