This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-rs.git
The following commit(s) were added to refs/heads/main by this push:
new e905793f20 Redact Parquet encryption keys from Debug output (#11614)
e905793f20 is described below
commit e905793f20b79d4431a678dd3bb0fedb8cfcab90
Author: autoCAT <[email protected]>
AuthorDate: Fri Oct 16 13:18:34 2026 +0000
Redact Parquet encryption keys from Debug output (#11614)
# Which issue does this PR close?
Relates to #7373. This addresses Debug-output redaction; key-memory
zeroization remains separate work.
# Rationale for this change
Debug-printing `FileEncryptionProperties`, or a `WriterProperties`
configuration containing it, currently includes raw footer and column
encryption keys. Logging these configurations can consequently expose
the keys along with otherwise useful diagnostics.
# What changes are included in this PR?
Give the private `EncryptionKey` type a manual Debug implementation that
prints only its type, following the existing `FileDecryptionProperties`
formatting style. Enclosing encryption and writer configurations retain
their other diagnostic fields.
# Are these changes tested?
Regression tests exercise ordinary and pretty Debug formatting through
the public encryption properties, writer builder, and built writer
properties. They fail against the previous derived implementation. The
tests also verify that explicit key and key-metadata getters preserve
their bytes.
Both new regressions fail before the change and pass after it. The
Parquet suite with encryption enabled passes (1,798 tests; 7 ignored),
as does the full workspace suite (7,330 tests; 29 ignored). Workspace
Clippy with all targets and features, formatting, and spelling checks
pass. An independent path-dependent application also confirms that
public encryption and writer configurations no longer expose either key.
# Are there any significant user-facing changes?
The public API and encryption behavior are unchanged. Debug formatting
no longer includes the contents of the private key objects.
Signed-off-by: Yanzhi Xin <[email protected]>
---
parquet/src/encryption/encrypt.rs | 9 ++++-
parquet/tests/encryption/encryption.rs | 62 ++++++++++++++++++++++++++++++++++
2 files changed, 70 insertions(+), 1 deletion(-)
diff --git a/parquet/src/encryption/encrypt.rs
b/parquet/src/encryption/encrypt.rs
index eb0cd09e83..591eac802d 100644
--- a/parquet/src/encryption/encrypt.rs
+++ b/parquet/src/encryption/encrypt.rs
@@ -26,15 +26,22 @@ use crate::parquet_thrift::{ThriftCompactOutputProtocol,
WriteThrift};
use crate::schema::types::{ColumnDescPtr, SchemaDescriptor};
use ring::rand::{SecureRandom, SystemRandom};
use std::collections::{HashMap, HashSet};
+use std::fmt::Formatter;
use std::io::Write;
use std::sync::Arc;
-#[derive(Debug, Clone, PartialEq)]
+#[derive(Clone, PartialEq)]
struct EncryptionKey {
key: Vec<u8>,
key_metadata: Option<Vec<u8>>,
}
+impl std::fmt::Debug for EncryptionKey {
+ fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
+ write!(f, "EncryptionKey {{ }}")
+ }
+}
+
impl EncryptionKey {
fn new(key: Vec<u8>) -> EncryptionKey {
Self {
diff --git a/parquet/tests/encryption/encryption.rs
b/parquet/tests/encryption/encryption.rs
index 9a507ebf50..74d2bc364a 100644
--- a/parquet/tests/encryption/encryption.rs
+++ b/parquet/tests/encryption/encryption.rs
@@ -46,6 +46,68 @@ use parquet::schema::parser::parse_message_type;
use std::fs::File;
use std::sync::Arc;
+#[test]
+fn test_encryption_properties_key_debug_redaction() {
+ let footer_key = b"SYNTHETIC-FOOTER".to_vec();
+ let column_key = b"SYNTHETIC-COLUMN".to_vec();
+ let footer_key_metadata = b"footer-key-id".to_vec();
+ let column_key_metadata = b"column-key-id".to_vec();
+ let properties = FileEncryptionProperties::builder(footer_key.clone())
+ .with_footer_key_metadata(footer_key_metadata.clone())
+ .with_column_key_and_metadata(
+ "private_column",
+ column_key.clone(),
+ column_key_metadata.clone(),
+ )
+ .build()
+ .unwrap();
+
+ for output in [format!("{properties:?}"), format!("{properties:#?}")] {
+ let compact: String = output.split_whitespace().collect();
+ for key in [&footer_key, &column_key] {
+ let key_debug: String =
format!("{key:?}").split_whitespace().collect();
+ assert!(!compact.contains(&key_debug));
+ }
+ assert!(output.contains("FileEncryptionProperties"));
+ assert!(output.contains("private_column"));
+ }
+
+ assert_eq!(properties.footer_key(), &footer_key);
+ assert_eq!(properties.footer_key_metadata(), Some(&footer_key_metadata));
+ assert_eq!(
+ properties.column_keys(),
+ (
+ vec!["private_column".to_string()],
+ vec![column_key],
+ vec![column_key_metadata],
+ )
+ );
+}
+
+#[test]
+fn test_writer_properties_key_debug_redaction() {
+ let footer_key = b"SYNTHETIC-FOOTER".to_vec();
+ let column_key = b"SYNTHETIC-COLUMN".to_vec();
+ let encryption = FileEncryptionProperties::builder(footer_key.clone())
+ .with_column_key("private_column", column_key.clone())
+ .build()
+ .unwrap();
+ let builder =
WriterProperties::builder().with_file_encryption_properties(encryption);
+ let builder_debug = [format!("{builder:?}"), format!("{builder:#?}")];
+ let properties = builder.build();
+ for output in builder_debug
+ .into_iter()
+ .chain([format!("{properties:?}"), format!("{properties:#?}")])
+ {
+ let compact: String = output.split_whitespace().collect();
+ for key in [&footer_key, &column_key] {
+ let key_debug: String =
format!("{key:?}").split_whitespace().collect();
+ assert!(!compact.contains(&key_debug));
+ }
+ assert!(output.contains("private_column"));
+ }
+}
+
#[test]
fn test_non_uniform_encryption_plaintext_footer() {
fn non_uniform_encryption_plaintext_footer(footer_key: &[u8], column_keys:
&[(&str, &[u8])]) {
