adamreeve commented on issue #50969: URL: https://github.com/apache/arrow/issues/50969#issuecomment-5401534599
The spec says that plaintext footer mode "allows legacy Parquet versions (released before the encryption support) to access unencrypted columns in encrypted files." But we (G-Research) like using plaintext footer mode because it makes it easy for engineers to inspect Parquet files and see non-sensitive columns without needing to configure the KMS and decryption settings. So I don't think plaintext footer mode should be removed, and agree with Gidon that we should probably update the spec wording. If newer readers can handle the missing metadata OK, then maybe the plaintext column metadata for encrypted columns could be completely removed. But we should verify that first, I wouldn't be surprised if that caused problems for readers that do support encryption when they don't have access to those columns, even though it would be spec compliant. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
