yihua commented on code in PR #20079:
URL: https://github.com/apache/hudi/pull/20079#discussion_r4169931397
##########
hudi-hadoop-common/src/main/java/org/apache/hudi/common/util/ParquetUtils.java:
##########
@@ -196,6 +198,23 @@ public void close() {
}
}
+ /**
+ * Sets the Hadoop read options of a reader built with {@code
ParquetReader.Builder(InputFile)} from the
+ * file's {@link Configuration}, as {@code ParquetReader.Builder(Path)}
followed by {@code withConf} does,
+ * without creating a new {@link Configuration}. On parquet 1.15+ that
constructor builds plain
+ * {@code ParquetReadOptions}, which never consult the {@code
parquet.crypto.factory.class} decryption
+ * factory, and {@code withConf} drops the file path, so the decryption
properties are resolved here with it.
+ */
+ public static <T> ParquetReader.Builder<T>
withHadoopReadOptions(ParquetReader.Builder<T> builder, HadoopInputFile file) {
+ Configuration conf = file.getConfiguration();
+ builder.withConf(conf);
+ DecryptionPropertiesFactory decryptionFactory =
DecryptionPropertiesFactory.loadFactory(conf);
+ if (decryptionFactory != null) {
+
builder.withDecryption(decryptionFactory.getFileDecryptionProperties(conf,
file.getPath()));
+ }
Review Comment:
Is this the only config we need to add? How could we make this maintainable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
