voonhous commented on code in PR #19926:
URL: https://github.com/apache/hudi/pull/19926#discussion_r4625018240
##########
hudi-hadoop-common/src/main/java/org/apache/hudi/common/util/ParquetUtils.java:
##########
@@ -452,10 +452,27 @@ public HoodieFileFormat getFormat() {
*/
@Override
public List<GenericRecord> readAvroRecords(HoodieStorage storage,
StoragePath filePath) {
+ // A schema-less read takes the schema from the file; the copy drops any
read schema or
+ // projection a previous caller left on the shared configuration.
+ Configuration conf = storage.getConf().unwrapCopyAs(Configuration.class);
+ conf.unset("parquet.avro.read.schema");
+ conf.unset(AvroReadSupport.AVRO_REQUESTED_PROJECTION);
+ return readAvroRecords(conf, filePath);
+ }
+
+ @Override
+ public List<GenericRecord> readAvroRecords(HoodieStorage storage,
StoragePath filePath, HoodieSchema schema) {
+ // The read schema goes on a private copy so the caller's configuration is
left untouched.
+ Configuration conf = storage.getConf().unwrapCopyAs(Configuration.class);
+ AvroReadSupport.setAvroReadSchema(conf, schema.toAvroSchema());
Review Comment:
Fixed: both overloads now build their private copy through one helper,
`readConf`, which unsets `parquet.avro.read.schema` and
`AVRO_REQUESTED_PROJECTION`. `testReadAvroRecordsLeavesCallerConfUntouched` now
plants a key-only projection and then does an explicit-schema read: the stored
`_hoodie_partition_path` comes back as `p1`. Before the fix it came back null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]