VitoMakarevich commented on code in PR #11450:
URL: https://github.com/apache/hudi/pull/11450#discussion_r1640275918
##########
hudi-hadoop-common/src/main/java/org/apache/parquet/avro/HoodieAvroReadSupport.java:
##########
@@ -51,6 +51,13 @@ public ReadContext init(Configuration configuration,
Map<String, String> keyValu
configuration.set(AvroWriteSupport.WRITE_OLD_LIST_STRUCTURE,
"false", "support reading avro from non-legacy map/list in parquet
file");
}
+ // If old file is written with legacy mode, read with legacy mode, even if
now it's non-legacy mode in conf.
+ // Since by changing the property users want to control how files are
written, not how they are read.
+ // Later the value of WRITE_OLD_LIST_STRUCTURE will be picked from conf,
thus either keeping old mode or writing new mode.
+ if (legacyMode) {
+ configuration.set(AvroWriteSupport.WRITE_OLD_LIST_STRUCTURE,
Review Comment:
I think indeed this change requires thoroughful check, I believe experts in
this code can find a better solution. At least the test I've written should
test the potential other scenarios (for any fix being developed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
