nsivabalan commented on code in PR #17601:
URL: https://github.com/apache/hudi/pull/17601#discussion_r2820275041
##########
hudi-hadoop-common/src/main/java/org/apache/parquet/avro/HoodieAvroReadSupport.java:
##########
@@ -143,4 +151,16 @@ private List<Type> convertLegacyMap(List<Type> oldTypes) {
}
return newTypes;
}
+
+ private MessageType repairLogicalTypes(MessageType fileSchema,
Option<MessageType> tableSchemaOpt) {
Review Comment:
hey @linliu-code :
looks like this is also being invoked from w/n spark tasks.
Similar to how we fixed AVRO_SCHEMA_REPAIR_CLASS usage in AvroSchemaUtils,
should we not try to fix this as well. So, that we avoid each spark task trying
to use reflection to call this.
and esply for older spark versions, we return right away w/o relying on
exceptions.
CC @yihua
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
