linliu-code commented on code in PR #17601:
URL: https://github.com/apache/hudi/pull/17601#discussion_r2738863806
##########
hudi-common/src/main/java/org/apache/hudi/avro/AvroSchemaUtils.java:
##########
@@ -471,4 +494,15 @@ public static Schema asNullable(Schema schema) {
(change, field) -> change.updateColumnNullability(field, true));
return convert(SchemaChangeUtils.applyTableChanges2Schema(internalSchema,
schemaChange), schema.getFullName());
}
+
+ public static Schema getRepairedSchema(Schema writerSchema, Schema
readerSchema) {
Review Comment:
Good question.
Since for <= spark 3.3, this class is not needed.
The further why? if we include `AvroSchemaRepair` for all spark versions, it
will throws since `TimestampNTZ` class is not found for spark <= 3.2. Spark3.3
has this class, but the support for it is not complete. Therefore, we only
include `AvroSchemaRepair` class for spark3.4+, and we need to use reflection
to use this class when it is available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
