yihua commented on code in PR #12390:
URL: https://github.com/apache/hudi/pull/12390#discussion_r1869227673
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -1488,4 +1488,23 @@ public static Comparable<?>
unwrapAvroValueWrapper(Object avroValueWrapper) {
}
}
+ public static boolean containsUnsupportedTypesForFileGroupReader(Schema
schema) {
+ switch (schema.getType()) {
+ case RECORD:
+ for (Field field : schema.getFields()) {
+ if (containsUnsupportedTypesForFileGroupReader(field.schema())) {
+ return true;
+ }
+ }
+ return false;
+ case ARRAY:
+ return
containsUnsupportedTypesForFileGroupReader(schema.getElementType());
+ case MAP:
+ return
containsUnsupportedTypesForFileGroupReader(schema.getValueType());
+ case UNION:
+ return
containsUnsupportedTypesForFileGroupReader(getActualSchemaFromUnion(schema,
null));
+ default:
+ return schema.getType() == Schema.Type.ENUM || schema.getLogicalType()
instanceof LogicalTypes.Decimal;
Review Comment:
Now it's supported by fixing the Spark Parquet writer.
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -1488,4 +1488,23 @@ public static Comparable<?>
unwrapAvroValueWrapper(Object avroValueWrapper) {
}
}
+ public static boolean containsUnsupportedTypesForFileGroupReader(Schema
schema) {
+ switch (schema.getType()) {
+ case RECORD:
+ for (Field field : schema.getFields()) {
+ if (containsUnsupportedTypesForFileGroupReader(field.schema())) {
+ return true;
+ }
+ }
+ return false;
+ case ARRAY:
+ return
containsUnsupportedTypesForFileGroupReader(schema.getElementType());
+ case MAP:
+ return
containsUnsupportedTypesForFileGroupReader(schema.getValueType());
+ case UNION:
+ return
containsUnsupportedTypesForFileGroupReader(getActualSchemaFromUnion(schema,
null));
+ default:
+ return schema.getType() == Schema.Type.ENUM || schema.getLogicalType()
instanceof LogicalTypes.Decimal;
Review Comment:
Now Decimal is supported by fixing the Spark Parquet writer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
