cshuo commented on code in PR #18702:
URL: https://github.com/apache/hudi/pull/18702#discussion_r3207198632
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/parquet/ParquetSchemaConverter.java:
##########
@@ -190,6 +201,38 @@ public static MessageType
convertToParquetMessageType(String name, RowType rowTy
return new MessageType(name, types);
}
+ /**
+ * Checks whether the group carries the Parquet {@code VARIANT} logical type
annotation.
+ * Uses class-name matching so this compiles against parquet-java versions
that predate the
+ * {@code VariantLogicalTypeAnnotation} class (< 1.15.2).
+ */
+ private static boolean hasVariantAnnotation(LogicalTypeAnnotation
logicalType) {
Review Comment:
Since the writer does not attach the variant logical annotation (todo in
line 227), a Flink-written variant column will read back as a plain row and
HoodieSchemaConverter.convertToSchema() will lose the VARIANT type?
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/util/AvroToRowDataConverters.java:
##########
@@ -212,6 +217,25 @@ private static AvroToRowDataConverter
createMapConverter(LogicalType type, boole
};
}
+ /**
+ * Creates a converter for Flink 2.1+ VARIANT LogicalType. The converter
receives an Avro
+ * GenericRecord carrying metadata/value binary fields and produces a Flink
+ * {@code BinaryVariant}.
+ */
+ private static AvroToRowDataConverter createVariantConverter() {
+ return avroObject -> {
+ IndexedRecord record = (IndexedRecord) avroObject;
+ byte[] metadata = convertToBytes(record.get(0));
+ byte[] value = convertToBytes(record.get(1));
+
+ try {
+ return DataTypeAdapter.createVariant(value, metadata);
+ } catch (Exception e) {
+ throw new RuntimeException("Failed to create Flink BinaryVariant via
reflection.", e);
Review Comment:
This is a direct constructor call, not reflection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
