voonhous commented on code in PR #18036:
URL: https://github.com/apache/hudi/pull/18036#discussion_r3040218472
##########
hudi-common/src/main/java/org/apache/hudi/common/schema/HoodieSchema.java:
##########
@@ -866,6 +866,112 @@ public static HoodieSchema.Vector createVector(String
name, int dimension, Vecto
return new HoodieSchema.Vector(vectorSchema);
}
+ /**
+ * Creates a shredded field struct per the Parquet variant shredding spec.
The returned struct contains two nullable fields:
+ * <ul>
+ * <li>{@code value}: nullable bytes (fallback binary representation)</li>
+ * <li>{@code typed_value}: nullable type (the typed representation)</li>
+ * </ul>
+ *
+ * <p>Example output structure:
+ * <pre>
+ * fieldName: struct
+ * |-- value: binary (nullable)
+ * |-- typed_value: <fieldType> (nullable)
+ * </pre></p>
+ *
+ * @param fieldName the name for the record (used as the Avro record name)
+ * @param fieldType the schema for the typed_value within this field
+ * @return a new HoodieSchema representing the shredded field struct
+ */
+ public static HoodieSchema createShreddedFieldStruct(String fieldName,
HoodieSchema fieldType) {
+ ValidationUtils.checkArgument(fieldName != null && !fieldName.isEmpty(),
"Field name cannot be null or empty");
+ ValidationUtils.checkArgument(fieldType != null, "Field type cannot be
null");
+ List<HoodieSchemaField> fields = Arrays.asList(
+ HoodieSchemaField.of(
+ Variant.VARIANT_VALUE_FIELD,
+ HoodieSchema.createNullable(HoodieSchemaType.BYTES),
+ "Fallback binary representation",
+ NULL_VALUE
+ ),
+ HoodieSchemaField.of(
+ Variant.VARIANT_TYPED_VALUE_FIELD,
+ HoodieSchema.createNullable(fieldType),
+ "Typed value representation",
+ NULL_VALUE
+ )
+ );
+ return HoodieSchema.createRecord(fieldName, null, null, fields);
+ }
Review Comment:
Wow, good catch, addressed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
