rahil-c commented on code in PR #18190:
URL: https://github.com/apache/hudi/pull/18190#discussion_r2805777249
##########
hudi-common/src/main/java/org/apache/hudi/common/schema/HoodieSchema.java:
##########
@@ -80,14 +80,40 @@
* @since 1.2.0
*/
public class HoodieSchema implements Serializable {
+ private static final long serialVersionUID = 1L;
/**
* Constant representing a null JSON value, equivalent to
JsonProperties.NULL_VALUE.
* This provides compatibility with Avro's JsonProperties while maintaining
Hudi's API.
*/
public static final Object NULL_VALUE = JsonProperties.NULL_VALUE;
public static final HoodieSchema NULL_SCHEMA =
HoodieSchema.create(HoodieSchemaType.NULL);
- private static final long serialVersionUID = 1L;
+ /**
+ * Constant to use when attaching type metadata to external schema systems
like Spark's StructType.
+ */
+ public static final String TYPE_METADATA_FIELD = "hudi_type";
+ public static final String TYPE_METADATA_PROPS_FIELD = "hudi_type_metadata";
+
+ /**
+ * Builds a comma-separated key=value metadata string from the given map.
+ * Example: {"vector.dimension": "128"} → "vector.dimension=128"
+ */
+ public static String buildTypeMetadata(Map<String, String> props) {
+ return props.entrySet().stream()
+ .map(e -> e.getKey() + "=" + e.getValue())
+ .collect(Collectors.joining(","));
+ }
+
+ /**
Review Comment:
Good point, right now i modeled this as the user passing in a string with
value of `"vector.dimension=128,vector.propB=someValue"` so need to think if we
should either just throw an exception to user saying the format was not adhered
for the property `hudi_type_metadata`. Or we can filter out malformed entries
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
