voonhous commented on code in PR #19809:
URL: https://github.com/apache/hudi/pull/19809#discussion_r3915867255
##########
hudi-common/src/main/java/org/apache/hudi/common/schema/HoodieSchemaUtils.java:
##########
@@ -731,11 +721,41 @@ public static HoodieSchema
getRecordKeyPartitionPathSchema() {
return HoodieSchema.createRecord("HoodieRecordKey", "", "", false,
toBeAddedFields);
}
+ /**
+ * Schema of a native delete log record: the record key plus the ordering
fields, which are
+ * always nullable (see the comment in the body).
+ */
+ public static HoodieSchema createDeleteLogSchema(HoodieSchema tableSchema,
List<String> orderingFieldNames) {
+ // Native delete logs store only the record key plus optional ordering
values, so ordering fields in
+ // the delete-log schema must always be nullable even when the table
schema marks them required.
+ // A delete record such as HoodieEmptyRecord may carry
OrderingValues.getDefault() as an in-memory
+ // sentinel rather than a real field value. Persist NULL for that missing
value so readers can map it
+ // back to the default ordering without confusing it with a real business
value such as 0.
+ List<HoodieSchemaField> fields = Stream.concat(
+ Stream.of(createNewSchemaField(
+ HoodieRecord.RECORD_KEY_METADATA_FIELD,
HoodieSchema.create(HoodieSchemaType.STRING), null, null)),
+ orderingFieldNames.stream().map(orderingFieldName ->
tableSchema.getField(orderingFieldName)
Review Comment:
Filed an issue for. this here, will fix it outside of this PR and keep this
PR purely for refactoring.
https://github.com/apache/hudi/issues/19823
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
