xiarixiaoyao commented on code in PR #10426:
URL: https://github.com/apache/hudi/pull/10426#discussion_r1440446787
##########
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java:
##########
@@ -257,6 +259,8 @@ private Option<Schema>
getTableSchemaFromCommitMetadata(HoodieInstant instant, b
Schema schema = new Schema.Parser().parse(existingSchemaStr);
if (includeMetadataFields) {
schema = HoodieAvroUtils.addMetadataFields(schema,
hasOperationField.get());
+ } else {
+ schema = HoodieAvroUtils.removeMetadataFields(schema);
Review Comment:
agree with remove metadata fields if includeMetadataFields is false.
Only internalSchema requires metadata fields,
Spark on hudi handles metadata fields on its own, so whether there are
metadata fields in the avor schema does not have much impact on Spark's read
and write operations.
However, from the perspective of function definition, it is indeed necessary
to remove metadata fields here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
