rzhang10 commented on code in PR #6004:
URL: https://github.com/apache/iceberg/pull/6004#discussion_r1005296940
##########
core/src/main/java/org/apache/iceberg/avro/BuildAvroProjection.java:
##########
@@ -102,9 +102,17 @@ public Schema record(Schema record, List<String> names,
Iterable<Schema.Field> s
} else {
Preconditions.checkArgument(
- field.isOptional() ||
MetadataColumns.metadataFieldIds().contains(field.fieldId()),
- "Missing required field: %s",
+ (field.isRequired() && field.initialDefault() != null)
+ || field.isOptional()
+ ||
MetadataColumns.metadataFieldIds().contains(field.fieldId()),
+ "Missing required field that doesn't have a default value: %s",
field.name());
+ // If the field from Iceberg schema has initial default value, we just
pass and don't
+ // project it to the avro file read schema with the generated _r field
name,
+ // the default value will be directly read from the Iceberg layer
+ if (field.initialDefault() != null) {
Review Comment:
I see, I originally thought using `continue` can just be an implementation
trick, since although the output avro schema of the buildavroprojection process
is lacking the field, it will be taken care of when they are being compared
again in the `ValueReader`.
I think I can make the code logic more consistent by still creating a fake
field with some other suffix like "_d" instead of "_r". The "_d" field will not
be projected in the real file either, and the default value read is still taken
care of by `ValueReader`. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
