wmoustafa commented on code in PR #6004:
URL: https://github.com/apache/iceberg/pull/6004#discussion_r1156291710
##########
core/src/main/java/org/apache/iceberg/avro/BuildAvroProjection.java:
##########
@@ -102,9 +102,17 @@ public Schema record(Schema record, List<String> names,
Iterable<Schema.Field> s
} else {
Preconditions.checkArgument(
- field.isOptional() ||
MetadataColumns.metadataFieldIds().contains(field.fieldId()),
- "Missing required field: %s",
+ (field.isRequired() && field.initialDefault() != null)
+ || field.isOptional()
+ ||
MetadataColumns.metadataFieldIds().contains(field.fieldId()),
+ "Missing required field that doesn't have a default value: %s",
field.name());
+ // If the field from Iceberg schema has initial default value, we just
pass and don't
+ // project it to the avro file read schema with the generated _r field
name,
+ // the default value will be directly read from the Iceberg layer
+ if (field.initialDefault() != null) {
Review Comment:
I think it depends on the implementation of `ValueReaders`. In the case of
defaults, `ValueReaders` takes care of filling in the default value. In the
case of optionals, it is delegated to Avro to fill in the null values. I like
the idea of doing it in the `ValueReaders` so it is consistent across all file
formats, even when the file format does not support the semantic (e.g., ORC
schema does not support default values).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
