srinipunuru commented on a change in pull request #1149: SAMZA-2316: Validate
that all non-default value fields in output schema are set i…
URL: https://github.com/apache/samza/pull/1149#discussion_r320880224
##########
File path:
samza-sql/src/main/java/org/apache/samza/sql/avro/AvroTypeFactoryImpl.java
##########
@@ -60,62 +59,68 @@ private SqlSchema convertSchema(List<Schema.Field> fields)
{
SqlSchemaBuilder schemaBuilder = SqlSchemaBuilder.builder();
for (Schema.Field field : fields) {
- SqlFieldSchema fieldSchema = convertField(field.schema());
+ // Consider any field with default value as nullable. Is it the right
assumption ?
Review comment:
This is slightly tricky. Nullable fields in avro is of type union with one
of the value being null.
The fields with default value means they are optional during serializing and
deserializing.
The behavior on the consumption side.
1. All avro fields should have values including nullable fields as well as
optional fields(fields with default values)
Behavior on the producer side.
1. fields with default values are optional
2. Nullable fields may still need values if they are not optional
So nullability and optionality are slightly different. I am not sure it is a
good idea to conflate them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
