rdblue commented on a change in pull request #4301:
URL: https://github.com/apache/iceberg/pull/4301#discussion_r824302927
##########
File path: format/spec.md
##########
@@ -193,10 +193,38 @@ Notes:
For details on how to serialize a schema to JSON, see Appendix C.
+#### Default value
+Default values can be assigned to top-level columns or nested fields. Default
values are used during schema evolution when adding a new column. The default
value is used to read rows belonging to the files that lack the column or
nested field prior to the schema evolution.
Review comment:
> I was saying it would be the natural behavior even if we combine both
concepts to one but assuming a rewrite takes place immediately.
This is basically what I did in the SQL snippet above, so I don't think that
you can distinguish between setting the initial default and then setting the
write default vs setting the initial default and then immediately changing it.
But yeah, I think we're in agreement mostly.
> I expect both concepts have to be exposed somehow, at least because
sometimes we say we can change it freely (for future rows) and sometimes we say
we can only change it behind an allowIncompatibleChanges API (for existing
rows). If we expose two ways or two concepts of default values, it might be
unnecessarily complex.
In that case, I'd skip allowing users to change the initial default. If
you've really just created a new column and need to update the default, you can
always drop it and re-create it with the right default!
As long as we don't have that, then we can hide everything behind the API.
The initial default is set when creating a field, and this also sets the write
default. Setting the default value for a field only changes the write default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
