kumarpritam863 commented on PR #15209: URL: https://github.com/apache/iceberg/pull/15209#issuecomment-3919027881
@bryanck Totally agree with the core point, but I think the default value in the schema exists primarily so that — when the destination table lacks a value for a column — it falls back to this default. This aligns closely with the original intent behind introducing initial-default and write-default in Apache Iceberg: initial-default fills in values for existing/historical rows (those written before the column was added), without requiring a data rewrite. write-default applies to new writes when the writer doesn't supply a value (and this one can be updated later via schema evolution). In practice, we should support properly reflecting these defaults during reads / schema projection / merging — otherwise, we're losing an important piece of the schema contract. One idea I've been thinking about: what if we dynamically update a column's effective default when the schema's default changes? That way, the system stays in sync with schema evolution without manual intervention. A key observation though: in real-world usage, these defaults rarely change after being set — they tend to remain constant over the table's lifetime. So the overhead / complexity of handling dynamic default changes should be quite low in most cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
