RussellSpitzer commented on code in PR #13936: URL: https://github.com/apache/iceberg/pull/13936#discussion_r2308659031
########## format/spec.md: ########## @@ -1861,6 +1861,12 @@ Java writes `-1` for "no current snapshot" with V1 and V2 tables and considers t Some implementations require that GZIP compressed files have the suffix `.gz.metadata.json` to be read correctly. The Java reference implementation can additionally read GZIP compressed files with the suffix `metadata.json.gz`. +### Schema evolution and writing with old schemas + +Writers should write out all fields with the types specified from the table schema. Writers may use an outdated schema for insert and upsert (replace an entire row) operations. Writers may not use a outdated schema for an update where some cells of a new row come from an existing row (e.g. `update table set col_y = 'xyz'`). For these types of updates old schemas would potentially allow for dropping data for new columns added to the schema. Allowing writes with stale schemas implies that changes to `write-default` values may not be reflected immediately after the change is committed to the table. Review Comment: I think the last line here is the most important one. So I'd try to make that more prominent. `Writes should not be preformed with non-current schemas. Column initial defaults rely on missing columns for proper resolution so writing a new file without all specified columns will apply initial defaults instead of write defaults." Something like that, then put the notes about existing rows after that -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
