emkornfield commented on code in PR #13936: URL: https://github.com/apache/iceberg/pull/13936#discussion_r2310813773
########## format/spec.md: ########## @@ -1861,6 +1861,12 @@ Java writes `-1` for "no current snapshot" with V1 and V2 tables and considers t Some implementations require that GZIP compressed files have the suffix `.gz.metadata.json` to be read correctly. The Java reference implementation can additionally read GZIP compressed files with the suffix `metadata.json.gz`. +### Schema evolution and writing with old schemas + +Writers should write out all fields with the types specified from the table schema. Writers may use an outdated schema for insert and upsert (replace an entire row) operations. Writers may not use a outdated schema for an update where some cells of a new row come from an existing row (e.g. `update table set col_y = 'xyz'`). For these types of updates old schemas would potentially allow for dropping data for new columns added to the schema. Allowing writes with stale schemas implies that changes to `write-default` values may not be reflected immediately after the change is committed to the table. Review Comment: reworded this to try to bring this to the top. I used 'must' for writing out all values, and 'should' for using the latest schema because there seemed to be consensus that in some cases using an old schema would be OK. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
