iceberg-kitten opened a new pull request, #2663:
URL: https://github.com/apache/iceberg-go/pull/2663

   ## Why
   
   Compacting v3 files without their inherited data sequence number can reset 
`_last_updated_sequence_number` on unchanged rows. If one task lacks a first 
row ID, the rewrite can also assign new row IDs to the whole group. REST scan 
tasks omit the manifest data sequence number, and a catalog can now require 
this planning path for `iceberg compact`. [Iceberg row lineage 
rules](https://iceberg.apache.org/spec/#row-lineage-assignment) · [Go REST 
decoder](https://github.com/apache/iceberg-go/blob/f7dc20dfcf567302c443cc538839bd6421cc7312/catalog/rest/scan_task_decoder.go#L236-L244)
 · [Server-side planning 
directive](https://github.com/apache/iceberg-go/pull/2162)
   
   ## What
   
   Reject incomplete v3 input groups before they read or write data. Complete 
groups preserve both lineage fields. A transaction can still compact a branch 
whose parent predates row lineage and assign IDs in its first v3 snapshot. [Row 
lineage for upgraded 
tables](https://iceberg.apache.org/spec/#row-lineage-for-upgraded-tables)
   
   ## Implementation
   
   Check the inherited first row ID and data sequence number before starting 
the shared compaction worker. Use the target branch's parent when a transaction 
supplies the branch. Standalone workers use the table's row-ID counter because 
the branch is unknown. Spark verifies that ordinary compaction preserves both 
fields. Java's REST file parser has the same sequence-number limitation. [Spark 
rewrite 
behavior](https://github.com/apache/iceberg/blob/95aac2f4b100db7f8f37563d58d09daf01196d47/spark/v4.1/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteDataFilesAction.java#L2216-L2249)
 · [Java REST file 
parser](https://github.com/apache/iceberg/blob/95aac2f4b100db7f8f37563d58d09daf01196d47/core/src/main/java/org/apache/iceberg/ContentFileParser.java#L124-L188)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to