twuebi opened a new pull request, #1237: URL: https://github.com/apache/iceberg-go/pull/1237
## Problem Scanning or compacting a v3 table renumbers surviving rows' `_row_id` whenever rows are dropped — by a deletion vector, positional/equality deletes, or a row filter. The spec requires `_row_id = first_row_id + the row's original position in its file`, so survivors must keep their ids, instead they came out as a dense `0..N` sequence. ## Cause Row-lineage synthesis runs last in the scan pipeline, after the row-dropping steps, and counts positions over the already-filtered batch. Independently, Parquet row-group pruning skips whole groups before the batch reaches the pipeline, leaving survivor positions non-contiguous. ## Fix Synthesize `_row_id` / `_last_updated_sequence_number` first, before any row-dropping step, counting over the full unfiltered batch; later filters drop rows already carrying the correct id. Skip row-group pruning while synthesizing lineage so positions stay contiguous — the per-row filter still enforces the predicate. ## Tests `TestRewriteDataFilesPreservesRowIDThroughDeletionVector` (atomic + partial-progress) and `TestScanRowLineagePreservedAcrossPrunedRowGroups` (a filter prunes a leading row group; survivors keep their ids). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
