karubian opened a new pull request, #19036:
URL: https://github.com/apache/iceberg/pull/19036

   Fixes #19035. `rewrite_position_delete_files` keeps only the position 
deletes whose data file appears in a `data_files` scan filtered to the file 
group's partition. That scan skips manifests by their partition summaries, and 
Iceberg before 1.11.0 could write wrong bounds for binary partition fields, 
#15128, so a live data file can look gone and its deletes are dropped from the 
current snapshot. The runner now leaves binary partition fields out of that 
filter; fields of other types keep it. On a table partitioned only by binary 
fields, every file group reads all data manifests and joins against all live 
data files, which can turn the broadcast join into a shuffle. Spark 3.5, 4.0 
and 4.1 follow in a backport PR.
   
   Test plan: on a v2 table whose binary partition summary excludes its own 
data file, the rewrite keeps the delete of that live file and drops the delete 
of a removed file in the same group.
   
   ---
   **AI Disclosure**
   - Model: Claude Opus 5.5
   - Platform/Tool: Claude Code
   - Human Oversight: [unknown - human to fill in]
   - Prompt Summary: Investigate a position delete that 
`rewrite_position_delete_files` dropped in the author's correctness test suite, 
then write a regression test and a fix in `SparkRewritePositionDeleteRunner`. 
The runner change, the test and this description are generated.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to