laskoviymishka opened a new pull request, #2622:
URL: https://github.com/apache/iceberg-go/pull/2622

   UpdateSchema only asserted the current schema ID, so with retries on by 
default since #2462 a stale writer could replay an AddSchema that reuses a 
field ID another writer had assigned and then dropped. Dropping the column 
returns the table to its old schema ID, so the schema assertion still passes 
after refresh, and the new column reads the dropped column's data. When 
UpdateSchema adds a schema, it now also sends AssertLastAssignedFieldID with 
the base last-column-id, which matches Java's UpdateRequirements. The 
requirement type, its REST encoding, local validation in the SQL, Glue, Hive, 
Hadoop and DynamoDB catalogs, and the retry path's check of non-rebased 
requirements were already on main, so the fix is a few lines in BuildUpdates. 
Fixes #2609.
   
   The new SQL catalog test is Badal's two-writer repro. Writer A stages adding 
x. Writer B adds y, writes 42 to it and drops it. A then commits with default 
retries. The commit now fails with ErrCommitFailed, nothing is committed, and a 
scan never returns 42. Without the fix, the commit succeeds. A BuildUpdates 
unit test checks that the requirement carries the base's last-column-id, and 
another commits two column additions in one transaction and checks that they 
stage a single assertion of that base value. The existing test for switching 
back to an existing schema still expects only the schema ID assertion. The JSON 
round-trip for assert-last-assigned-field-id is already covered in 
table/requirement_test.go. go build, go vet and the table and catalog tests 
pass.
   
   One edge case is left open. When stored metadata has a stale last-column-id 
(lower than a field ID in schema history), catalogs check the stored value but 
the transaction checks the builder's raised value, so the assertion is skipped 
there to keep those tables writable. The first commit writes the raised 
counter, so later updates are protected. A possible follow-up is to re-run the 
UpdateSchema ops against the refreshed base and reassign IDs instead of failing.
   
   AI-assisted, reviewed by a human.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to