dragonknightk opened a new pull request, #11178:
URL: https://github.com/apache/paimon/pull/11178

   ### Purpose
   
   Fixes #6862
   
   With `partial-update.remove-record-on-delete`, the merge result now becomes 
DELETE when the last message of a key is a lone UPDATE_BEFORE (-U). This is a 
behavior contract change.
   
   On -U the merge function keeps the merged row, while DELETE re-initializes 
the row from its own value. A -U only carries one writer's view, so 
re-initializing from it would let a following +U drop columns written by other 
writers.
   
   The trigger is a common Flink pattern: `INSERT INTO t SELECT * FROM src 
WHERE delete = 0`. When a row flips to `delete = 1`, the filter passes the -U 
and drops the +U. Before this change, the merge function only reacted to 
DELETE, so the -U was ignored and the stale row survived.
   
   Old and new results within one merge for `+I(1, 10, null)` and `+I(1, null, 
20)` from two writers, then `-U(1, 10, null)`:
   
   ```
   old: -U ignored           -> +I(1, 10, 20)  stale row
   new: -U                   -> -D(1, 10, 20)  merged row kept in the DELETE
   new: then +U(1, 11, null) -> +I(1, 11, 20)  f2 from the other writer kept
   ```
   
   - With `partial-update.remove-record-on-sequence-group`, the merge result 
also becomes DELETE after an accepted -U of a configured group. The group 
fields are retracted as before instead of re-initialized from the -U.
   - A later accepted +I or +U restores the row, under the same rules as after 
a DELETE.
   - When -U and +U carry different partition values, they reach different 
keys, so the merge result in the old partition becomes DELETE.
   - `aggregation.remove-record-on-delete` is unchanged. Its docs define -U as 
a per-field retraction, so changing it is outside issue 6862 and needs its own 
discussion.
   
   The root cause was confirmed in the review of #6844, which was closed 
without merging after a rebase request.
   
   ### Tests
   
   - Without sequence groups, the merge result becomes DELETE after a lone -U 
and carries the merged row. A following +U in the same merge restores the row 
on top of the merged row.
   - With a configured sequence group, the merge result becomes DELETE after an 
accepted lone -U. The row kind stays INSERT after an older -U of that group or 
a -U of another group. An accepted +U restores the row.
   - At table level, a read after a committed lone -U returns no row for both 
options.
   - In a chain table, the merge result becomes DELETE in the partition of a -U 
whose +U lands in another partition.
   - With `remove-record-on-delete`, a key that receives only one -U and no 
prior INSERT merges to DELETE. This already holds on the code before this 
change, and a test locks it in. Every other new test and each test with a 
changed expectation fails on the code before this change.
   
   ## AI Disclosure
   
   Human-led contribution. Grok Bot, working for dragonknightk, assisted with 
this change, including the code, tests, and description. All submitted content 
was reviewed by a human before submission. (๑•̀ㅂ•́)و✧
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to