houlinchn opened a new pull request, #11274:
URL: https://github.com/apache/paimon/pull/11274

   ### Purpose
   
   Fixes #11273
   
   A partial-update table with sequence groups and a `sum` aggregate 
double-counts when a merge starts with a retraction. With `amount = 90` 
committed, a later commit of `-U(90), +U(90)` should leave `90` but reads `180`.
   
   Since #7645, `initRow` copies every field of the first retract record. The 
retracted value therefore becomes the starting accumulator, `retract(90, 90)` 
cancels it, and the `+U` is added on top.
   
   This change:
   
   - Initializes only NOT NULL fields when a sequence-group merge starts with a 
retraction, and leaves retractable aggregate fields (aggregators in a sequence 
group that are not `ignore-retract`) empty. A leading `-U(90)` now contributes 
`-90`, as it did before #7645.
   - With `ignore-delete`, keeps the retract payload when a key only has 
retractions, so the DELETE result is unchanged, and resets the row before the 
first positive record so the ignored value is not aggregated.
   - Leaves the whole-row delete paths 
(`partial-update.remove-record-on-delete`, 
`partial-update.remove-record-on-sequence-group`) on the full `initRow` that 
#7645 introduced.
   
   ### Tests
   
   - `PartialUpdateFirstRetractTest` (merge-function level):
     - a leading `UPDATE_BEFORE` or `DELETE` followed by `+U`, then compaction 
with the old value;
     - NOT NULL aggregate fields;
     - consecutive retractions;
     - a null sequence group;
     - `ignore-retract` fields;
     - `ignore-delete` with historical retractions, including a key with only 
retractions;
     - `remove-record-on-delete` payloads.
   - `PartialUpdateFirstRetractTableTest` (table level): commits `+I(90)` and 
then `-U(90), +U(90 | 60)`. It checks the data read and the changelog for 
`changelog-producer` = `none`, `input` and `lookup`.
   
   Results on JDK 11:
   
   - On master `df565c64f7` without the main-code change, 14 of the 16 new 
cases fail. The table test reads `amount = 180` instead of `90` with all three 
changelog producers. The other 2 cases pass on master too; they are 
compatibility controls for the DELETE-only payload and 
`remove-record-on-delete`.
   - With the change, `PartialUpdateFirstRetractTest`, 
`PartialUpdateFirstRetractTableTest`, `PartialUpdateMergeFunctionTest` and 
`LookupChangelogMergeFunctionWrapperTest` pass: 94 tests. The run used Maven 
`verify` with `-pl paimon-core -am`, which also passed Checkstyle, Spotless, 
Enforcer and RAT.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to