leaves12138 opened a new pull request, #10296:
URL: https://github.com/apache/paimon/pull/10296
### Purpose
Fix whole-row deletion state being lost while merging partial updates.
`PartialUpdateMergeFunction.add()` currently clears `currentDeleteRow` for
every input. After a configured sequence group deletes a row, a subsequent
retraction or an update to another group can clear that state. If an earlier
insert was seen, the merged result becomes `INSERT` again, and lookup
compaction can emit an update changelog instead of a delete.
Preserve the deletion state across inputs and initialize it in `reset()`.
Clear it only for a normal non-sequence-group update or an accepted
`INSERT`/`UPDATE_AFTER` to a group configured in
`partial-update.remove-record-on-sequence-group`. The existing newer-or-equal
group-version comparison still applies. Retractions, unrelated groups, and
older or all-null ordering fields must not revive a deleted row.
Also preserve `partial-update.remove-record-on-delete` across a following
`UPDATE_BEFORE`, while allowing a subsequent insert. Document the
sequence-group deletion and restoration semantics. No on-disk format changes
are introduced.
### Tests
Followed a red/green TDD workflow with synthetic records:
- Before the production fix: 70 tests ran; all 15 new regression cases
failed with an unexpected `INSERT`, while the 55 existing cases passed.
- After the fix: the same 70 tests passed.
- Expanded final verification: 90 tests passed with zero failures, errors,
or skips. This run did not enable `fast-build`, so the normal lifecycle quality
checks were enabled.
Regression coverage includes:
- All 120 orderings of five sequence-group deletes, for both equal and
increasing internal sequence numbers.
- Subsequent `DELETE`/`UPDATE_BEFORE`, unrelated updates, stale
controlling-group updates, and valid reinsertion.
- Carrying a tombstone across a merge boundary and resetting between keys.
- Composite ordering fields and five projections, including an empty and a
reordered projection.
- Lookup changelog generation with the old row supplied either by input or
by lookup: the result remains deleted and emits a single `DELETE`, not an
update pair.
Focused red/green command:
```bash
mvn -B -ntp -pl paimon-core -am -Pfast-build \
-DfailIfNoTests=false -DwildcardSuites=none \
'-Dtest=PartialUpdateMergeFunctionTest,LookupChangelogMergeFunctionWrapperTest'
test
```
Final verification command:
```bash
mvn -B -ntp -pl paimon-core -am \
-DfailIfNoTests=false -DwildcardSuites=none \
'-Dtest=PartialUpdateMergeFunctionTest,LookupChangelogMergeFunctionWrapperTest,LookupMergeFunctionTest,LookupMergeFunctionUnitTest,LookupMergeTreeCompactRewriterTest,PartialUpdateTableTest,PrimaryKeyPartialUpdateBlobTest'
test
```
The full repository and Flink version matrix were not run locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
