JingsongLi commented on code in PR #10952:
URL: https://github.com/apache/paimon/pull/10952#discussion_r4451428130
##########
paimon-core/src/main/java/org/apache/paimon/mergetree/compact/aggregate/AggregateMergeFunction.java:
##########
@@ -81,12 +81,19 @@ public void reset() {
public void add(KeyValue kv) {
latestKv = kv;
- currentDeleteRow = removeRecordOnDelete && kv.valueKind() ==
RowKind.DELETE;
- if (currentDeleteRow) {
+ if (removeRecordOnDelete && kv.valueKind() == RowKind.DELETE) {
+ currentDeleteRow = true;
row = new GenericRow(getters.length);
initRow(row, kv.value());
return;
}
+ if (currentDeleteRow) {
+ // The record was removed by the previous delete, so later records
start a new
+ // aggregation instead of accumulating onto the deleted record's
values.
+ currentDeleteRow = false;
Review Comment:
[P1] Preserve the delete barrier through buffer preaggregation
Resetting this accumulator is insufficient when DELETE and the first
replacement INSERT are in the same checkpoint.
`MergeTreeWriter.flushWriteBuffer` invokes this function before persisting that
buffer, so `getResult()` emits an ordinary INSERT and loses the fact that older
aggregates must be discarded. I reproduced this by adapting the new table test
with supported `write-only=true`: commit `(pt=1,a=1,b=10,c=1)`, then buffer its
DELETE and `(1,1,3,7)` together before the next prepareCommit, then commit
`(1,1,4,8)`. The actual table read returns `+I[1,1,17,1]` instead of
`+I[1,1,7,7]`; putting DELETE in its own commit passes the identical control.
Thus the claimed delete/reinsert fix still depends on checkpoint grouping, and
old sum/first_value values survive. Please preserve reset/tombstone semantics
through buffer reduction and partial compaction, or avoid collapsing this
boundary until older versions participate, and add a same-checkpoint
persistence regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]