leaves12138 commented on code in PR #8881:
URL: https://github.com/apache/paimon/pull/8881#discussion_r3663506663
##########
paimon-common/src/main/java/org/apache/paimon/lookup/sort/db/RecordCombiningWriter.java:
##########
@@ -55,12 +55,8 @@ void put(MemorySlice key, byte[] value) throws IOException {
}
if (isTombstone(value)) {
- if (pendingKey != null &&
mergeOperator.canMergeTombstone(pendingKey, key)) {
- pendingKeys.add(MemorySlice.wrap(key.copyBytes()));
- } else {
- flushPending();
- consumer.accept(key, value);
- }
+ flushPending();
Review Comment:
[P1] Please keep synthetic merge tombstones from splitting a logical
ListState group. Partial compaction writes tombstones for the physical keys
absorbed by `RecordCombiningWriter`; treating every such tombstone as an
unconditional boundary means a later TTL compaction can expire one middle
fragment independently while newer values for the same logical list are still
live. I reproduced this on the current head with the real `LocalKvListState`:
TTL = 100 ms; flush four values at t=1000, then four at t=1050, t=1100, and
t=1150 (the default L0 trigger creates L3/L2/L1 partial-compaction runs); full
compact at t=1210. `state.get(1)` returns `[0,1,2,3,4,5,6,7,12,13,14,15]`, so
values 8-11 are lost. The previous `canMergeTombstone` path kept these
synthetic tombstones inside the merge group. Please preserve or identify
synthetic tombstones (or otherwise guarantee whole-logical-list TTL semantics)
and add a multi-round partial-compaction TTL regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
