lxy-9602 commented on code in PR #525:
URL: https://github.com/apache/paimon-cpp/pull/525#discussion_r4492853727
##########
src/paimon/core/utils/primary_key_table_utils.cpp:
##########
@@ -103,13 +103,15 @@ Status
PrimaryKeyTableUtils::ValidateRealtimeOptions(const CoreOptions& options,
if (options.GetBucket() <= 0) {
return Status::NotImplemented("PK realtime requires fixed buckets");
}
- if (options.GetMergeEngine() != MergeEngine::DEDUPLICATE) {
- return Status::NotImplemented("PK realtime supports only the
DEDUPLICATE merge engine");
+ if (options.GetMergeEngine() != MergeEngine::DEDUPLICATE &&
+ options.GetMergeEngine() != MergeEngine::PARTIAL_UPDATE) {
+ return Status::NotImplemented(
+ "PK realtime supports only the DEDUPLICATE and PARTIAL_UPDATE
merge engines");
Review Comment:
With `partial-update` and DVs enabled, pushing value predicates into
high-level disk files can discard rows needed to complete an in-memory partial
update.
For example:
```text
Disk (after compaction): (id=1, a="old", b="keep")
Memory partial update: (id=1, a="new", b=null)
Query: WHERE a = 'new'
Expected: (id=1, a="new", b="keep")
Actual: (id=1, a="new", b=null)
```
The disk row is filtered out before merging, so `b="keep"` is lost.
Please either push down only primary-key predicates across all disk levels
and evaluate value predicates after merging, or reject the combination of
realtime partial-update and DVs for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]