JeonDaehong opened a new pull request, #1370: URL: https://github.com/apache/paimon-rust/pull/1370
### Purpose Linked issue: none, follows #1164 #1164 reads a single-run section without the sort merge for deduplicate and first-row. Aggregation can take the same path: when a key has one version, the merge function already returns that row as is (`singleton_reducer_result`) and omits a lone retract, which is what the unmerged path does per batch. The exception is `changelog-producer = lookup`, where aggregation reduces singletons too, so it keeps merging. ### Brief change log - The single-run check also accepts aggregation unless `changelog-producer = lookup`. - `reduces_singletons(options)` holds that rule, and the three merge functions that had it inline use it. PK table, 2M keys in 4 files with disjoint keys, `sum` on one column, full read (release, in-memory FileIO): 187–194 → 89–90 ms, the same as deduplicate. Measured in one binary with the merge path switched back by an env var, 3 alternating rounds. With overlapping rounds (the merge still runs) and for deduplicate, times are unchanged. ### Tests - `aggregation_reads_disjoint_files_without_merge_and_overlaps_still_merge`: disjoint files read as written, an overlapping file still sums. - `single_run_read_merges_repeated_keys_under_aggregation`: a pre-#839 file that repeats a key still sums. - `single_run_read_drops_retracts_hidden_by_legacy_metadata` gets an aggregation case. The new tests also pass on main, so the output matches the merge path. Ran the ci.yml steps locally (fmt, clippy, msrv, unit, rest-server, integration, datafusion, python, go). ### API and Format None. ### Documentation None. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
