JeonDaehong opened a new pull request, #1370:
URL: https://github.com/apache/paimon-rust/pull/1370

   ### Purpose
   
   Linked issue: none, follows #1164
   
   #1164 reads a single-run section without the sort merge for deduplicate and 
first-row. Aggregation can take the same path: when a key has one version, the 
merge function already returns that row as is (`singleton_reducer_result`) and 
omits a lone retract, which is what the unmerged path does per batch. The 
exception is `changelog-producer = lookup`, where aggregation reduces 
singletons too, so it keeps merging.
   
   ### Brief change log
   
   - The single-run check also accepts aggregation unless `changelog-producer = 
lookup`.
   - `reduces_singletons(options)` holds that rule, and the three merge 
functions that had it inline use it.
   
   PK table, 2M keys in 4 files with disjoint keys, `sum` on one column, full 
read (release, in-memory FileIO): 187–194 → 89–90 ms, the same as deduplicate. 
Measured in one binary with the merge path switched back by an env var, 3 
alternating rounds. With overlapping rounds (the merge still runs) and for 
deduplicate, times are unchanged.
   
   ### Tests
   
   - `aggregation_reads_disjoint_files_without_merge_and_overlaps_still_merge`: 
disjoint files read as written, an overlapping file still sums.
   - `single_run_read_merges_repeated_keys_under_aggregation`: a pre-#839 file 
that repeats a key still sums.
   - `single_run_read_drops_retracts_hidden_by_legacy_metadata` gets an 
aggregation case.
   
   The new tests also pass on main, so the output matches the merge path. Ran 
the ci.yml steps locally (fmt, clippy, msrv, unit, rest-server, integration, 
datafusion, python, go).
   
   ### API and Format
   
   None.
   
   ### Documentation
   
   None.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to