JeonDaehong opened a new pull request, #1338:
URL: https://github.com/apache/paimon-rust/pull/1338

   ### Purpose
   
   Linked issue: part of #1336
   
   `last_value`, `first_value`, `last_non_null_value`, `first_non_null_value` 
and `primary-key` hold the current winner as a one-row slice and take a new 
slice for every input row. Columns without an aggregate function default to 
`last_non_null_value`, so almost every aggregation table pays this per row.
   
   ### Brief change log
   
   - `FieldAggregator::agg_array(&ArrayRef, row)` defaults to `agg`. The pick 
aggregators override it to keep the input array and row index, and `result()` 
slices once.
   - `merge_ordered` and the partial-update aggregator path pass the column as 
an `ArrayRef`. Columns without an aggregator keep the existing path.
   
   PK table, 2M keys written in 3 overlapping rounds, `sum` on one column, full 
read (release, in-memory FileIO): 2,105–2,135 → 1,827–1,856 ms, measured in one 
binary with the old and new call paths switched by an env var, 3 alternating 
rounds. Allocations per merged key go from 10.2 to 8.2. Deduplicate and 
partial-update are unchanged.
   
   ### Tests
   
   - `agg_array_matches_agg_for_every_policy`: each pick policy gives the same 
result through `agg_array` and `agg`, with NULLs among the inputs.
   - `agg_array_winner_feeds_reversed_and_delete_paths`: `agg_reversed` and 
`replace_with_delete` after `agg_array`.
   
   Ran the ci.yml steps locally (fmt, clippy, msrv, unit, rest-server, 
integration, datafusion, python, go).
   
   ### API and Format
   
   None. `agg_array` is crate-internal.
   
   ### Documentation
   
   None.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to