JeonDaehong opened a new pull request, #1338: URL: https://github.com/apache/paimon-rust/pull/1338
### Purpose Linked issue: part of #1336 `last_value`, `first_value`, `last_non_null_value`, `first_non_null_value` and `primary-key` hold the current winner as a one-row slice and take a new slice for every input row. Columns without an aggregate function default to `last_non_null_value`, so almost every aggregation table pays this per row. ### Brief change log - `FieldAggregator::agg_array(&ArrayRef, row)` defaults to `agg`. The pick aggregators override it to keep the input array and row index, and `result()` slices once. - `merge_ordered` and the partial-update aggregator path pass the column as an `ArrayRef`. Columns without an aggregator keep the existing path. PK table, 2M keys written in 3 overlapping rounds, `sum` on one column, full read (release, in-memory FileIO): 2,105–2,135 → 1,827–1,856 ms, measured in one binary with the old and new call paths switched by an env var, 3 alternating rounds. Allocations per merged key go from 10.2 to 8.2. Deduplicate and partial-update are unchanged. ### Tests - `agg_array_matches_agg_for_every_policy`: each pick policy gives the same result through `agg_array` and `agg`, with NULLs among the inputs. - `agg_array_winner_feeds_reversed_and_delete_paths`: `agg_reversed` and `replace_with_delete` after `agg_array`. Ran the ci.yml steps locally (fmt, clippy, msrv, unit, rest-server, integration, datafusion, python, go). ### API and Format None. `agg_array` is crate-internal. ### Documentation None. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
