Rachelint opened a new pull request, #23384: URL: https://github.com/apache/datafusion/pull/23384
## Which issue does this PR close? - N/A ## Rationale for this change Final aggregation commonly receives near-unique `StringView` group keys from partial aggregation. For those inputs, copying every non-inline string into compact buffers adds overhead while providing little compaction benefit. ## What changes are included in this PR? - Add a `GroupValues::mark_near_unique` hook. - Add final-stage sampling for single-column `Utf8View` group keys: after sampling 100,000 rows, mark group values near-unique when `group_values.len() / sampled_rows > 0.8`. - Teach single-column `StringView` group values to retain cloned input buffers in near-unique mode and remap appended views to those buffers. - Add coverage verifying near-unique `StringView` interning retains the input data buffer. ## Are these changes tested? Yes: ```bash cargo fmt --all cargo test -p datafusion-physical-expr-common binary_view_map::tests::test_bytes_view_map_mark_near_unique_retains_input_buffer cargo test -p datafusion-physical-plan group_values cargo clippy --all-targets --all-features -- -D warnings ``` ## Are there any user-facing changes? No. This is an internal performance optimization. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
