mbutrovich opened a new pull request, #23576: URL: https://github.com/apache/datafusion/pull/23576
## Which issue does this PR close? - Backport of #22815 to `branch-54` (for 54.1.0, tracked in #22547). - Fixes wrong results #22796 (regression introduced by the inline-u128 hashing fast path in #21064, which is present on `branch-54` as commit 1cb4de41c). ## Rationale for this change `approx_distinct` over-counts `Utf8View` values when the same short (<= 12 byte) string appears across batches with different layouts: an all-inline batch hashes it as a raw `u128` view, while a mixed batch (with a data buffer) hashed it as a `&str`, so the same value is counted twice. ## What changes are included in this PR? This is an **adapted** backport, not a clean cherry-pick. `branch-54` only has the scalar `StringViewHLLAccumulator` path; the group `HllGroupsAccumulator` / `Utf8ViewHasher` code the upstream fix also touched comes from #22768, which is **not** on `branch-54`. - Applied verbatim from #22815: the scalar `StringViewHLLAccumulator::update_batch` mixed-batch fix (per-row length check) and its regression test `utf8view_acc_split_batches_match_single_mixed_batch`. - Omitted as not applicable: the `Utf8ViewHasher` group-accumulator hunk and its group test. - Added: a `#[cfg(test)] mod tests` scaffold and imports, since `branch-54`'s `approx_distinct.rs` had no test module. ## Are these changes tested? Yes. The scalar regression test passes with the fix and fails without it (over-count), confirming it exercises the regression. ## Are there any user-facing changes? `approx_distinct` on `Utf8View` / `VARCHAR VIEW` columns now returns correct (lower) counts. Results may differ from the previously incorrect values. No API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
