Rich-T-kid opened a new pull request, #26250:
URL: https://github.com/apache/datafusion/pull/26250
## Which issue does this PR close?
<!-- none; this adds reproducibility tooling. Happy to open a tracking issue
first if preferred. -->
- Closes #.
## Rationale for this change
Several recent PRs that touch dictionary / `Utf8View` operator paths have had
to answer the same question: *"at what cardinality does dictionary encoding
stop being a win over `Utf8View`?"* There is no in-tree harness that answers
this reproducibly — contributors have been spinning up ad-hoc scripts, each
measuring slightly different things.
This PR adds `dict_encoding_sweep`: a focused benchmark driver that generates
two in-memory tables sharing identical values (one `Utf8View`, one
`Dictionary<Int32, Utf8>`), runs the same SQL against each, and reports
median/min/p25/p75 runtimes to CSV. One invocation sweeps across an NDV
grid for a chosen operator group so the crossover point is directly visible.
## What changes are included in this PR?
- `benchmarks/src/dict_encoding_sweep.rs` — harness library (deterministic
data gen, interleaved warmup/timing loop, per-group query catalog, CSV
writer).
- `benchmarks/src/bin/dict_encoding_sweep.rs` — CLI driver; one invocation
handles one `(group, rows, width, nulls, batch_size)` combination and
sweeps across the provided `--ndvs`.
- `benchmarks/README.md` — new *Dictionary vs Utf8View Encoding Sweep*
section documenting operator groups, flags, and CSV schema.
Nothing under `datafusion/` is touched.
## What is the testing strategy for this PR?
The sweep binary is itself the artifact — a smoke run (`--rows 100000 --ndvs
10,1000 --iters 3 --warmups 1`) completes end-to-end and writes a valid CSV.
No new unit tests are added; the harness is a self-contained reproduction
tool rather than a code path exercised by production queries. `cargo fmt
--all --check` and `cargo clippy -p datafusion-benchmarks --lib --bins
--all-features -- -D warnings` both pass on the new files.
## Are there any user-facing changes?
No. New files under `benchmarks/`.
## Results
Numbers below come from running the harness on this branch, 2026-10-12, with
`--iters 15 --warmups 3 --dict-chunk 100000 --ndvs
10,100,500,1000,4000,8192`.
Cells are **`dict median / view median`** — values `< 1.00` mean dictionary
is faster, `> 1.00` means `Utf8View` wins. Variance on this machine is
roughly ±20 %; treat differences inside that band as noise. Full CSVs are
reproducible from the commands in `benchmarks/README.md`.
### filter — 1,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|--------------------|------|------|------|------|------|------|
| filter_1pct_group | 0.43 | 0.54 | 0.24 | 0.29 | 0.53 | 1.05 |
| filter_eq | 0.29 | 0.49 | 0.63 | 0.72 | 1.45 | 2.41 |
| filter_gt | 0.36 | 0.65 | 0.79 | 0.92 | 1.82 | 3.05 |
| filter_in10 | 0.09 | 0.10 | 0.18 | 0.25 | 0.62 | 1.13 |
| filter_like_prefix | 0.26 | 0.30 | 0.30 | 0.35 | 0.60 | 0.95 |
| filter_ne | 0.30 | 0.49 | 0.62 | 0.74 | 1.41 | 2.37 |
### filter — 5,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|--------------------|------|------|------|------|------|------|
| filter_1pct_group | 0.41 | 0.46 | 0.24 | 0.28 | 0.53 | 1.01 |
| filter_eq | 0.28 | 0.39 | 0.57 | 0.73 | 1.41 | 2.32 |
| filter_gt | 0.34 | 0.54 | 0.75 | 0.94 | 1.75 | 3.01 |
| filter_in10 | 0.09 | 0.10 | 0.18 | 0.25 | 0.62 | 1.11 |
| filter_like_prefix | 0.25 | 0.31 | 0.30 | 0.35 | 0.59 | 0.93 |
| filter_ne | 0.29 | 0.41 | 0.60 | 0.73 | 1.40 | 2.36 |
### aggregation — 1,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|----------------------|------|------|------|------|------|------|
| agg_count_distinct_k | 0.70 | 0.78 | 0.71 | 0.71 | 0.79 | 0.91 |
| agg_group_count_k | 0.75 | 0.81 | 0.75 | 0.74 | 0.79 | 0.94 |
| agg_group_count_kw | 0.70 | 0.72 | 0.69 | 0.71 | 0.82 | 0.90 |
| agg_group_sum_v_k | 0.76 | 0.83 | 0.75 | 0.74 | 0.80 | 0.92 |
| agg_having_k | 0.75 | 0.81 | 0.76 | 0.74 | 0.79 | 0.94 |
### aggregation — 5,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|----------------------|------|------|------|------|------|------|
| agg_count_distinct_k | 0.70 | 0.79 | 0.71 | 0.71 | 0.77 | 0.89 |
| agg_group_count_k | 0.75 | 0.81 | 0.73 | 0.74 | 0.78 | 0.92 |
| agg_group_count_kw | 0.70 | 0.72 | 0.69 | 0.71 | 0.80 | 0.86 |
| agg_group_sum_v_k | 0.75 | 0.82 | 0.75 | 0.74 | 0.77 | 0.91 |
| agg_having_k | 0.75 | 0.81 | 0.74 | 0.73 | 0.78 | 0.92 |
### sort — 1,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|-------------------|------|------|------|------|------|------|
| sort_k | 0.50 | 0.58 | 0.61 | 0.64 | 0.90 | 1.36 |
| sort_k_desc | 0.50 | 0.59 | 0.65 | 0.67 | 0.95 | 1.41 |
| sort_k_limit_1000 | 0.35 | 0.65 | 0.80 | 1.12 | 1.46 | 1.91 |
| sort_kw | 1.02 | 1.75 | 1.71 | 1.65 | 1.60 | 1.66 |
| sort_v_by_k | 0.54 | 0.50 | 0.71 | 0.73 | 1.11 | 1.87 |
### sort — 5,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|-------------------|------|------|------|------|------|------|
| sort_k | 0.51 | 0.74 | 0.72 | 0.74 | 1.11 | 1.86 |
| sort_k_desc | 0.50 | 0.74 | 0.73 | 0.75 | 1.15 | 1.89 |
| sort_k_limit_1000 | 0.31 | 0.57 | 0.76 | 0.90 | 1.79 | 3.68 |
| sort_kw | 1.02 | 3.16 | 2.73 | 2.63 | 2.60 | 2.60 |
| sort_v_by_k | 0.56 | 1.15 | 0.99 | 0.95 | 1.50 | 2.81 |
### project — 1,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|----------------|------|------|------|------|------|------|
| proj_case_like | 0.40 | 0.46 | 0.46 | 0.51 | 0.73 | 0.93 |
| proj_concat | 0.91 | 0.92 | 0.92 | 0.91 | 0.92 | 0.94 |
| proj_length | 0.03 | 0.03 | 0.04 | 0.06 | 0.15 | 0.34 |
| proj_substr | 1.71 | 1.73 | 1.71 | 1.75 | 1.75 | 1.74 |
| proj_upper | 0.01 | 0.01 | 0.01 | 0.02 | 0.04 | 0.09 |
### project — 5,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|----------------|------|------|------|------|------|------|
| proj_case_like | 0.42 | 0.47 | 0.47 | 0.52 | 0.77 | 0.99 |
| proj_concat | 0.92 | 0.92 | 0.92 | 0.90 | 0.92 | 0.92 |
| proj_length | 0.03 | 0.03 | 0.04 | 0.07 | 0.17 | 0.37 |
| proj_substr | 1.75 | 1.74 | 1.74 | 1.74 | 1.74 | 1.75 |
| proj_upper | 0.01 | 0.01 | 0.01 | 0.01 | 0.04 | 0.08 |
### join — 1,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|--------------------|------|------|------|------|------|------|
| join_anti_half | 0.98 | 1.00 | 0.97 | 0.97 | 1.00 | 1.06 |
| join_inner | 0.94 | 1.05 | 1.00 | 0.99 | 1.05 | 1.24 |
| join_inner_project | 0.94 | 0.98 | 1.00 | 0.96 | 1.04 | 1.23 |
| join_left | 0.95 | 1.02 | 0.95 | 0.95 | 1.05 | 1.24 |
| join_semi | 0.94 | 1.00 | 0.98 | 0.96 | 1.06 | 1.21 |
### join — 5,000,000 rows
| query \ ndv | 10 | 100 | 500 | 1000 | 4000 | 8192 |
|--------------------|------|------|------|------|------|------|
| join_anti_half | 0.98 | 1.01 | 0.96 | 0.98 | 0.99 | 1.05 |
| join_inner | 0.97 | 1.04 | 1.00 | 0.96 | 1.06 | 1.26 |
| join_inner_project | 0.93 | 1.01 | 0.96 | 0.96 | 1.05 | 1.21 |
| join_left | 0.98 | 1.00 | 0.98 | 0.96 | 1.06 | 1.24 |
| join_semi | 0.97 | 1.00 | 0.99 | 0.99 | 1.05 | 1.21 |
Headline takeaways consistent across both row counts:
- **Filter / projection / aggregation at low NDV**: dictionary dominates
(ratios 0.03–0.50); the crossover against `Utf8View` sits around NDV ≈ 1k.
- **Sort**: dictionary wins at low NDV but loses from NDV ≈ 4k upward; the
`sort_kw` ties-breaker column exposes a large regression (~1.7–3×).
- **Join**: nearly a wash at low NDV; mild `Utf8View` edge (~1.05–1.26×)
past NDV 4k.
- **proj_substr**: `Utf8View` is consistently faster regardless of NDV,
because `substr` on a view can skip materialization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
