adriangb commented on PR #25123: URL: https://github.com/apache/datafusion/pull/25123#issuecomment-5619491260
## Benchmark summary All numbers are from the bench bot on a 12 vCPU `c4a-highmem-16`, against baseline `40488988ad`. A ratio above 1.00 means this PR is faster. Because a single run of any of these suites moves by several percent on its own, every claim below is backed by repeat runs, and by **control runs that compare `40488988ad` against itself**. The control tells you what a ratio of "no change" actually looks like on this machine, which turned out to matter twice. ### `clickbench_extended` — where the adapter is exercised | Query | control (3 runs) | this PR (6 runs) | | --- | --- | --- | | Q14 `COUNT(DISTINCT <string>)` by `SearchPhrase` | 0.97, 0.99, 1.02 | **1.16, 1.14, 1.14, 1.12, 1.09, 1.22** | | Q5 `APPROX_PERCENTILE_CONT` by `ClientIP, WatchID` | 0.98, 1.00, 0.98 | **1.04, 1.06, 1.08, 1.08, 1.04, 1.03** | | Q2 4x `COUNT(DISTINCT)` by `BrowserCountry` | 1.00, 1.00, 1.01 | 0.99, 0.99, 0.99, 0.97, 0.99, 0.99 | Q14 and Q5 are the two queries in the suite whose aggregate has no `GroupsAccumulator` and whose `GROUP BY` has many values. Both are faster in all six runs, by roughly 14% and 5%. Suite totals: 1.032, 1.036, 1.026. Q2 is the query that this change could hurt, because it groups by a column with 110 different values and has four `COUNT(DISTINCT)` aggregates, thus four adapters that each route every batch. It stays level with its control. ### Every other suite: no change | Suite | Runs | Result | | --- | --- | --- | | `clickbench_partitioned` | 3 + 2 controls | 0.969, 0.989, 0.999 against controls of 0.984 and 0.992 | | `tpcds_sf1` | 1 | 0.994 total, 1 faster / 4 slower / 94 no change | | `tpch_sf1` | 1 | 0.998 total, 0 faster / 0 slower / 22 no change | | `external_aggr` | 1 | 0.992 total, 0 faster / 0 slower / 8 no change | `clickbench_partitioned` is worth a word. Its first run came back at 0.969 with 13 queries marked slower, which looks like a regression. It is not one. Two more runs gave 0.989 and 0.999, and the two control runs, which compare `40488988ad` against itself, gave 0.984 and 0.992 with 5 and 6 queries marked slower. That suite moves by that much on its own. No query in it is outside its control band in all three runs. Every aggregate in `tpch` has a `GroupsAccumulator`, so it never reaches the code this PR changes. It is there as a control, and it is flat. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
