jayzhan211 commented on PR #25724:
URL: https://github.com/apache/datafusion/pull/25724#issuecomment-5970191885
On the Q25 "2.04x slower" in the latest run: I don't think it comes from
this PR.
- **Q25 does not aggregate.** It is `SELECT "SearchPhrase" ... WHERE
"SearchPhrase" <> '' ORDER BY "SearchPhrase" LIMIT 10`, planned as
`DataSourceExec → FilterExec → SortExec TopK → SortPreservingMergeExec` with or
without the option. `hash_aggregate_bucket_threshold` is only read inside the
hash aggregation streams, so Q25 runs the same code on both sides.
- **The fastest iteration is unchanged:** 106.12 ms on main, 106.56 ms on
the branch. The 2.04x compares means, and the branch mean is pulled up by slow
iterations (max 498 ms).
- **Main shows the same stalls, in the same place.** In all three runs, one
of the cheap queries right after Q23 (`SELECT *`, the most memory-heavy query)
has a slow iteration, on either side:
| run | main (fastest → slowest of 5) | branch (fastest → slowest of 5) |
|---|---|---|
| 1 | Q24 43 → 177 ms | Q24 42 → 646 ms |
| 2 | Q24 41 → 111 ms, Q26 42 → 126 ms | Q25 104 → 663 ms |
| 3 | **Q24 40 → 789 ms** | Q25 107 → 499 ms |
In this run, main's Q24 stall is larger than the branch's Q25 one. Which
query absorbs the stall changes between runs; what stays the same is that it
follows Q23. Q19 and Q29 get similar outliers on both sides.
- **Locally it does not reproduce.** I ran the whole suite in one process,
as the runner does, alternating the option off and on: 15 iterations of Q25 per
side, no outlier (slowest 98 ms off, 82 ms on).
To separate the option from the binary, I can ask for a run where both sides
use this commit and only the threshold differs (0 vs 262144).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
