pepijnve commented on PR #18152:
URL: https://github.com/apache/datafusion/pull/18152#issuecomment-3446581126
> yeah, sorry, I would really like to try and get the benchmarks more
repeatable but haven't had the time
I didn't mean that as criticism towards the benchmark runs. Only a statement
of fact that I don't have a good explanation.
On a positive note, just for kicks I was comparing the latest state of thing
on my 'project' branch with 50.3.0. Seeing -99.9% is pretty hilarious.
```
case_when 8192x100: CASE WHEN c1 < 0 THEN 0 WHEN c1 < 1000 THEN 1 ... WHEN
c1 < n * 1000 THEN n ELSE...
time: [75.926 µs 76.047 µs 76.175 µs]
change: [-99.935% -99.935% -99.935%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
case_when 8192x100: CASE c1 WHEN 0 THEN 0 WHEN 1 THEN 1 ... WHEN n THEN n
ELSE n + 1 END
time: [24.309 ms 24.372 ms 24.440 ms]
change: [-97.655% -97.647% -97.639%] (p = 0.00 <
0.05)
Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
12 (12.00%) high mild
2 (2.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
