andygrove commented on PR #3049:
URL:
https://github.com/apache/datafusion-comet/pull/3049#issuecomment-3716273510
New output after re-adding `i8` and `i16` benchmarks. Note that the
benchmark results do fluctuate between runs, which is not ideal.
```
cast_string_to_int/legacy/i8
time: [6.6654 µs 6.6882 µs 6.7107 µs]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
cast_string_to_int/legacy/i16
time: [6.8855 µs 7.0007 µs 7.1618 µs]
Found 10 outliers among 100 measurements (10.00%)
5 (5.00%) high mild
5 (5.00%) high severe
cast_string_to_int/legacy/i32
time: [11.650 µs 11.695 µs 11.741 µs]
change: [−6.9011% −4.5301% −2.4419%] (p = 0.00 <
0.05)
Performance has improved.
cast_string_to_int/legacy/i64
time: [12.295 µs 12.330 µs 12.364 µs]
change: [+5.2149% +5.6287% +6.0241%] (p = 0.00 <
0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
cast_string_to_int/ansi/i8
time: [5.7430 µs 5.7633 µs 5.7841 µs]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
cast_string_to_int/ansi/i16
time: [5.7197 µs 5.7496 µs 5.7824 µs]
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
cast_string_to_int/ansi/i32
time: [10.469 µs 10.866 µs 11.369 µs]
change: [−23.119% −17.167% −10.789%] (p = 0.00 <
0.05)
Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) high mild
8 (8.00%) high severe
cast_string_to_int/ansi/i64
time: [10.262 µs 10.302 µs 10.344 µs]
change: [−5.2336% −1.9457% +1.0536%] (p = 0.25 >
0.05)
No change in performance detected.
cast_string_to_int/try/i8
time: [6.0008 µs 6.0224 µs 6.0429 µs]
cast_string_to_int/try/i16
time: [6.1752 µs 6.4688 µs 6.8385 µs]
Found 12 outliers among 100 measurements (12.00%)
4 (4.00%) high mild
8 (8.00%) high severe
cast_string_to_int/try/i32
time: [10.460 µs 10.506 µs 10.564 µs]
change: [−19.766% −17.645% −15.299%] (p = 0.00 <
0.05)
Performance has improved.
Found 23 outliers among 100 measurements (23.00%)
23 (23.00%) high severe
cast_string_to_int/try/i64
time: [11.172 µs 11.374 µs 11.645 µs]
change: [+13.774% +18.852% +24.117%] (p = 0.00 <
0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high mild
cast_string_to_int/legacy_decimals/i32
time: [12.402 µs 12.437 µs 12.471 µs]
change: [−20.604% −17.705% −15.094%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
cast_string_to_int/legacy_decimals/i64
time: [12.605 µs 12.740 µs 12.928 µs]
change: [−3.4087% −2.5954% −1.6264%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
3 (3.00%) high mild
5 (5.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
