tustvold commented on PR #5851: URL: https://github.com/apache/arrow-datafusion/pull/5851#issuecomment-1497728039
Using the benchmarks in https://github.com/apache/arrow-datafusion/pull/5881 ``` merge sorted i64 time: [9.3364 ms 9.3444 ms 9.3527 ms] change: [+1.8876% +2.0141% +2.1272%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 5 (5.00%) high mild sort merge i64 time: [10.242 ms 10.275 ms 10.308 ms] change: [+8.8783% +9.2851% +9.6918%] (p = 0.00 < 0.05) Performance has regressed. sort i64 time: [14.140 ms 14.159 ms 14.179 ms] change: [+107.76% +108.22% +108.67%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 4 (4.00%) high mild 1 (1.00%) high severe sort partitioned i64 time: [1.0789 ms 1.1030 ms 1.1282 ms] change: [+88.867% +135.91% +186.60%] (p = 0.00 < 0.05) Performance has regressed. Found 4 outliers among 100 measurements (4.00%) 1 (1.00%) high mild 3 (3.00%) high severe merge sorted f64 time: [9.3495 ms 9.3668 ms 9.3846 ms] change: [+2.1584% +2.4336% +2.6916%] (p = 0.00 < 0.05) Performance has regressed. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild sort merge f64 time: [10.088 ms 10.118 ms 10.147 ms] change: [+7.6324% +8.0106% +8.3531%] (p = 0.00 < 0.05) Performance has regressed. sort f64 time: [14.030 ms 14.055 ms 14.083 ms] change: [+89.060% +89.511% +89.963%] (p = 0.00 < 0.05) Performance has regressed. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high severe Benchmarking sort partitioned f64: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.0s, enable flat sampling, or reduce sample count to 70. sort partitioned f64 time: [1.0817 ms 1.1193 ms 1.1585 ms] change: [+97.943% +151.36% +215.06%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe merge sorted utf8 low cardinality time: [6.5726 ms 6.5842 ms 6.5967 ms] change: [+1.1399% +1.3642% +1.6082%] (p = 0.00 < 0.05) Performance has regressed. Found 4 outliers among 100 measurements (4.00%) 2 (2.00%) high mild 2 (2.00%) high severe sort merge utf8 low cardinality time: [7.7322 ms 7.7734 ms 7.8145 ms] change: [+7.3910% +8.1105% +8.7923%] (p = 0.00 < 0.05) Performance has regressed. sort utf8 low cardinality time: [12.349 ms 12.367 ms 12.388 ms] change: [+37.984% +38.424% +38.870%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe Benchmarking sort partitioned utf8 low cardinality: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.7s, enable flat sampling, or reduce sample count to 60. sort partitioned utf8 low cardinality time: [1.4059 ms 1.4462 ms 1.4889 ms] change: [+43.382% +70.074% +100.65%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high severe merge sorted utf8 high cardinality time: [10.614 ms 10.682 ms 10.759 ms] change: [+1.6235% +2.2767% +3.0579%] (p = 0.00 < 0.05) Performance has regressed. Found 17 outliers among 100 measurements (17.00%) 1 (1.00%) high mild 16 (16.00%) high severe sort merge utf8 high cardinality time: [12.340 ms 12.403 ms 12.467 ms] change: [+4.1153% +4.9344% +5.7495%] (p = 0.00 < 0.05) Performance has regressed. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild sort utf8 high cardinality time: [17.160 ms 17.276 ms 17.402 ms] change: [-10.181% -9.3309% -8.5146%] (p = 0.00 < 0.05) Performance has improved. Found 12 outliers among 100 measurements (12.00%) 7 (7.00%) high mild 5 (5.00%) high severe Benchmarking sort partitioned utf8 high cardinality: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.4s, enable flat sampling, or reduce sample count to 50. sort partitioned utf8 high cardinality time: [1.5979 ms 1.6394 ms 1.6849 ms] change: [+39.021% +68.819% +101.76%] (p = 0.00 < 0.05) Performance has regressed. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high severe merge sorted utf8 tuple time: [17.925 ms 17.957 ms 17.992 ms] change: [+0.1716% +0.4876% +0.7910%] (p = 0.00 < 0.05) Change within noise threshold. Found 5 outliers among 100 measurements (5.00%) 5 (5.00%) high mild sort merge utf8 tuple time: [22.085 ms 22.189 ms 22.292 ms] change: [+2.9040% +3.5604% +4.2505%] (p = 0.00 < 0.05) Performance has regressed. sort utf8 tuple time: [31.425 ms 31.555 ms 31.685 ms] change: [-51.893% -51.637% -51.382%] (p = 0.00 < 0.05) Performance has improved. sort partitioned utf8 tuple time: [3.0074 ms 3.1288 ms 3.2508 ms] change: [+16.143% +22.209% +28.961%] (p = 0.00 < 0.05) Performance has regressed. merge sorted utf8 dictionary time: [5.9205 ms 5.9264 ms 5.9327 ms] change: [+0.0729% +0.2192% +0.3634%] (p = 0.00 < 0.05) Change within noise threshold. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild sort merge utf8 dictionary time: [6.7928 ms 6.8186 ms 6.8448 ms] change: [+11.083% +11.644% +12.191%] (p = 0.00 < 0.05) Performance has regressed. sort utf8 dictionary time: [10.554 ms 10.571 ms 10.588 ms] change: [+95.615% +96.620% +97.550%] (p = 0.00 < 0.05) Performance has regressed. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe Benchmarking sort partitioned utf8 dictionary: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.5s, enable flat sampling, or reduce sample count to 60. sort partitioned utf8 dictionary time: [1.1748 ms 1.2139 ms 1.2542 ms] change: [+71.229% +128.47% +201.74%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 2 (2.00%) high mild 3 (3.00%) high severe merge sorted utf8 dictionary tuple time: [8.5027 ms 8.5127 ms 8.5234 ms] change: [+0.2938% +0.4564% +0.6329%] (p = 0.00 < 0.05) Change within noise threshold. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe sort merge utf8 dictionary tuple time: [10.015 ms 10.075 ms 10.136 ms] change: [+3.0689% +4.0106% +4.9160%] (p = 0.00 < 0.05) Performance has regressed. sort utf8 dictionary tuple time: [17.383 ms 17.407 ms 17.432 ms] change: [-28.942% -28.653% -28.362%] (p = 0.00 < 0.05) Performance has improved. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild Benchmarking sort partitioned utf8 dictionary tuple: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 8.5s, enable flat sampling, or reduce sample count to 50. sort partitioned utf8 dictionary tuple time: [1.7916 ms 1.8438 ms 1.8995 ms] change: [+11.399% +19.720% +29.069%] (p = 0.00 < 0.05) Performance has regressed. Found 2 outliers among 100 measurements (2.00%) 1 (1.00%) high mild 1 (1.00%) high severe merge sorted mixed dictionary tuple time: [15.183 ms 15.203 ms 15.227 ms] change: [+1.1594% +1.3422% +1.5391%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe sort merge mixed dictionary tuple time: [17.153 ms 17.220 ms 17.285 ms] change: [+4.3339% +4.9897% +5.6123%] (p = 0.00 < 0.05) Performance has regressed. sort mixed dictionary tuple time: [25.322 ms 25.384 ms 25.454 ms] change: [-57.042% -56.886% -56.727%] (p = 0.00 < 0.05) Performance has improved. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe Benchmarking sort partitioned mixed dictionary tuple: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 9.3s, enable flat sampling, or reduce sample count to 50. sort partitioned mixed dictionary tuple time: [1.9511 ms 1.9985 ms 2.0512 ms] change: [+5.3542% +12.351% +20.206%] (p = 0.00 < 0.05) Performance has regressed. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high severe merge sorted mixed tuple time: [17.414 ms 17.437 ms 17.461 ms] change: [+0.8745% +1.0492% +1.2293%] (p = 0.00 < 0.05) Change within noise threshold. Found 6 outliers among 100 measurements (6.00%) 6 (6.00%) high mild sort merge mixed tuple time: [20.549 ms 20.620 ms 20.693 ms] change: [+4.5839% +5.0752% +5.5380%] (p = 0.00 < 0.05) Performance has regressed. sort mixed tuple time: [26.823 ms 26.899 ms 26.977 ms] change: [-9.2725% -8.8104% -8.3425%] (p = 0.00 < 0.05) Performance has improved. sort partitioned mixed tuple time: [2.3462 ms 2.4398 ms 2.5338 ms] change: [+40.731% +50.245% +60.230%] (p = 0.00 < 0.05) Performance has regressed. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
