wesm commented on pull request #7525:
URL: https://github.com/apache/arrow/pull/7525#issuecomment-648270948
OK I'm done twiddling this, here is the latest comparison of the hash
benchmarks versus master with gcc-8:
```
benchmark baseline contender change %
counters
35 UniqueInt64/6 3.391 GiB/sec 21.874 GiB/sec 544.979
{'iterations': 77, 'null_percent': 100.0, 'num_unique': 100.0}
22 UniqueInt64/13 3.407 GiB/sec 21.807 GiB/sec 540.146
{'iterations': 77, 'null_percent': 100.0, 'num_unique': 100000.0}
33 UniqueInt64/5 2.588 GiB/sec 8.809 GiB/sec 240.359
{'iterations': 58, 'null_percent': 99.0, 'num_unique': 100.0}
27 UniqueInt64/12 2.282 GiB/sec 5.873 GiB/sec 157.319
{'iterations': 50, 'null_percent': 99.0, 'num_unique': 100000.0}
30 UniqueUInt8/6 1.763 GiB/sec 2.712 GiB/sec 53.829
{'iterations': 315, 'null_percent': 100.0, 'num_unique': 200.0}
44 BuildDictionary 978.092 MiB/sec 1.401 GiB/sec 46.647
{'iterations': 172, 'null_percent': 11.111047470674487}
8 UniqueInt64/11 613.864 MiB/sec 836.145 MiB/sec 36.210
{'iterations': 13, 'null_percent': 50.0, 'num_unique': 100000.0}
2 UniqueInt64/10 559.826 MiB/sec 743.501 MiB/sec 32.809
{'iterations': 11, 'null_percent': 10.0, 'num_unique': 100000.0}
21 UniqueUInt8/3 434.059 MiB/sec 553.941 MiB/sec 27.619
{'iterations': 76, 'null_percent': 10.0, 'num_unique': 200.0}
25 UniqueUInt8/4 211.533 MiB/sec 251.887 MiB/sec 19.077
{'iterations': 36, 'null_percent': 50.0, 'num_unique': 200.0}
13 UniqueUInt8/5 1.115 GiB/sec 1.290 GiB/sec 15.775
{'iterations': 199, 'null_percent': 99.0, 'num_unique': 200.0}
5 UniqueUInt8/2 855.076 MiB/sec 977.493 MiB/sec 14.316
{'iterations': 148, 'null_percent': 1.0, 'num_unique': 200.0}
7 UniqueInt64/4 1.072 GiB/sec 1.224 GiB/sec 14.136
{'iterations': 24, 'null_percent': 50.0, 'num_unique': 100.0}
43 UniqueInt64/9 697.824 MiB/sec 744.400 MiB/sec 6.674
{'iterations': 15, 'null_percent': 1.0, 'num_unique': 100000.0}
40 UniqueString10bytes/0 889.829 MiB/sec 924.593 MiB/sec 3.907
{'iterations': 15, 'null_percent': 0.0, 'num_unique': 100.0}
10 UniqueInt64/3 1.490 GiB/sec 1.549 GiB/sec 3.907
{'iterations': 34, 'null_percent': 10.0, 'num_unique': 100.0}
18 UniqueString10bytes/1 862.556 MiB/sec 890.677 MiB/sec 3.260
{'iterations': 15, 'null_percent': 0.1, 'num_unique': 100.0}
41 UniqueString100bytes/9 1.023 GiB/sec 1.051 GiB/sec 2.746
{'iterations': 2, 'null_percent': 1.0, 'num_unique': 100000.0}
38 UniqueString100bytes/11 1.941 GiB/sec 1.982 GiB/sec 2.152
{'iterations': 3, 'null_percent': 50.0, 'num_unique': 100000.0}
29 UniqueString100bytes/8 1.021 GiB/sec 1.037 GiB/sec 1.487
{'iterations': 2, 'null_percent': 0.1, 'num_unique': 100000.0}
34 UniqueString100bytes/10 1.109 GiB/sec 1.126 GiB/sec 1.484
{'iterations': 2, 'null_percent': 10.0, 'num_unique': 100000.0}
3 UniqueString100bytes/1 4.719 GiB/sec 4.775 GiB/sec 1.190
{'iterations': 8, 'null_percent': 0.1, 'num_unique': 100.0}
46 UniqueString10bytes/2 855.155 MiB/sec 862.082 MiB/sec 0.810
{'iterations': 15, 'null_percent': 1.0, 'num_unique': 100.0}
6 UniqueString100bytes/0 4.769 GiB/sec 4.799 GiB/sec 0.635
{'iterations': 9, 'null_percent': 0.0, 'num_unique': 100.0}
45 UniqueString100bytes/2 4.735 GiB/sec 4.759 GiB/sec 0.515
{'iterations': 8, 'null_percent': 1.0, 'num_unique': 100.0}
11 UniqueInt64/2 1.812 GiB/sec 1.812 GiB/sec -0.018
{'iterations': 41, 'null_percent': 1.0, 'num_unique': 100.0}
14 UniqueString10bytes/10 332.210 MiB/sec 331.356 MiB/sec -0.257
{'iterations': 6, 'null_percent': 10.0, 'num_unique': 100000.0}
48 UniqueInt64/0 2.123 GiB/sec 2.115 GiB/sec -0.357
{'iterations': 47, 'null_percent': 0.0, 'num_unique': 100.0}
15 UniqueString10bytes/9 311.568 MiB/sec 310.197 MiB/sec -0.440
{'iterations': 5, 'null_percent': 1.0, 'num_unique': 100000.0}
32 UniqueString10bytes/8 312.322 MiB/sec 310.270 MiB/sec -0.657
{'iterations': 5, 'null_percent': 0.1, 'num_unique': 100000.0}
47 UniqueInt64/1 1.987 GiB/sec 1.972 GiB/sec -0.750
{'iterations': 44, 'null_percent': 0.1, 'num_unique': 100.0}
42 UniqueString10bytes/7 315.974 MiB/sec 312.873 MiB/sec -0.981
{'iterations': 6, 'null_percent': 0.0, 'num_unique': 100000.0}
28 UniqueString100bytes/3 4.938 GiB/sec 4.869 GiB/sec -1.391
{'iterations': 9, 'null_percent': 10.0, 'num_unique': 100.0}
20 UniqueString100bytes/4 6.436 GiB/sec 6.315 GiB/sec -1.888
{'iterations': 12, 'null_percent': 50.0, 'num_unique': 100.0}
16 UniqueString10bytes/3 857.256 MiB/sec 833.623 MiB/sec -2.757
{'iterations': 15, 'null_percent': 10.0, 'num_unique': 100.0}
19 UniqueUInt8/1 1.536 GiB/sec 1.491 GiB/sec -2.979
{'iterations': 274, 'null_percent': 0.1, 'num_unique': 200.0}
17 UniqueString10bytes/11 519.556 MiB/sec 501.897 MiB/sec -3.399
{'iterations': 9, 'null_percent': 50.0, 'num_unique': 100000.0}
9 BuildStringDictionary 82.664 MiB/sec 78.620 MiB/sec -4.892
{'iterations': 192}
37 UniqueString10bytes/4 1002.825 MiB/sec 943.038 MiB/sec -5.962
{'iterations': 18, 'null_percent': 50.0, 'num_unique': 100.0}
31 UniqueInt64/8 819.931 MiB/sec 758.491 MiB/sec -7.493
{'iterations': 18, 'null_percent': 0.1, 'num_unique': 100000.0}
26 UniqueUInt8/0 2.043 GiB/sec 1.838 GiB/sec -10.027
{'iterations': 361, 'null_percent': 0.0, 'num_unique': 200.0}
0 UniqueInt64/7 861.692 MiB/sec 771.106 MiB/sec -10.513
{'iterations': 19, 'null_percent': 0.0, 'num_unique': 100000.0}
24 UniqueString100bytes/7 1.210 GiB/sec 1.042 GiB/sec -13.869
{'iterations': 2, 'null_percent': 0.0, 'num_unique': 100000.0}
1 UniqueString100bytes/12 53.895 GiB/sec 45.921 GiB/sec -14.796
{'iterations': 94, 'null_percent': 99.0, 'num_unique': 100000.0}
12 UniqueString10bytes/12 6.783 GiB/sec 5.344 GiB/sec -21.216
{'iterations': 116, 'null_percent': 99.0, 'num_unique': 100000.0}
4 UniqueString100bytes/5 88.160 GiB/sec 66.376 GiB/sec -24.709
{'iterations': 158, 'null_percent': 99.0, 'num_unique': 100.0}
50 UniqueString10bytes/5 9.337 GiB/sec 6.986 GiB/sec -25.171
{'iterations': 164, 'null_percent': 99.0, 'num_unique': 100.0}
23 UniqueString100bytes/13 271.062 GiB/sec 129.169 GiB/sec -52.347
{'iterations': 487, 'null_percent': 100.0, 'num_unique': 100000.0}
36 UniqueString10bytes/13 27.239 GiB/sec 12.890 GiB/sec -52.678
{'iterations': 489, 'null_percent': 100.0, 'num_unique': 100000.0}
49 UniqueString100bytes/6 272.431 GiB/sec 128.761 GiB/sec -52.736
{'iterations': 490, 'null_percent': 100.0, 'num_unique': 100.0}
39 UniqueString10bytes/6 27.250 GiB/sec 12.859 GiB/sec -52.811
{'iterations': 490, 'null_percent': 100.0, 'num_unique': 100.0}
```
The "regressions" at the bottom for the mostly-null case are basically
meaningless whereas the perf improvements seem more relevant
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
