alamb commented on PR #10136: URL: https://github.com/apache/arrow-rs/pull/10136#issuecomment-5814248129
I ran the filter_kernel benchmarks on a GCP VM with `-target native` 5 times (aka ensuring to use the `pext` instruction) This branch is reproducably better (by up to 14x 🤯 ) Full Results <details> ```text ==> Paired runs, only rows differing by more than 5% --- base-1 vs pr-1 group base-1 pr-1 ----- ------ ---- filter context decimal128 high selectivity (kept 1023/1024) 1.08 57.7±3.68µs ? ?/sec 1.00 53.6±0.64µs ? ?/sec filter context f32 (kept 1/2) 1.76 117.9±1.33µs ? ?/sec 1.00 66.8±0.05µs ? ?/sec filter context fsb with value length 20 (kept 1/2) 14.80 54.8±0.64µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 20 high selectivity (kept 1023/1024) 14.73 54.6±0.77µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 20 low selectivity (kept 1/1024) 14.77 54.6±0.64µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 (kept 1/2) 14.83 54.8±0.69µs ? ?/sec 1.00 3.7±0.03µs ? ?/sec filter context fsb with value length 5 high selectivity (kept 1023/1024) 14.80 54.7±0.92µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 low selectivity (kept 1/1024) 14.75 54.6±0.60µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 (kept 1/2) 14.80 54.8±0.61µs ? ?/sec 1.00 3.7±0.05µs ? ?/sec filter context fsb with value length 50 high selectivity (kept 1023/1024) 14.78 54.6±0.54µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 14.74 54.6±0.55µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context i32 w NULLs (kept 1/2) 1.76 117.9±0.51µs ? ?/sec 1.00 66.9±0.63µs ? ?/sec filter context mixed string view (kept 1/2) 2.26 122.0±0.80µs ? ?/sec 1.00 53.9±3.14µs ? ?/sec filter context mixed string view high selectivity (kept 1023/1024) 1.14 62.1±4.39µs ? ?/sec 1.00 54.3±0.31µs ? ?/sec filter context short string view (kept 1/2) 1.63 117.7±2.20µs ? ?/sec 1.00 72.1±0.32µs ? ?/sec filter context short string view high selectivity (kept 1023/1024) 1.12 59.8±4.02µs ? ?/sec 1.00 53.4±0.66µs ? ?/sec filter context string (kept 1/2) 1.10 543.0±10.83µs ? ?/sec 1.00 492.4±6.71µs ? ?/sec filter context string dictionary high selectivity (kept 1023/1024) 1.17 7.2±0.24µs ? ?/sec 1.00 6.1±0.13µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.76 117.9±1.27µs ? ?/sec 1.00 66.8±0.11µs ? ?/sec filter context string high selectivity (kept 1023/1024) 1.05 713.3±41.05µs ? ?/sec 1.00 676.7±34.64µs ? ?/sec filter context u8 w NULLs (kept 1/2) 4.39 65.9±0.27µs ? ?/sec 1.00 15.0±0.18µs ? ?/sec filter decimal128 (kept 1/2) 1.00 41.3±4.07µs ? ?/sec 1.27 52.6±0.31µs ? ?/sec filter f32 (kept 1/2) 3.45 125.7±2.06µs ? ?/sec 1.00 36.4±0.39µs ? ?/sec filter fsb with value length 20 (kept 1/2) 1.05 94.5±0.80µs ? ?/sec 1.00 89.6±2.05µs ? ?/sec filter fsb with value length 20 high selectivity (kept 1023/1024) 1.00 74.6±0.69µs ? ?/sec 1.17 87.4±6.96µs ? ?/sec filter fsb with value length 20 low selectivity (kept 1/1024) 1.00 2.8±0.04µs ? ?/sec 1.05 2.9±0.03µs ? ?/sec filter fsb with value length 5 high selectivity (kept 1023/1024) 1.05 10.6±0.63µs ? ?/sec 1.00 10.0±0.37µs ? ?/sec filter fsb with value length 50 (kept 1/2) 1.00 157.7±16.84µs ? ?/sec 1.15 180.7±4.45µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.00 201.6±8.40µs ? ?/sec 1.29 260.1±3.59µs ? ?/sec filter optimize high selectivity (kept 1023/1024) 1.13 2.5±0.03µs ? ?/sec 1.00 2.2±0.03µs ? ?/sec --- base-2 vs pr-2 group base-2 pr-2 ----- ------ ---- filter context decimal128 high selectivity (kept 1023/1024) 1.12 58.4±0.11µs ? ?/sec 1.00 52.0±0.54µs ? ?/sec filter context f32 (kept 1/2) 1.76 118.0±1.35µs ? ?/sec 1.00 66.9±0.15µs ? ?/sec filter context fsb with value length 20 (kept 1/2) 14.88 54.8±0.51µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 20 high selectivity (kept 1023/1024) 14.83 54.8±0.63µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 20 low selectivity (kept 1/1024) 14.71 54.5±0.08µs ? ?/sec 1.00 3.7±0.05µs ? ?/sec filter context fsb with value length 5 (kept 1/2) 14.85 54.8±0.60µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 high selectivity (kept 1023/1024) 14.78 54.5±0.08µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 5 low selectivity (kept 1/1024) 14.77 54.6±0.54µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 (kept 1/2) 14.85 54.7±0.50µs ? ?/sec 1.00 3.7±0.03µs ? ?/sec filter context fsb with value length 50 high selectivity (kept 1023/1024) 14.78 54.6±0.15µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 14.77 54.6±0.09µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context i32 w NULLs (kept 1/2) 1.76 117.8±1.12µs ? ?/sec 1.00 66.9±0.71µs ? ?/sec filter context mixed string view (kept 1/2) 1.77 108.6±1.28µs ? ?/sec 1.00 61.3±8.88µs ? ?/sec filter context mixed string view high selectivity (kept 1023/1024) 1.19 62.5±0.25µs ? ?/sec 1.00 52.4±0.69µs ? ?/sec filter context short string view (kept 1/2) 1.93 103.0±3.32µs ? ?/sec 1.00 53.3±0.62µs ? ?/sec filter context short string view high selectivity (kept 1023/1024) 1.08 65.8±0.14µs ? ?/sec 1.00 60.7±1.07µs ? ?/sec filter context string (kept 1/2) 1.10 545.9±2.00µs ? ?/sec 1.00 494.5±10.86µs ? ?/sec filter context string dictionary high selectivity (kept 1023/1024) 1.12 6.9±0.22µs ? ?/sec 1.00 6.2±0.16µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.76 118.2±1.23µs ? ?/sec 1.00 67.1±0.63µs ? ?/sec filter context string dictionary w NULLs high selectivity (kept 1023/1024) 1.00 8.3±0.05µs ? ?/sec 1.12 9.3±0.59µs ? ?/sec filter context u8 w NULLs (kept 1/2) 4.39 66.1±0.95µs ? ?/sec 1.00 15.0±0.17µs ? ?/sec filter decimal128 (kept 1/2) 1.20 52.3±1.06µs ? ?/sec 1.00 43.5±4.36µs ? ?/sec filter decimal128 high selectivity (kept 1023/1024) 1.08 61.8±0.60µs ? ?/sec 1.00 57.5±0.23µs ? ?/sec filter f32 (kept 1/2) 3.45 125.5±0.15µs ? ?/sec 1.00 36.4±0.46µs ? ?/sec filter fsb with value length 20 high selectivity (kept 1023/1024) 1.25 87.8±0.62µs ? ?/sec 1.00 70.3±2.86µs ? ?/sec filter fsb with value length 5 high selectivity (kept 1023/1024) 1.00 9.9±0.28µs ? ?/sec 1.12 11.0±0.62µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.00 201.7±6.48µs ? ?/sec 1.30 262.4±0.87µs ? ?/sec filter optimize high selectivity (kept 1023/1024) 1.12 2.5±0.03µs ? ?/sec 1.00 2.2±0.02µs ? ?/sec --- base-3 vs pr-3 group base-3 pr-3 ----- ------ ---- filter context decimal128 (kept 1/2) 1.00 50.4±0.59µs ? ?/sec 1.24 62.2±6.46µs ? ?/sec filter context f32 (kept 1/2) 1.77 118.1±1.86µs ? ?/sec 1.00 66.9±0.70µs ? ?/sec filter context fsb with value length 20 (kept 1/2) 14.81 54.7±0.57µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 20 high selectivity (kept 1023/1024) 14.78 54.6±0.56µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 20 low selectivity (kept 1/1024) 14.74 54.6±0.59µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 (kept 1/2) 14.84 54.8±0.67µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 5 high selectivity (kept 1023/1024) 14.79 54.6±0.12µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 low selectivity (kept 1/1024) 14.79 54.6±0.50µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 50 (kept 1/2) 14.80 54.6±0.11µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 high selectivity (kept 1023/1024) 14.77 54.6±0.60µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 14.76 54.6±0.50µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context i32 w NULLs (kept 1/2) 1.76 117.9±1.22µs ? ?/sec 1.00 67.0±0.75µs ? ?/sec filter context mixed string view (kept 1/2) 2.15 116.2±1.40µs ? ?/sec 1.00 53.9±0.61µs ? ?/sec filter context short string view (kept 1/2) 1.61 114.2±5.76µs ? ?/sec 1.00 71.1±3.09µs ? ?/sec filter context short string view high selectivity (kept 1023/1024) 1.00 52.7±0.74µs ? ?/sec 1.06 56.0±2.90µs ? ?/sec filter context string (kept 1/2) 1.22 591.3±13.08µs ? ?/sec 1.00 484.3±7.28µs ? ?/sec filter context string dictionary high selectivity (kept 1023/1024) 1.15 7.1±0.15µs ? ?/sec 1.00 6.2±0.21µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.77 118.4±1.86µs ? ?/sec 1.00 66.9±0.66µs ? ?/sec filter context string high selectivity (kept 1023/1024) 1.08 771.2±54.10µs ? ?/sec 1.00 713.3±33.32µs ? ?/sec filter context u8 w NULLs (kept 1/2) 4.40 66.0±1.29µs ? ?/sec 1.00 15.0±0.16µs ? ?/sec filter decimal128 (kept 1/2) 1.00 40.2±0.28µs ? ?/sec 1.16 46.6±6.76µs ? ?/sec filter decimal128 high selectivity (kept 1023/1024) 1.00 52.2±1.32µs ? ?/sec 1.19 61.9±0.46µs ? ?/sec filter f32 (kept 1/2) 3.46 125.8±1.33µs ? ?/sec 1.00 36.3±0.39µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.07 227.3±20.88µs ? ?/sec 1.00 212.3±2.24µs ? ?/sec filter optimize high selectivity (kept 1023/1024) 1.12 2.5±0.05µs ? ?/sec 1.00 2.2±0.03µs ? ?/sec --- base-4 vs pr-4 group base-4 pr-4 ----- ------ ---- filter context decimal128 high selectivity (kept 1023/1024) 1.00 49.2±0.89µs ? ?/sec 1.26 62.1±0.69µs ? ?/sec filter context f32 (kept 1/2) 1.76 118.0±1.12µs ? ?/sec 1.00 67.0±0.70µs ? ?/sec filter context f32 high selectivity (kept 1023/1024) 1.05 8.7±0.41µs ? ?/sec 1.00 8.3±0.15µs ? ?/sec filter context fsb with value length 20 (kept 1/2) 14.81 54.7±0.56µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 20 high selectivity (kept 1023/1024) 14.78 54.6±0.56µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 20 low selectivity (kept 1/1024) 14.79 54.6±0.58µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 5 (kept 1/2) 14.81 54.7±0.57µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 high selectivity (kept 1023/1024) 14.74 54.6±0.51µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 low selectivity (kept 1/1024) 14.74 54.6±0.15µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 (kept 1/2) 14.77 54.6±0.10µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 50 high selectivity (kept 1023/1024) 14.79 54.7±1.15µs ? ?/sec 1.00 3.7±0.03µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 14.78 54.6±0.60µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context i32 high selectivity (kept 1023/1024) 1.07 6.6±0.41µs ? ?/sec 1.00 6.2±0.27µs ? ?/sec filter context i32 low selectivity (kept 1/1024) 1.06 248.9±4.42ns ? ?/sec 1.00 235.9±2.20ns ? ?/sec filter context i32 w NULLs (kept 1/2) 1.77 118.1±1.22µs ? ?/sec 1.00 66.8±0.23µs ? ?/sec filter context mixed string view (kept 1/2) 1.91 117.9±1.23µs ? ?/sec 1.00 61.8±5.35µs ? ?/sec filter context mixed string view high selectivity (kept 1023/1024) 1.08 56.4±2.45µs ? ?/sec 1.00 52.2±0.83µs ? ?/sec filter context short string view (kept 1/2) 2.27 116.9±0.31µs ? ?/sec 1.00 51.5±1.39µs ? ?/sec filter context string (kept 1/2) 1.20 573.2±11.63µs ? ?/sec 1.00 479.7±8.30µs ? ?/sec filter context string dictionary high selectivity (kept 1023/1024) 1.15 7.3±0.18µs ? ?/sec 1.00 6.4±0.31µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.76 118.1±1.29µs ? ?/sec 1.00 66.9±0.64µs ? ?/sec filter context string low selectivity (kept 1/1024) 1.05 822.6±8.51ns ? ?/sec 1.00 780.8±8.95ns ? ?/sec filter context u8 w NULLs (kept 1/2) 4.40 66.0±0.65µs ? ?/sec 1.00 15.0±0.16µs ? ?/sec filter decimal128 (kept 1/2) 1.00 42.1±1.93µs ? ?/sec 1.25 52.5±0.26µs ? ?/sec filter f32 (kept 1/2) 3.45 125.6±1.23µs ? ?/sec 1.00 36.4±0.12µs ? ?/sec filter fsb with value length 20 (kept 1/2) 1.05 93.0±2.09µs ? ?/sec 1.00 88.5±2.77µs ? ?/sec filter fsb with value length 20 high selectivity (kept 1023/1024) 1.37 90.8±0.19µs ? ?/sec 1.00 66.4±0.20µs ? ?/sec filter fsb with value length 50 (kept 1/2) 1.00 174.9±5.42µs ? ?/sec 1.05 183.8±1.00µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.00 212.6±7.26µs ? ?/sec 1.23 261.4±1.03µs ? ?/sec --- base-5 vs pr-5 group base-5 pr-5 ----- ------ ---- filter context decimal128 (kept 1/2) 1.40 67.1±0.38µs ? ?/sec 1.00 47.9±0.40µs ? ?/sec filter context decimal128 high selectivity (kept 1023/1024) 1.13 60.6±0.65µs ? ?/sec 1.00 53.9±1.63µs ? ?/sec filter context f32 (kept 1/2) 1.76 117.9±1.11µs ? ?/sec 1.00 66.9±0.72µs ? ?/sec filter context fsb with value length 20 (kept 1/2) 14.82 54.7±0.50µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 20 high selectivity (kept 1023/1024) 14.79 54.6±0.39µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 20 low selectivity (kept 1/1024) 14.77 54.6±0.87µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context fsb with value length 5 (kept 1/2) 14.81 54.7±0.50µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 high selectivity (kept 1023/1024) 14.81 54.6±0.13µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 5 low selectivity (kept 1/1024) 14.78 54.7±0.57µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 (kept 1/2) 14.79 54.7±0.50µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 high selectivity (kept 1023/1024) 14.75 54.6±0.52µs ? ?/sec 1.00 3.7±0.04µs ? ?/sec filter context fsb with value length 50 low selectivity (kept 1/1024) 14.84 54.7±0.49µs ? ?/sec 1.00 3.7±0.01µs ? ?/sec filter context i32 w NULLs (kept 1/2) 1.76 117.7±1.09µs ? ?/sec 1.00 67.0±0.66µs ? ?/sec filter context mixed string view (kept 1/2) 1.94 112.3±0.99µs ? ?/sec 1.00 57.9±7.21µs ? ?/sec filter context short string view (kept 1/2) 1.85 103.0±2.97µs ? ?/sec 1.00 55.7±2.75µs ? ?/sec filter context short string view high selectivity (kept 1023/1024) 1.20 65.3±0.60µs ? ?/sec 1.00 54.6±0.66µs ? ?/sec filter context string (kept 1/2) 1.07 534.6±7.43µs ? ?/sec 1.00 497.9±11.11µs ? ?/sec filter context string dictionary high selectivity (kept 1023/1024) 1.13 7.0±0.34µs ? ?/sec 1.00 6.2±0.26µs ? ?/sec filter context string dictionary w NULLs (kept 1/2) 1.76 117.8±0.13µs ? ?/sec 1.00 67.0±0.84µs ? ?/sec filter context string dictionary w NULLs high selectivity (kept 1023/1024) 1.00 8.8±0.44µs ? ?/sec 1.06 9.3±0.46µs ? ?/sec filter context u8 w NULLs (kept 1/2) 4.41 66.0±0.72µs ? ?/sec 1.00 15.0±0.17µs ? ?/sec filter f32 (kept 1/2) 3.45 125.6±1.23µs ? ?/sec 1.00 36.4±0.61µs ? ?/sec filter fsb with value length 20 high selectivity (kept 1023/1024) 1.11 76.6±0.33µs ? ?/sec 1.00 68.8±0.79µs ? ?/sec filter fsb with value length 50 high selectivity (kept 1023/1024) 1.00 221.6±7.25µs ? ?/sec 1.05 233.8±7.41µs ? ?/sec filter optimize high selectivity (kept 1023/1024) 1.09 2.5±0.03µs ? ?/sec 1.00 2.3±0.05µs ? ?/sec </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
