adriangbot commented on PR #10554: URL: https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404777428
🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404308632) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) Comparing claude/parquet-exact-value-windows-10538 (c7591d17b58bd3db5cd2201a272653837da9c8c6) to 3577093f26591aee806a9133c891b6b8e5255477 [diff](https://github.com/apache/arrow-rs/compare/3577093f26591aee806a9133c891b6b8e5255477..c7591d17b58bd3db5cd2201a272653837da9c8c6) <details><summary>Run configuration</summary> ```yaml run benchmark arrow_writer env: BENCH_FILTER: "^[m-z]" baseline: ref: "3577093f26591aee806a9133c891b6b8e5255477" ``` </details> <details><summary>CPU Details (lscpu)</summary> ``` Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping: r0p1 BogoMIPS: 2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache: 32 MiB (16 instances) L3 cache: 80 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; __user pointer sanitization Vulnerability Spectre v2: Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` </details> <details><summary>Details</summary> <p> ``` group claude_parquet-exact-value-windows-10538 main ----- ---------------------------------------- ---- medium_string_shared_prefix_nullable/delta_byte_array 1.00 35.1±0.08ms 7.1 GB/sec 1.14 39.9±0.23ms 6.3 GB/sec primitive/bloom_filter 1.00 145.5±0.63ms 308.3 MB/sec 1.02 148.1±0.62ms 302.9 MB/sec primitive/cdc 1.00 152.7±0.66ms 293.8 MB/sec 1.02 155.5±0.60ms 288.6 MB/sec primitive/default 1.00 113.1±0.47ms 396.7 MB/sec 1.00 113.4±0.52ms 395.6 MB/sec primitive/number_distinct_values 1.00 363.4±1.51ms 123.5 MB/sec 1.01 366.4±1.47ms 122.5 MB/sec primitive/parquet_2 1.00 128.1±0.46ms 350.4 MB/sec 1.00 128.3±0.55ms 349.8 MB/sec primitive/zstd 1.00 142.9±0.62ms 314.0 MB/sec 1.00 143.3±0.44ms 313.1 MB/sec primitive/zstd_parquet_2 1.00 160.2±0.56ms 280.2 MB/sec 1.01 161.4±0.51ms 278.0 MB/sec primitive_all_null/bloom_filter 1.00 900.9±5.45µs 48.6 GB/sec 1.02 919.9±5.73µs 47.6 GB/sec primitive_all_null/cdc 1.00 18.4±0.29ms 2.4 GB/sec 1.04 19.0±0.40ms 2.3 GB/sec primitive_all_null/default 1.00 277.1±1.16µs 158.1 GB/sec 1.00 277.6±2.26µs 157.9 GB/sec primitive_all_null/number_distinct_values 1.00 277.6±1.16µs 157.9 GB/sec 1.00 277.8±2.28µs 157.8 GB/sec primitive_all_null/parquet_2 1.00 283.2±1.18µs 154.7 GB/sec 1.00 283.5±1.92µs 154.6 GB/sec primitive_all_null/zstd 1.00 425.1±1.11µs 103.1 GB/sec 1.01 427.3±2.08µs 102.6 GB/sec primitive_all_null/zstd_parquet_2 1.00 378.4±0.95µs 115.8 GB/sec 1.00 380.1±2.20µs 115.3 GB/sec primitive_non_null/bloom_filter 1.00 98.1±0.42ms 448.4 MB/sec 1.06 104.3±0.59ms 422.0 MB/sec primitive_non_null/cdc 1.00 82.7±0.49ms 532.1 MB/sec 1.01 83.4±0.32ms 527.8 MB/sec primitive_non_null/default 1.00 60.0±0.34ms 733.4 MB/sec 1.02 61.0±0.35ms 721.1 MB/sec primitive_non_null/number_distinct_values 1.00 486.9±1.99ms 90.4 MB/sec 1.01 490.7±2.26ms 89.7 MB/sec primitive_non_null/parquet_2 1.00 81.6±0.33ms 539.0 MB/sec 1.01 82.8±0.38ms 531.4 MB/sec primitive_non_null/zstd 1.00 90.7±0.30ms 484.9 MB/sec 1.01 91.9±0.37ms 479.0 MB/sec primitive_non_null/zstd_parquet_2 1.00 114.7±0.32ms 383.8 MB/sec 1.01 116.1±0.35ms 379.1 MB/sec primitive_sparse_99pct_null/bloom_filter 1.00 12.8±0.09ms 3.4 GB/sec 1.01 12.9±0.08ms 3.4 GB/sec primitive_sparse_99pct_null/cdc 1.00 30.0±0.25ms 1497.2 MB/sec 1.08 32.4±0.39ms 1385.1 MB/sec primitive_sparse_99pct_null/default 1.00 10.9±0.04ms 4.0 GB/sec 1.01 11.0±0.04ms 4.0 GB/sec primitive_sparse_99pct_null/number_distinct_values 1.00 14.7±0.11ms 3.0 GB/sec 1.01 14.9±0.10ms 2.9 GB/sec primitive_sparse_99pct_null/parquet_2 1.00 10.9±0.05ms 4.0 GB/sec 1.01 11.1±0.05ms 4.0 GB/sec primitive_sparse_99pct_null/zstd 1.00 14.4±0.04ms 3.0 GB/sec 1.00 14.5±0.04ms 3.0 GB/sec primitive_sparse_99pct_null/zstd_parquet_2 1.00 12.9±0.04ms 3.4 GB/sec 1.01 13.0±0.05ms 3.4 GB/sec short_string_non_null/bloom_filter 1.00 26.8±0.06ms 448.3 MB/sec 1.00 26.6±0.07ms 450.6 MB/sec short_string_non_null/cdc 1.00 20.3±0.04ms 592.3 MB/sec 1.01 20.4±0.05ms 588.3 MB/sec short_string_non_null/default 1.00 16.1±0.03ms 744.1 MB/sec 1.01 16.3±0.04ms 737.6 MB/sec short_string_non_null/number_distinct_values 1.00 68.6±0.28ms 174.9 MB/sec 1.01 69.4±0.34ms 172.9 MB/sec short_string_non_null/parquet_2 1.00 25.9±0.08ms 463.2 MB/sec 1.00 26.0±0.10ms 462.0 MB/sec short_string_non_null/zstd 1.00 36.3±0.09ms 330.4 MB/sec 1.01 36.5±0.11ms 328.7 MB/sec short_string_non_null/zstd_parquet_2 1.00 28.8±0.04ms 417.1 MB/sec 1.00 28.8±0.07ms 416.7 MB/sec small_string_distinct/delta_byte_array 1.01 1025.3±4.26µs 7.6 GB/sec 1.00 1015.9±6.94µs 7.7 GB/sec small_string_distinct/plain 1.01 754.3±5.71µs 10.4 GB/sec 1.00 744.0±7.67µs 10.5 GB/sec small_string_partial_prefix/delta_byte_array 1.00 1035.2±9.84µs 7.6 GB/sec 1.00 1036.5±8.63µs 7.6 GB/sec small_string_partial_prefix/plain 1.00 987.0±8.35µs 7.9 GB/sec 1.00 988.9±11.62µs 7.9 GB/sec small_string_shared_prefix/delta_byte_array 1.00 988.1±6.51µs 7.9 GB/sec 1.00 989.4±5.17µs 7.9 GB/sec small_string_shared_prefix/plain 1.04 1099.9±3.75µs 7.1 GB/sec 1.00 1059.9±3.24µs 7.4 GB/sec string/bloom_filter 1.04 229.0±26.28ms 2.2 GB/sec 1.00 220.3±22.07ms 2.3 GB/sec string/cdc 1.00 208.0±8.97ms 2.5 GB/sec 1.02 211.8±10.50ms 2.4 GB/sec string/default 1.00 112.6±12.68ms 4.5 GB/sec 1.22 137.3±24.40ms 3.7 GB/sec string/number_distinct_values 1.00 230.5±10.11ms 2.2 GB/sec 1.05 242.5±21.98ms 2.1 GB/sec string/parquet_2 1.01 187.0±0.57ms 2.7 GB/sec 1.00 185.4±0.55ms 2.8 GB/sec string/zstd 1.00 453.6±17.40ms 1155.7 MB/sec 1.05 476.9±17.64ms 1099.2 MB/sec string/zstd_parquet_2 1.00 412.1±22.04ms 1272.3 MB/sec 1.09 448.7±1.74ms 1168.4 MB/sec string_and_binary_view/bloom_filter 1.00 65.7±0.42ms 491.2 MB/sec 1.03 67.6±0.47ms 476.9 MB/sec string_and_binary_view/cdc 1.00 61.0±0.24ms 528.7 MB/sec 1.01 61.5±0.22ms 524.0 MB/sec string_and_binary_view/default 1.00 49.7±0.18ms 648.3 MB/sec 1.02 50.7±0.19ms 635.8 MB/sec string_and_binary_view/number_distinct_values 1.00 49.8±0.15ms 647.6 MB/sec 1.02 50.7±0.23ms 635.8 MB/sec string_and_binary_view/parquet_2 1.00 60.6±0.21ms 532.0 MB/sec 1.02 61.6±0.28ms 523.2 MB/sec string_and_binary_view/zstd 1.00 86.9±0.23ms 371.2 MB/sec 1.01 88.1±0.27ms 366.0 MB/sec string_and_binary_view/zstd_parquet_2 1.00 74.5±0.20ms 432.7 MB/sec 1.01 75.6±0.24ms 426.7 MB/sec string_dictionary/bloom_filter 1.00 86.3±0.69ms 3.0 GB/sec 1.27 109.5±1.63ms 2.4 GB/sec string_dictionary/cdc 1.00 48.4±0.35ms 5.3 GB/sec 1.43 69.3±0.95ms 3.7 GB/sec string_dictionary/default 1.00 43.8±0.28ms 5.9 GB/sec 1.47 64.3±0.41ms 4.0 GB/sec string_dictionary/number_distinct_values 1.00 75.7±0.50ms 3.4 GB/sec 1.30 98.2±0.52ms 2.6 GB/sec string_dictionary/parquet_2 1.00 47.6±0.13ms 5.4 GB/sec 1.36 64.8±0.22ms 4.0 GB/sec string_dictionary/zstd 1.00 209.0±0.61ms 1263.8 MB/sec 1.08 226.6±1.13ms 1165.8 MB/sec string_dictionary/zstd_parquet_2 1.00 192.0±0.17ms 1375.8 MB/sec 1.06 202.6±5.48ms 1303.4 MB/sec string_dictionary_low_cardinality_100/bloom_filter 1.00 33.8±0.38ms 118.6 MB/sec 1.00 33.8±0.13ms 118.4 MB/sec string_dictionary_low_cardinality_100/cdc 1.00 26.8±0.08ms 149.3 MB/sec 1.02 27.3±0.15ms 146.7 MB/sec string_dictionary_low_cardinality_100/default 1.00 24.6±0.39ms 162.6 MB/sec 1.01 25.0±0.12ms 160.4 MB/sec string_dictionary_low_cardinality_100/number_distinct_values 1.00 40.5±0.40ms 98.9 MB/sec 1.01 40.7±0.16ms 98.3 MB/sec string_dictionary_low_cardinality_100/parquet_2 1.00 24.7±0.38ms 162.4 MB/sec 1.01 25.0±0.13ms 160.4 MB/sec string_dictionary_low_cardinality_100/zstd 1.00 25.0±0.38ms 160.2 MB/sec 1.01 25.3±0.13ms 158.2 MB/sec string_dictionary_low_cardinality_100/zstd_parquet_2 1.00 25.0±0.39ms 160.2 MB/sec 1.01 25.3±0.12ms 158.4 MB/sec string_dictionary_low_cardinality_20/bloom_filter 1.03 33.7±0.91ms 118.7 MB/sec 1.00 32.9±0.16ms 121.8 MB/sec string_dictionary_low_cardinality_20/cdc 1.01 27.1±0.06ms 147.9 MB/sec 1.00 26.8±0.14ms 149.7 MB/sec string_dictionary_low_cardinality_20/default 1.04 25.3±0.89ms 158.2 MB/sec 1.00 24.4±0.16ms 164.1 MB/sec string_dictionary_low_cardinality_20/number_distinct_values 1.02 40.9±0.91ms 97.9 MB/sec 1.00 40.0±0.21ms 100.2 MB/sec string_dictionary_low_cardinality_20/parquet_2 1.03 25.3±0.89ms 158.2 MB/sec 1.00 24.5±0.16ms 163.2 MB/sec string_dictionary_low_cardinality_20/zstd 1.04 25.5±0.89ms 157.3 MB/sec 1.00 24.6±0.16ms 163.1 MB/sec string_dictionary_low_cardinality_20/zstd_parquet_2 1.04 25.4±0.88ms 157.5 MB/sec 1.00 24.6±0.16ms 163.1 MB/sec string_dictionary_low_cardinality_400/bloom_filter 1.01 34.5±0.44ms 116.2 MB/sec 1.00 34.2±0.13ms 117.3 MB/sec string_dictionary_low_cardinality_400/cdc 1.00 27.4±0.06ms 146.7 MB/sec 1.00 27.3±0.14ms 147.1 MB/sec string_dictionary_low_cardinality_400/default 1.00 25.3±0.45ms 158.8 MB/sec 1.00 25.3±0.15ms 158.8 MB/sec string_dictionary_low_cardinality_400/number_distinct_values 1.00 41.2±0.46ms 97.4 MB/sec 1.00 41.1±0.16ms 97.6 MB/sec string_dictionary_low_cardinality_400/parquet_2 1.00 25.3±0.45ms 158.7 MB/sec 1.00 25.2±0.15ms 158.9 MB/sec string_dictionary_low_cardinality_400/zstd 1.00 25.8±0.45ms 155.3 MB/sec 1.00 25.7±0.13ms 156.0 MB/sec string_dictionary_low_cardinality_400/zstd_parquet_2 1.00 25.8±0.46ms 155.6 MB/sec 1.00 25.7±0.14ms 155.9 MB/sec string_non_null/bloom_filter 1.00 233.0±16.10ms 2.2 GB/sec 1.16 270.7±28.72ms 1935.4 MB/sec string_non_null/cdc 1.03 258.0±7.47ms 2031.0 MB/sec 1.00 249.6±5.78ms 2.1 GB/sec string_non_null/default 1.00 126.3±21.28ms 4.1 GB/sec 1.06 134.3±26.58ms 3.8 GB/sec string_non_null/number_distinct_values 1.02 294.5±10.53ms 1779.5 MB/sec 1.00 287.9±4.98ms 1819.8 MB/sec string_non_null/parquet_2 1.00 124.0±9.64ms 4.1 GB/sec 1.59 196.9±0.63ms 2.6 GB/sec string_non_null/zstd 1.00 560.3±3.18ms 935.2 MB/sec 1.04 581.5±15.72ms 901.1 MB/sec string_non_null/zstd_parquet_2 1.00 500.5±3.86ms 1047.0 MB/sec 1.00 499.9±2.67ms 1048.1 MB/sec string_ree/bloom_filter 1.00 156.1±0.86ms 105.1 MB/sec 1.01 157.8±2.44ms 104.0 MB/sec string_ree/cdc 1.27 224.7±0.64ms 73.0 MB/sec 1.00 177.1±1.07ms 92.7 MB/sec string_ree/default 1.00 128.6±0.57ms 127.6 MB/sec 1.01 129.6±1.18ms 126.6 MB/sec string_ree/number_distinct_values 1.39 165.5±0.56ms 99.1 MB/sec 1.00 118.9±1.67ms 138.0 MB/sec string_ree/parquet_2 1.60 127.9±0.38ms 128.3 MB/sec 1.00 79.7±0.32ms 205.8 MB/sec string_ree/zstd 1.34 189.1±0.70ms 86.8 MB/sec 1.00 141.4±2.32ms 116.0 MB/sec string_ree/zstd_parquet_2 1.51 142.7±0.39ms 114.9 MB/sec 1.00 94.6±0.32ms 173.4 MB/sec string_ree_95pct_null/bloom_filter 1.00 26.6±0.06ms 52.9 MB/sec 1.00 26.7±0.05ms 52.8 MB/sec string_ree_95pct_null/cdc 1.00 32.6±0.07ms 43.1 MB/sec 1.00 32.7±0.10ms 43.0 MB/sec string_ree_95pct_null/default 1.00 25.1±0.04ms 56.2 MB/sec 1.00 25.0±0.05ms 56.3 MB/sec string_ree_95pct_null/number_distinct_values 1.00 27.1±0.06ms 51.9 MB/sec 1.01 27.3±0.04ms 51.6 MB/sec string_ree_95pct_null/parquet_2 1.00 24.9±0.04ms 56.4 MB/sec 1.00 25.0±0.05ms 56.3 MB/sec string_ree_95pct_null/zstd 1.00 25.8±0.05ms 54.6 MB/sec 1.00 25.9±0.04ms 54.4 MB/sec string_ree_95pct_null/zstd_parquet_2 1.00 25.7±0.04ms 54.8 MB/sec 1.00 25.7±0.04ms 54.7 MB/sec struct_all_null/bloom_filter 1.00 372.1±1.46µs 42.3 GB/sec 1.02 380.1±2.53µs 41.4 GB/sec struct_all_null/cdc 1.00 7.3±0.07ms 2.2 GB/sec 1.06 7.8±0.16ms 2.0 GB/sec struct_all_null/default 1.01 119.3±0.46µs 132.1 GB/sec 1.00 117.9±0.91µs 133.6 GB/sec struct_all_null/number_distinct_values 1.00 119.9±0.50µs 131.3 GB/sec 1.00 119.4±0.96µs 131.9 GB/sec struct_all_null/parquet_2 1.00 122.0±0.66µs 129.1 GB/sec 1.00 121.6±0.86µs 129.5 GB/sec struct_all_null/zstd 1.00 183.6±0.41µs 85.8 GB/sec 1.00 183.9±0.80µs 85.6 GB/sec struct_all_null/zstd_parquet_2 1.00 163.4±0.43µs 96.4 GB/sec 1.00 163.9±1.01µs 96.1 GB/sec struct_non_null/bloom_filter 1.00 41.5±0.21ms 385.6 MB/sec 1.06 43.8±0.25ms 365.4 MB/sec struct_non_null/cdc 1.00 41.5±0.24ms 385.9 MB/sec 1.02 42.3±0.17ms 378.3 MB/sec struct_non_null/default 1.00 28.0±0.18ms 571.0 MB/sec 1.02 28.7±0.17ms 557.4 MB/sec struct_non_null/number_distinct_values 1.13 217.4±1.07ms 73.6 MB/sec 1.00 191.8±1.08ms 83.4 MB/sec struct_non_null/parquet_2 1.00 36.7±0.25ms 435.6 MB/sec 1.02 37.6±0.16ms 425.7 MB/sec struct_non_null/zstd 1.00 36.5±0.18ms 438.6 MB/sec 1.03 37.4±0.18ms 427.6 MB/sec struct_non_null/zstd_parquet_2 1.00 50.4±0.30ms 317.8 MB/sec 1.02 51.3±0.21ms 312.0 MB/sec struct_sparse_99pct_null/bloom_filter 1.00 6.9±0.05ms 2.3 GB/sec 1.06 7.2±0.07ms 2.2 GB/sec struct_sparse_99pct_null/cdc 1.00 13.6±0.08ms 1190.0 MB/sec 1.13 15.4±0.13ms 1050.4 MB/sec struct_sparse_99pct_null/default 1.00 6.1±0.05ms 2.6 GB/sec 1.06 6.5±0.03ms 2.4 GB/sec struct_sparse_99pct_null/number_distinct_values 1.00 7.6±0.09ms 2.1 GB/sec 1.06 8.0±0.08ms 2004.2 MB/sec struct_sparse_99pct_null/parquet_2 1.00 6.1±0.04ms 2.6 GB/sec 1.07 6.5±0.04ms 2.4 GB/sec struct_sparse_99pct_null/zstd 1.00 7.5±0.05ms 2.1 GB/sec 1.06 8.0±0.03ms 2020.5 MB/sec struct_sparse_99pct_null/zstd_parquet_2 1.00 6.9±0.04ms 2.3 GB/sec 1.06 7.3±0.03ms 2.1 GB/sec ``` </p> </details> <details><summary>Resource Usage</summary> **base (merge-base)** | Metric | Value | |--------|-------| | Wall time | 1895.4s | | Peak memory | 3.0 GiB | | Avg memory | 2.5 GiB | | CPU user | 1782.9s | | CPU sys | 109.1s | | Peak spill | 0 B | **branch** | Metric | Value | |--------|-------| | Wall time | 1900.4s | | Peak memory | 2.9 GiB | | Avg memory | 2.5 GiB | | CPU user | 1784.5s | | CPU sys | 110.8s | | Peak spill | 0 B | </details> --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
