adriangbot commented on PR #10554: URL: https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404770389
🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404308729) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) Comparing claude/parquet-exact-value-windows-10538 (c7591d17b58bd3db5cd2201a272653837da9c8c6) to 3577093f26591aee806a9133c891b6b8e5255477 [diff](https://github.com/apache/arrow-rs/compare/3577093f26591aee806a9133c891b6b8e5255477..c7591d17b58bd3db5cd2201a272653837da9c8c6) <details><summary>Run configuration</summary> ```yaml run benchmark arrow_writer env: BENCH_FILTER: "^[m-z]" baseline: ref: "3577093f26591aee806a9133c891b6b8e5255477" ``` </details> <details><summary>CPU Details (lscpu)</summary> ``` Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping: r0p1 BogoMIPS: 2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache: 32 MiB (16 instances) L3 cache: 80 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; __user pointer sanitization Vulnerability Spectre v2: Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` </details> <details><summary>Details</summary> <p> ``` group claude_parquet-exact-value-windows-10538 main ----- ---------------------------------------- ---- medium_string_shared_prefix_nullable/delta_byte_array 1.00 37.6±0.10ms 6.7 GB/sec 1.02 38.3±0.21ms 6.5 GB/sec primitive/bloom_filter 1.00 139.5±0.44ms 321.7 MB/sec 1.02 142.4±0.64ms 315.1 MB/sec primitive/cdc 1.00 150.2±0.66ms 298.8 MB/sec 1.01 151.9±0.43ms 295.4 MB/sec primitive/default 1.00 109.9±0.37ms 408.5 MB/sec 1.02 112.5±2.42ms 398.7 MB/sec primitive/number_distinct_values 1.00 353.6±0.75ms 126.9 MB/sec 1.00 355.0±1.31ms 126.4 MB/sec primitive/parquet_2 1.00 124.9±0.40ms 359.3 MB/sec 1.01 126.4±0.47ms 355.1 MB/sec primitive/zstd 1.00 139.5±0.36ms 321.6 MB/sec 1.01 141.0±0.39ms 318.4 MB/sec primitive/zstd_parquet_2 1.00 157.6±0.34ms 284.7 MB/sec 1.01 159.1±0.41ms 282.0 MB/sec primitive_all_null/bloom_filter 1.00 876.3±1.85µs 50.0 GB/sec 1.00 880.5±2.37µs 49.8 GB/sec primitive_all_null/cdc 1.00 18.4±0.36ms 2.4 GB/sec 1.03 18.9±0.29ms 2.3 GB/sec primitive_all_null/default 1.00 276.3±1.33µs 158.6 GB/sec 1.00 276.8±1.90µs 158.3 GB/sec primitive_all_null/number_distinct_values 1.00 277.0±1.32µs 158.2 GB/sec 1.01 278.4±1.94µs 157.4 GB/sec primitive_all_null/parquet_2 1.00 283.8±1.27µs 154.4 GB/sec 1.00 283.9±1.41µs 154.4 GB/sec primitive_all_null/zstd 1.00 425.4±1.10µs 103.0 GB/sec 1.01 428.5±2.08µs 102.3 GB/sec primitive_all_null/zstd_parquet_2 1.00 379.0±0.74µs 115.6 GB/sec 1.00 379.8±2.06µs 115.4 GB/sec primitive_non_null/bloom_filter 1.00 95.9±0.38ms 458.9 MB/sec 1.01 96.6±0.52ms 455.5 MB/sec primitive_non_null/cdc 1.00 81.4±0.42ms 540.8 MB/sec 1.00 81.0±0.35ms 543.4 MB/sec primitive_non_null/default 1.00 58.7±0.36ms 750.1 MB/sec 1.00 58.6±0.43ms 750.5 MB/sec primitive_non_null/number_distinct_values 1.00 469.1±1.35ms 93.8 MB/sec 1.00 467.2±1.69ms 94.2 MB/sec primitive_non_null/parquet_2 1.00 80.3±0.34ms 548.2 MB/sec 1.00 80.4±0.37ms 547.3 MB/sec primitive_non_null/zstd 1.00 89.4±0.39ms 492.3 MB/sec 1.00 89.2±0.36ms 493.5 MB/sec primitive_non_null/zstd_parquet_2 1.00 113.7±0.51ms 387.0 MB/sec 1.00 113.4±0.34ms 388.2 MB/sec primitive_sparse_99pct_null/bloom_filter 1.00 11.8±0.06ms 3.7 GB/sec 1.00 11.8±0.06ms 3.7 GB/sec primitive_sparse_99pct_null/cdc 1.00 29.0±0.33ms 1545.9 MB/sec 1.08 31.2±0.30ms 1437.8 MB/sec primitive_sparse_99pct_null/default 1.00 10.5±0.03ms 4.2 GB/sec 1.00 10.5±0.03ms 4.2 GB/sec primitive_sparse_99pct_null/number_distinct_values 1.00 13.7±0.05ms 3.2 GB/sec 1.00 13.7±0.04ms 3.2 GB/sec primitive_sparse_99pct_null/parquet_2 1.00 10.5±0.03ms 4.2 GB/sec 1.00 10.6±0.03ms 4.1 GB/sec primitive_sparse_99pct_null/zstd 1.00 13.9±0.04ms 3.2 GB/sec 1.00 13.8±0.04ms 3.2 GB/sec primitive_sparse_99pct_null/zstd_parquet_2 1.00 12.4±0.05ms 3.5 GB/sec 1.00 12.4±0.03ms 3.5 GB/sec short_string_non_null/bloom_filter 1.04 26.5±0.08ms 453.6 MB/sec 1.00 25.4±0.07ms 471.8 MB/sec short_string_non_null/cdc 1.00 20.0±0.04ms 600.8 MB/sec 1.00 20.0±0.05ms 599.8 MB/sec short_string_non_null/default 1.00 15.8±0.03ms 759.3 MB/sec 1.00 15.8±0.04ms 761.2 MB/sec short_string_non_null/number_distinct_values 1.01 68.4±0.13ms 175.4 MB/sec 1.00 67.5±0.16ms 177.8 MB/sec short_string_non_null/parquet_2 1.01 25.7±0.06ms 467.6 MB/sec 1.00 25.4±0.08ms 472.0 MB/sec short_string_non_null/zstd 1.00 36.1±0.10ms 332.8 MB/sec 1.00 36.0±0.10ms 333.7 MB/sec short_string_non_null/zstd_parquet_2 1.01 28.5±0.05ms 421.3 MB/sec 1.00 28.3±0.06ms 423.5 MB/sec small_string_distinct/delta_byte_array 1.02 962.1±4.39µs 8.2 GB/sec 1.00 946.1±6.34µs 8.3 GB/sec small_string_distinct/plain 1.00 713.7±3.71µs 11.0 GB/sec 1.02 724.5±1.98µs 10.8 GB/sec small_string_partial_prefix/delta_byte_array 1.00 967.7±1.79µs 8.1 GB/sec 1.00 964.8±2.04µs 8.1 GB/sec small_string_partial_prefix/plain 1.00 941.5±2.17µs 8.3 GB/sec 1.02 955.8±2.40µs 8.2 GB/sec small_string_shared_prefix/delta_byte_array 1.01 967.4±1.56µs 8.1 GB/sec 1.00 958.0±1.32µs 8.2 GB/sec small_string_shared_prefix/plain 1.00 1063.9±8.86µs 7.4 GB/sec 1.02 1085.3±6.58µs 7.2 GB/sec string/bloom_filter 1.01 202.9±19.46ms 2.5 GB/sec 1.00 200.0±19.67ms 2.6 GB/sec string/cdc 1.00 206.1±10.21ms 2.5 GB/sec 1.00 205.2±9.52ms 2.5 GB/sec string/default 1.00 122.3±19.49ms 4.2 GB/sec 1.06 129.1±24.16ms 4.0 GB/sec string/number_distinct_values 1.03 235.6±22.05ms 2.2 GB/sec 1.00 228.9±19.89ms 2.2 GB/sec string/parquet_2 1.00 105.5±7.79ms 4.9 GB/sec 1.67 176.5±0.55ms 2.9 GB/sec string/zstd 1.00 455.8±14.54ms 1150.1 MB/sec 1.00 457.1±16.12ms 1147.0 MB/sec string/zstd_parquet_2 1.00 441.3±1.08ms 1187.9 MB/sec 1.00 439.2±1.69ms 1193.8 MB/sec string_and_binary_view/bloom_filter 1.00 64.0±0.20ms 503.5 MB/sec 1.00 63.8±0.21ms 505.3 MB/sec string_and_binary_view/cdc 1.00 60.1±0.11ms 536.9 MB/sec 1.00 60.2±0.14ms 536.0 MB/sec string_and_binary_view/default 1.00 49.2±0.08ms 656.1 MB/sec 1.01 49.5±0.10ms 651.1 MB/sec string_and_binary_view/number_distinct_values 1.00 49.2±0.09ms 656.0 MB/sec 1.01 49.5±0.10ms 651.3 MB/sec string_and_binary_view/parquet_2 1.00 60.0±0.10ms 537.3 MB/sec 1.00 60.2±0.10ms 535.6 MB/sec string_and_binary_view/zstd 1.00 86.1±0.12ms 374.6 MB/sec 1.00 86.4±0.13ms 373.3 MB/sec string_and_binary_view/zstd_parquet_2 1.00 74.0±0.16ms 436.1 MB/sec 1.00 74.1±0.13ms 435.2 MB/sec string_dictionary/bloom_filter 1.26 126.7±0.34ms 2.0 GB/sec 1.00 100.2±0.47ms 2.6 GB/sec string_dictionary/cdc 1.05 67.4±1.11ms 3.8 GB/sec 1.00 64.0±0.67ms 4.0 GB/sec string_dictionary/default 1.51 85.4±0.36ms 3.0 GB/sec 1.00 56.6±0.77ms 4.6 GB/sec string_dictionary/number_distinct_values 1.30 119.5±0.41ms 2.2 GB/sec 1.00 92.1±0.29ms 2.8 GB/sec string_dictionary/parquet_2 1.45 89.4±0.22ms 2.9 GB/sec 1.00 61.7±0.23ms 4.2 GB/sec string_dictionary/zstd 1.06 230.3±8.63ms 1147.1 MB/sec 1.00 217.2±0.36ms 1216.1 MB/sec string_dictionary/zstd_parquet_2 1.03 209.2±0.23ms 1262.6 MB/sec 1.00 202.2±0.79ms 1306.0 MB/sec string_dictionary_low_cardinality_100/bloom_filter 1.00 33.2±0.06ms 120.8 MB/sec 1.01 33.5±0.14ms 119.6 MB/sec string_dictionary_low_cardinality_100/cdc 1.00 26.8±0.07ms 149.2 MB/sec 1.00 26.9±0.11ms 149.0 MB/sec string_dictionary_low_cardinality_100/default 1.00 24.5±0.05ms 163.4 MB/sec 1.00 24.6±0.11ms 162.7 MB/sec string_dictionary_low_cardinality_100/number_distinct_values 1.00 40.2±0.13ms 99.7 MB/sec 1.00 40.2±0.17ms 99.5 MB/sec string_dictionary_low_cardinality_100/parquet_2 1.00 24.5±0.05ms 163.2 MB/sec 1.01 24.7±0.13ms 162.0 MB/sec string_dictionary_low_cardinality_100/zstd 1.00 24.9±0.05ms 160.9 MB/sec 1.00 25.0±0.13ms 160.3 MB/sec string_dictionary_low_cardinality_100/zstd_parquet_2 1.00 24.9±0.05ms 161.0 MB/sec 1.00 25.0±0.13ms 160.4 MB/sec string_dictionary_low_cardinality_20/bloom_filter 1.00 32.6±0.06ms 122.9 MB/sec 1.02 33.2±0.15ms 120.5 MB/sec string_dictionary_low_cardinality_20/cdc 1.00 26.9±0.06ms 148.9 MB/sec 1.00 27.0±0.13ms 148.3 MB/sec string_dictionary_low_cardinality_20/default 1.00 24.5±0.06ms 163.7 MB/sec 1.01 24.8±0.15ms 161.6 MB/sec string_dictionary_low_cardinality_20/number_distinct_values 1.00 39.8±0.13ms 100.6 MB/sec 1.01 40.1±0.20ms 99.9 MB/sec string_dictionary_low_cardinality_20/parquet_2 1.00 24.5±0.05ms 163.6 MB/sec 1.01 24.8±0.16ms 161.8 MB/sec string_dictionary_low_cardinality_20/zstd 1.00 24.6±0.05ms 162.8 MB/sec 1.01 24.9±0.14ms 161.1 MB/sec string_dictionary_low_cardinality_20/zstd_parquet_2 1.00 24.6±0.05ms 162.9 MB/sec 1.01 24.9±0.14ms 160.8 MB/sec string_dictionary_low_cardinality_400/bloom_filter 1.00 33.7±0.07ms 119.1 MB/sec 1.02 34.4±0.14ms 116.7 MB/sec string_dictionary_low_cardinality_400/cdc 1.00 27.0±0.06ms 148.5 MB/sec 1.01 27.2±0.12ms 147.6 MB/sec string_dictionary_low_cardinality_400/default 1.00 24.6±0.05ms 163.1 MB/sec 1.02 25.1±0.14ms 160.1 MB/sec string_dictionary_low_cardinality_400/number_distinct_values 1.00 40.3±0.09ms 99.6 MB/sec 1.01 40.8±0.18ms 98.4 MB/sec string_dictionary_low_cardinality_400/parquet_2 1.00 24.6±0.05ms 163.0 MB/sec 1.02 25.1±0.14ms 160.0 MB/sec string_dictionary_low_cardinality_400/zstd 1.00 25.1±0.06ms 159.7 MB/sec 1.01 25.5±0.14ms 157.4 MB/sec string_dictionary_low_cardinality_400/zstd_parquet_2 1.00 25.2±0.06ms 159.3 MB/sec 1.01 25.5±0.14ms 157.6 MB/sec string_non_null/bloom_filter 1.00 236.1±25.37ms 2.2 GB/sec 1.02 240.4±22.58ms 2.1 GB/sec string_non_null/cdc 1.00 242.8±3.26ms 2.1 GB/sec 1.00 243.2±4.58ms 2.1 GB/sec string_non_null/default 1.00 121.9±27.78ms 4.2 GB/sec 1.16 142.0±27.39ms 3.6 GB/sec string_non_null/number_distinct_values 1.00 270.6±4.46ms 1936.6 MB/sec 1.03 277.9±5.86ms 1885.3 MB/sec string_non_null/parquet_2 1.11 115.2±8.07ms 4.4 GB/sec 1.00 104.0±0.65ms 4.9 GB/sec string_non_null/zstd 1.00 521.1±0.98ms 1005.6 MB/sec 1.01 526.1±4.02ms 996.0 MB/sec string_non_null/zstd_parquet_2 1.00 487.6±4.35ms 1074.8 MB/sec 1.02 496.4±2.81ms 1055.6 MB/sec string_ree/bloom_filter 1.00 104.5±0.40ms 157.0 MB/sec 1.45 151.1±1.49ms 108.6 MB/sec string_ree/cdc 1.00 173.9±0.47ms 94.3 MB/sec 1.26 219.0±1.78ms 74.9 MB/sec string_ree/default 1.00 77.6±0.76ms 211.4 MB/sec 1.59 123.7±1.19ms 132.7 MB/sec string_ree/number_distinct_values 1.00 114.8±0.40ms 142.9 MB/sec 1.40 160.2±1.79ms 102.4 MB/sec string_ree/parquet_2 1.00 78.2±0.23ms 209.8 MB/sec 1.60 125.4±0.34ms 130.9 MB/sec string_ree/zstd 1.00 134.2±0.46ms 122.3 MB/sec 1.34 179.2±0.36ms 91.6 MB/sec string_ree/zstd_parquet_2 1.00 92.9±0.33ms 176.5 MB/sec 1.51 140.5±0.40ms 116.8 MB/sec string_ree_95pct_null/bloom_filter 1.00 25.9±0.05ms 54.4 MB/sec 1.00 25.9±0.05ms 54.4 MB/sec string_ree_95pct_null/cdc 1.00 31.9±0.06ms 44.1 MB/sec 1.00 32.0±0.05ms 44.0 MB/sec string_ree_95pct_null/default 1.00 24.4±0.04ms 57.6 MB/sec 1.00 24.5±0.03ms 57.5 MB/sec string_ree_95pct_null/number_distinct_values 1.00 26.5±0.05ms 53.0 MB/sec 1.00 26.6±0.05ms 53.0 MB/sec string_ree_95pct_null/parquet_2 1.00 24.4±0.03ms 57.6 MB/sec 1.00 24.5±0.03ms 57.5 MB/sec string_ree_95pct_null/zstd 1.00 25.3±0.05ms 55.7 MB/sec 1.00 25.3±0.04ms 55.7 MB/sec string_ree_95pct_null/zstd_parquet_2 1.00 25.2±0.05ms 55.9 MB/sec 1.00 25.2±0.04ms 55.8 MB/sec struct_all_null/bloom_filter 1.00 369.1±1.03µs 42.7 GB/sec 1.00 369.8±1.04µs 42.6 GB/sec struct_all_null/cdc 1.00 7.3±0.09ms 2.1 GB/sec 1.05 7.7±0.18ms 2.0 GB/sec struct_all_null/default 1.00 118.7±0.59µs 132.7 GB/sec 1.00 118.5±0.77µs 132.9 GB/sec struct_all_null/number_distinct_values 1.02 120.7±0.68µs 130.5 GB/sec 1.00 118.0±0.91µs 133.5 GB/sec struct_all_null/parquet_2 1.00 121.9±0.56µs 129.2 GB/sec 1.00 121.5±0.55µs 129.6 GB/sec struct_all_null/zstd 1.01 183.7±0.60µs 85.7 GB/sec 1.00 181.0±0.72µs 87.0 GB/sec struct_all_null/zstd_parquet_2 1.01 164.3±0.53µs 95.9 GB/sec 1.00 162.4±0.81µs 97.0 GB/sec struct_non_null/bloom_filter 1.00 40.4±0.21ms 396.5 MB/sec 1.03 41.5±0.20ms 385.3 MB/sec struct_non_null/cdc 1.00 40.9±0.19ms 391.1 MB/sec 1.01 41.5±0.19ms 385.5 MB/sec struct_non_null/default 1.00 27.6±0.22ms 579.8 MB/sec 1.01 27.8±0.21ms 575.7 MB/sec struct_non_null/number_distinct_values 1.00 183.7±0.30ms 87.1 MB/sec 1.00 184.1±0.52ms 86.9 MB/sec struct_non_null/parquet_2 1.00 36.3±0.19ms 440.7 MB/sec 1.01 36.8±0.32ms 434.4 MB/sec struct_non_null/zstd 1.00 36.0±0.18ms 444.0 MB/sec 1.01 36.5±0.19ms 438.3 MB/sec struct_non_null/zstd_parquet_2 1.00 50.2±0.96ms 318.6 MB/sec 1.00 50.4±0.20ms 317.3 MB/sec struct_sparse_99pct_null/bloom_filter 1.00 6.4±0.02ms 2.5 GB/sec 1.05 6.7±0.02ms 2.3 GB/sec struct_sparse_99pct_null/cdc 1.00 13.1±0.09ms 1228.1 MB/sec 1.13 14.8±0.09ms 1090.5 MB/sec struct_sparse_99pct_null/default 1.00 5.9±0.01ms 2.7 GB/sec 1.06 6.2±0.01ms 2.5 GB/sec struct_sparse_99pct_null/number_distinct_values 1.00 7.2±0.01ms 2.2 GB/sec 1.05 7.6±0.01ms 2.1 GB/sec struct_sparse_99pct_null/parquet_2 1.00 5.9±0.01ms 2.7 GB/sec 1.06 6.3±0.01ms 2.5 GB/sec struct_sparse_99pct_null/zstd 1.00 7.3±0.02ms 2.2 GB/sec 1.05 7.6±0.01ms 2.1 GB/sec struct_sparse_99pct_null/zstd_parquet_2 1.00 6.6±0.01ms 2.4 GB/sec 1.06 7.0±0.01ms 2.2 GB/sec ``` </p> </details> <details><summary>Resource Usage</summary> **base (merge-base)** | Metric | Value | |--------|-------| | Wall time | 1895.4s | | Peak memory | 2.9 GiB | | Avg memory | 2.5 GiB | | CPU user | 1769.3s | | CPU sys | 119.8s | | Peak spill | 0 B | **branch** | Metric | Value | |--------|-------| | Wall time | 1860.4s | | Peak memory | 2.8 GiB | | Avg memory | 2.5 GiB | | CPU user | 1773.9s | | CPU sys | 84.4s | | Peak spill | 0 B | </details> --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
