adriangbot commented on PR #10554: URL: https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404772075
🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10554#issuecomment-5404308975) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) Comparing 3577093f26591aee806a9133c891b6b8e5255477 (3577093f26591aee806a9133c891b6b8e5255477) to 3577093f26591aee806a9133c891b6b8e5255477 [diff](https://github.com/apache/arrow-rs/compare/3577093f26591aee806a9133c891b6b8e5255477..3577093f26591aee806a9133c891b6b8e5255477) <details><summary>Run configuration</summary> ```yaml run benchmark arrow_writer env: BENCH_FILTER: "^[m-z]" baseline: ref: "3577093f26591aee806a9133c891b6b8e5255477" changed: ref: "3577093f26591aee806a9133c891b6b8e5255477" ``` </details> <details><summary>CPU Details (lscpu)</summary> ``` Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping: r0p1 BogoMIPS: 2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache: 32 MiB (16 instances) L3 cache: 80 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; __user pointer sanitization Vulnerability Spectre v2: Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` </details> <details><summary>Details</summary> <p> ``` group claude_parquet-exact-value-windows-10538 main ----- ---------------------------------------- ---- medium_string_shared_prefix_nullable/delta_byte_array 1.00 38.4±0.14ms 6.5 GB/sec 1.02 39.1±0.28ms 6.4 GB/sec primitive/bloom_filter 1.00 141.1±0.57ms 318.1 MB/sec 1.01 143.2±0.59ms 313.4 MB/sec primitive/cdc 1.00 152.7±0.68ms 293.8 MB/sec 1.00 152.8±0.57ms 293.6 MB/sec primitive/default 1.00 111.0±0.53ms 404.3 MB/sec 1.01 112.5±2.32ms 398.9 MB/sec primitive/number_distinct_values 1.00 358.6±1.54ms 125.1 MB/sec 1.00 360.1±1.02ms 124.6 MB/sec primitive/parquet_2 1.00 126.0±0.48ms 356.2 MB/sec 1.00 126.0±0.50ms 356.1 MB/sec primitive/zstd 1.00 140.7±0.52ms 318.9 MB/sec 1.00 140.9±0.60ms 318.4 MB/sec primitive/zstd_parquet_2 1.00 158.9±0.43ms 282.4 MB/sec 1.00 159.2±0.58ms 281.9 MB/sec primitive_all_null/bloom_filter 1.00 884.1±2.46µs 49.6 GB/sec 1.01 889.7±5.60µs 49.3 GB/sec primitive_all_null/cdc 1.01 19.1±0.41ms 2.3 GB/sec 1.00 18.9±0.46ms 2.3 GB/sec primitive_all_null/default 1.00 276.4±1.79µs 158.6 GB/sec 1.00 277.0±2.02µs 158.2 GB/sec primitive_all_null/number_distinct_values 1.00 277.2±1.92µs 158.1 GB/sec 1.00 277.5±2.43µs 157.9 GB/sec primitive_all_null/parquet_2 1.00 283.4±1.90µs 154.7 GB/sec 1.00 282.7±1.66µs 155.0 GB/sec primitive_all_null/zstd 1.00 425.6±3.12µs 103.0 GB/sec 1.00 427.0±2.02µs 102.6 GB/sec primitive_all_null/zstd_parquet_2 1.00 379.6±2.43µs 115.4 GB/sec 1.00 379.2±2.26µs 115.6 GB/sec primitive_non_null/bloom_filter 1.00 97.1±0.48ms 453.1 MB/sec 1.03 99.5±0.67ms 442.0 MB/sec primitive_non_null/cdc 1.00 81.4±0.37ms 540.6 MB/sec 1.01 82.1±0.40ms 536.2 MB/sec primitive_non_null/default 1.00 59.2±0.45ms 742.8 MB/sec 1.00 59.5±0.41ms 739.6 MB/sec primitive_non_null/number_distinct_values 1.00 475.8±3.14ms 92.5 MB/sec 1.01 479.4±3.18ms 91.8 MB/sec primitive_non_null/parquet_2 1.00 80.8±0.39ms 544.4 MB/sec 1.01 81.5±0.47ms 539.9 MB/sec primitive_non_null/zstd 1.00 89.9±0.43ms 489.6 MB/sec 1.00 90.2±0.40ms 488.0 MB/sec primitive_non_null/zstd_parquet_2 1.00 113.9±0.38ms 386.3 MB/sec 1.00 114.5±0.45ms 384.4 MB/sec primitive_sparse_99pct_null/bloom_filter 1.00 12.2±0.24ms 3.6 GB/sec 1.01 12.4±0.24ms 3.5 GB/sec primitive_sparse_99pct_null/cdc 1.00 31.6±0.31ms 1421.3 MB/sec 1.01 31.8±0.35ms 1409.2 MB/sec primitive_sparse_99pct_null/default 1.00 10.6±0.04ms 4.1 GB/sec 1.01 10.7±0.06ms 4.1 GB/sec primitive_sparse_99pct_null/number_distinct_values 1.00 14.0±0.23ms 3.1 GB/sec 1.00 14.0±0.16ms 3.1 GB/sec primitive_sparse_99pct_null/parquet_2 1.00 10.6±0.04ms 4.1 GB/sec 1.01 10.7±0.05ms 4.1 GB/sec primitive_sparse_99pct_null/zstd 1.00 14.0±0.05ms 3.1 GB/sec 1.01 14.1±0.07ms 3.1 GB/sec primitive_sparse_99pct_null/zstd_parquet_2 1.00 12.5±0.06ms 3.5 GB/sec 1.01 12.6±0.07ms 3.5 GB/sec short_string_non_null/bloom_filter 1.02 26.5±0.07ms 452.1 MB/sec 1.00 26.1±0.10ms 459.8 MB/sec short_string_non_null/cdc 1.00 20.0±0.05ms 599.8 MB/sec 1.01 20.1±0.09ms 596.1 MB/sec short_string_non_null/default 1.00 15.9±0.04ms 753.7 MB/sec 1.01 16.0±0.06ms 749.8 MB/sec short_string_non_null/number_distinct_values 1.00 67.5±0.31ms 177.8 MB/sec 1.00 67.5±0.35ms 177.9 MB/sec short_string_non_null/parquet_2 1.00 25.7±0.13ms 467.1 MB/sec 1.00 25.7±0.09ms 467.8 MB/sec short_string_non_null/zstd 1.00 36.2±0.11ms 331.9 MB/sec 1.00 36.3±0.13ms 331.0 MB/sec short_string_non_null/zstd_parquet_2 1.00 28.5±0.04ms 421.0 MB/sec 1.00 28.6±0.11ms 419.8 MB/sec small_string_distinct/delta_byte_array 1.05 993.4±10.94µs 7.9 GB/sec 1.00 941.8±8.80µs 8.3 GB/sec small_string_distinct/plain 1.00 682.7±8.64µs 11.5 GB/sec 1.02 697.6±8.29µs 11.2 GB/sec small_string_partial_prefix/delta_byte_array 1.00 987.9±6.11µs 7.9 GB/sec 1.00 985.6±5.46µs 8.0 GB/sec small_string_partial_prefix/plain 1.00 916.0±8.35µs 8.6 GB/sec 1.01 921.2±7.41µs 8.5 GB/sec small_string_shared_prefix/delta_byte_array 1.00 956.2±1.46µs 8.2 GB/sec 1.01 965.0±8.40µs 8.1 GB/sec small_string_shared_prefix/plain 1.00 1025.0±11.72µs 7.7 GB/sec 1.02 1046.6±9.35µs 7.5 GB/sec string/bloom_filter 1.05 220.9±25.38ms 2.3 GB/sec 1.00 211.0±22.81ms 2.4 GB/sec string/cdc 1.00 206.0±9.19ms 2.5 GB/sec 1.01 208.4±10.89ms 2.5 GB/sec string/default 1.00 109.5±12.37ms 4.7 GB/sec 1.23 134.8±25.20ms 3.8 GB/sec string/number_distinct_values 1.00 226.6±9.77ms 2.3 GB/sec 1.05 237.9±22.69ms 2.2 GB/sec string/parquet_2 1.00 184.5±0.50ms 2.8 GB/sec 1.03 190.1±0.67ms 2.7 GB/sec string/zstd 1.00 447.7±17.17ms 1171.1 MB/sec 1.05 469.9±18.12ms 1115.6 MB/sec string/zstd_parquet_2 1.00 410.4±20.80ms 1277.4 MB/sec 1.09 447.9±1.86ms 1170.4 MB/sec string_and_binary_view/bloom_filter 1.00 64.9±0.22ms 497.3 MB/sec 1.01 65.2±0.31ms 494.3 MB/sec string_and_binary_view/cdc 1.00 60.4±0.17ms 533.7 MB/sec 1.00 60.7±0.23ms 531.6 MB/sec string_and_binary_view/default 1.00 49.7±0.13ms 649.5 MB/sec 1.01 50.0±0.24ms 645.5 MB/sec string_and_binary_view/number_distinct_values 1.00 49.6±0.12ms 650.2 MB/sec 1.01 49.9±0.12ms 646.9 MB/sec string_and_binary_view/parquet_2 1.00 60.4±0.17ms 534.2 MB/sec 1.00 60.7±0.16ms 531.7 MB/sec string_and_binary_view/zstd 1.00 86.7±0.17ms 372.0 MB/sec 1.00 87.1±0.28ms 370.4 MB/sec string_and_binary_view/zstd_parquet_2 1.00 74.3±0.19ms 434.0 MB/sec 1.00 74.6±0.22ms 432.5 MB/sec string_dictionary/bloom_filter 1.00 83.5±0.76ms 3.1 GB/sec 1.27 106.0±0.52ms 2.4 GB/sec string_dictionary/cdc 1.00 47.7±0.53ms 5.4 GB/sec 1.37 65.3±1.50ms 3.9 GB/sec string_dictionary/default 1.00 43.1±0.84ms 6.0 GB/sec 1.46 62.8±0.33ms 4.1 GB/sec string_dictionary/number_distinct_values 1.00 73.9±0.56ms 3.5 GB/sec 1.31 96.9±0.41ms 2.7 GB/sec string_dictionary/parquet_2 1.00 47.5±0.10ms 5.4 GB/sec 1.36 64.4±0.53ms 4.0 GB/sec string_dictionary/zstd 1.00 206.5±1.05ms 1279.0 MB/sec 1.07 221.9±0.53ms 1190.4 MB/sec string_dictionary/zstd_parquet_2 1.00 191.4±0.45ms 1380.0 MB/sec 1.03 196.8±6.14ms 1342.4 MB/sec string_dictionary_low_cardinality_100/bloom_filter 1.00 33.2±0.15ms 120.8 MB/sec 1.03 34.1±0.16ms 117.6 MB/sec string_dictionary_low_cardinality_100/cdc 1.00 26.7±0.15ms 150.0 MB/sec 1.02 27.2±0.12ms 147.2 MB/sec string_dictionary_low_cardinality_100/default 1.00 24.5±0.14ms 163.7 MB/sec 1.02 24.9±0.14ms 160.8 MB/sec string_dictionary_low_cardinality_100/number_distinct_values 1.00 40.2±0.21ms 99.5 MB/sec 1.01 40.6±0.18ms 98.6 MB/sec string_dictionary_low_cardinality_100/parquet_2 1.00 24.5±0.15ms 163.5 MB/sec 1.02 25.0±0.13ms 160.6 MB/sec string_dictionary_low_cardinality_100/zstd 1.00 24.8±0.14ms 161.3 MB/sec 1.02 25.3±0.14ms 158.2 MB/sec string_dictionary_low_cardinality_100/zstd_parquet_2 1.00 24.8±0.12ms 161.2 MB/sec 1.02 25.3±0.15ms 158.3 MB/sec string_dictionary_low_cardinality_20/bloom_filter 1.00 32.6±0.16ms 122.9 MB/sec 1.01 32.9±0.14ms 121.7 MB/sec string_dictionary_low_cardinality_20/cdc 1.00 26.8±0.14ms 149.5 MB/sec 1.01 26.9±0.13ms 148.7 MB/sec string_dictionary_low_cardinality_20/default 1.00 24.4±0.15ms 163.9 MB/sec 1.01 24.6±0.14ms 163.0 MB/sec string_dictionary_low_cardinality_20/number_distinct_values 1.00 40.0±0.27ms 100.2 MB/sec 1.00 40.1±0.17ms 100.0 MB/sec string_dictionary_low_cardinality_20/parquet_2 1.00 24.5±0.16ms 163.7 MB/sec 1.00 24.6±0.14ms 162.9 MB/sec string_dictionary_low_cardinality_20/zstd 1.00 24.6±0.15ms 162.9 MB/sec 1.01 24.8±0.14ms 161.2 MB/sec string_dictionary_low_cardinality_20/zstd_parquet_2 1.00 24.6±0.14ms 162.9 MB/sec 1.01 24.7±0.15ms 162.0 MB/sec string_dictionary_low_cardinality_400/bloom_filter 1.00 34.3±0.15ms 116.8 MB/sec 1.00 34.2±0.14ms 117.2 MB/sec string_dictionary_low_cardinality_400/cdc 1.01 27.4±0.16ms 146.4 MB/sec 1.00 27.1±0.14ms 147.9 MB/sec string_dictionary_low_cardinality_400/default 1.00 25.1±0.14ms 160.0 MB/sec 1.00 25.0±0.20ms 160.4 MB/sec string_dictionary_low_cardinality_400/number_distinct_values 1.00 40.9±0.15ms 98.2 MB/sec 1.00 40.9±0.29ms 98.2 MB/sec string_dictionary_low_cardinality_400/parquet_2 1.00 25.1±0.13ms 159.7 MB/sec 1.00 25.0±0.18ms 160.5 MB/sec string_dictionary_low_cardinality_400/zstd 1.01 25.7±0.13ms 156.1 MB/sec 1.00 25.5±0.19ms 157.1 MB/sec string_dictionary_low_cardinality_400/zstd_parquet_2 1.01 25.7±0.15ms 156.4 MB/sec 1.00 25.5±0.16ms 157.5 MB/sec string_non_null/bloom_filter 1.00 231.3±10.92ms 2.2 GB/sec 1.11 257.5±29.65ms 2035.1 MB/sec string_non_null/cdc 1.00 244.3±5.35ms 2.1 GB/sec 1.06 259.1±8.65ms 2022.3 MB/sec string_non_null/default 1.00 119.5±14.90ms 4.3 GB/sec 1.19 142.7±26.52ms 3.6 GB/sec string_non_null/number_distinct_values 1.00 274.5±4.60ms 1909.2 MB/sec 1.09 298.9±11.76ms 1752.9 MB/sec string_non_null/parquet_2 1.00 124.6±5.80ms 4.1 GB/sec 1.02 126.5±12.57ms 4.0 GB/sec string_non_null/zstd 1.00 551.7±5.96ms 949.7 MB/sec 1.00 551.6±2.15ms 950.0 MB/sec string_non_null/zstd_parquet_2 1.00 502.0±8.17ms 1043.8 MB/sec 1.00 504.4±6.29ms 1038.8 MB/sec string_ree/bloom_filter 1.00 154.3±0.78ms 106.3 MB/sec 1.02 158.0±1.32ms 103.9 MB/sec string_ree/cdc 1.00 174.2±0.58ms 94.2 MB/sec 1.30 226.7±1.22ms 72.4 MB/sec string_ree/default 1.00 126.6±0.54ms 129.6 MB/sec 1.03 130.2±1.81ms 126.0 MB/sec string_ree/number_distinct_values 1.00 114.6±0.99ms 143.1 MB/sec 1.47 168.0±1.12ms 97.7 MB/sec string_ree/parquet_2 1.00 78.6±0.29ms 208.6 MB/sec 1.66 130.2±0.41ms 126.0 MB/sec string_ree/zstd 1.00 134.9±0.45ms 121.6 MB/sec 1.37 185.4±1.74ms 88.5 MB/sec string_ree/zstd_parquet_2 1.00 93.0±0.27ms 176.5 MB/sec 1.56 145.2±0.77ms 113.0 MB/sec string_ree_95pct_null/bloom_filter 1.00 26.1±0.15ms 53.9 MB/sec 1.01 26.3±0.14ms 53.5 MB/sec string_ree_95pct_null/cdc 1.00 32.2±0.17ms 43.7 MB/sec 1.00 32.3±0.16ms 43.5 MB/sec string_ree_95pct_null/default 1.00 24.6±0.11ms 57.1 MB/sec 1.00 24.7±0.08ms 56.9 MB/sec string_ree_95pct_null/number_distinct_values 1.00 26.8±0.11ms 52.6 MB/sec 1.00 26.9±0.08ms 52.4 MB/sec string_ree_95pct_null/parquet_2 1.00 24.6±0.08ms 57.1 MB/sec 1.01 24.8±0.09ms 56.8 MB/sec string_ree_95pct_null/zstd 1.00 25.6±0.15ms 55.0 MB/sec 1.00 25.6±0.10ms 55.0 MB/sec string_ree_95pct_null/zstd_parquet_2 1.00 25.3±0.08ms 55.6 MB/sec 1.01 25.6±0.11ms 55.0 MB/sec struct_all_null/bloom_filter 1.00 370.3±1.63µs 42.5 GB/sec 1.00 371.8±3.51µs 42.3 GB/sec struct_all_null/cdc 1.01 7.7±0.20ms 2.0 GB/sec 1.00 7.6±0.09ms 2.1 GB/sec struct_all_null/default 1.00 118.2±0.83µs 133.3 GB/sec 1.00 118.2±0.92µs 133.2 GB/sec struct_all_null/number_distinct_values 1.00 118.8±0.98µs 132.6 GB/sec 1.00 118.2±0.99µs 133.2 GB/sec struct_all_null/parquet_2 1.00 121.0±0.74µs 130.2 GB/sec 1.00 120.5±0.81µs 130.7 GB/sec struct_all_null/zstd 1.00 181.6±1.02µs 86.7 GB/sec 1.00 181.2±0.89µs 86.9 GB/sec struct_all_null/zstd_parquet_2 1.00 162.8±0.90µs 96.8 GB/sec 1.00 162.9±0.86µs 96.6 GB/sec struct_non_null/bloom_filter 1.00 41.4±0.23ms 386.3 MB/sec 1.02 42.2±0.28ms 379.5 MB/sec struct_non_null/cdc 1.00 42.0±0.20ms 381.0 MB/sec 1.00 41.8±0.20ms 382.8 MB/sec struct_non_null/default 1.00 28.2±0.19ms 568.3 MB/sec 1.00 28.2±0.23ms 567.5 MB/sec struct_non_null/number_distinct_values 1.00 186.2±1.05ms 85.9 MB/sec 1.00 186.6±0.73ms 85.7 MB/sec struct_non_null/parquet_2 1.01 37.3±1.03ms 429.2 MB/sec 1.00 37.0±0.21ms 432.7 MB/sec struct_non_null/zstd 1.00 36.7±0.18ms 436.1 MB/sec 1.00 36.8±0.19ms 434.4 MB/sec struct_non_null/zstd_parquet_2 1.00 50.7±0.21ms 315.7 MB/sec 1.00 50.9±0.20ms 314.6 MB/sec struct_sparse_99pct_null/bloom_filter 1.01 7.0±0.10ms 2.3 GB/sec 1.00 6.9±0.05ms 2.3 GB/sec struct_sparse_99pct_null/cdc 1.00 15.0±0.10ms 1074.0 MB/sec 1.00 15.1±0.14ms 1070.3 MB/sec struct_sparse_99pct_null/default 1.00 6.3±0.08ms 2.5 GB/sec 1.00 6.3±0.03ms 2.5 GB/sec struct_sparse_99pct_null/number_distinct_values 1.00 7.6±0.05ms 2.1 GB/sec 1.02 7.7±0.21ms 2.0 GB/sec struct_sparse_99pct_null/parquet_2 1.00 6.3±0.04ms 2.5 GB/sec 1.01 6.4±0.04ms 2.5 GB/sec struct_sparse_99pct_null/zstd 1.00 7.7±0.03ms 2.1 GB/sec 1.00 7.7±0.04ms 2.0 GB/sec struct_sparse_99pct_null/zstd_parquet_2 1.00 7.1±0.03ms 2.2 GB/sec 1.01 7.1±0.07ms 2.2 GB/sec ``` </p> </details> <details><summary>Resource Usage</summary> **base (merge-base)** | Metric | Value | |--------|-------| | Wall time | 1915.4s | | Peak memory | 2.9 GiB | | Avg memory | 2.5 GiB | | CPU user | 1772.2s | | CPU sys | 139.8s | | Peak spill | 0 B | **branch** | Metric | Value | |--------|-------| | Wall time | 1870.4s | | Peak memory | 2.9 GiB | | Avg memory | 2.5 GiB | | CPU user | 1794.2s | | CPU sys | 74.1s | | Peak spill | 0 B | </details> --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
