adriangbot commented on PR #11424: URL: https://github.com/apache/arrow-rs/pull/11424#issuecomment-6123220676
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11424#issuecomment-6122594490) **Target:** arrow_writer Comparing many-opt-parquet-writer (f8550ef406fe3e7ff82d94ed7cc9968f04cf1146) to 0706d31 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/0706d317a94098cec4b631031efcdf16f51ed1f5..f8550ef406fe3e7ff82d94ed7cc9968f04cf1146) <details><summary>Run configuration</summary> ```yaml run benchmark arrow_writer env: BENCH_FILTER: dictionary resources: arch: amd64 shards: 1 ``` </details> <details><summary>Details</summary> ``` group base changed ----- ---- ------- bool_dictionary/bloom_filter 1.14 12.8±0.16ms 78.1 MElem/sec 1.00 11.2±0.15ms 89.0 MElem/sec bool_dictionary/cdc 1.20 10.7±0.22ms 93.9 MElem/sec 1.00 8.9±0.08ms 112.3 MElem/sec bool_dictionary/default 1.30 6.6±0.13ms 150.9 MElem/sec 1.00 5.1±0.10ms 196.3 MElem/sec bool_dictionary/number_distinct_values 1.11 18.2±0.09ms 54.8 MElem/sec 1.00 16.4±0.19ms 61.0 MElem/sec bool_dictionary/parquet_2 1.28 7.2±0.06ms 139.1 MElem/sec 1.00 5.6±0.05ms 177.5 MElem/sec bool_dictionary/zstd 1.30 6.6±0.06ms 150.7 MElem/sec 1.00 5.1±0.14ms 195.7 MElem/sec bool_dictionary/zstd_parquet_2 1.26 7.2±0.06ms 138.2 MElem/sec 1.00 5.7±0.09ms 174.7 MElem/sec decimal128_dictionary/bloom_filter 1.14 45.1±1.60ms 22.2 MElem/sec 1.00 39.5±2.38ms 25.3 MElem/sec decimal128_dictionary/cdc 1.16 36.8±0.16ms 27.2 MElem/sec 1.00 31.6±0.49ms 31.6 MElem/sec decimal128_dictionary/default 1.17 32.6±0.62ms 30.7 MElem/sec 1.00 27.8±1.68ms 36.0 MElem/sec decimal128_dictionary/number_distinct_values 1.10 65.5±2.09ms 15.3 MElem/sec 1.00 59.6±3.28ms 16.8 MElem/sec decimal128_dictionary/parquet_2 1.06 61.1±0.82ms 16.4 MElem/sec 1.00 57.5±2.47ms 17.4 MElem/sec decimal128_dictionary/zstd 1.14 36.1±0.42ms 27.7 MElem/sec 1.00 31.7±1.12ms 31.5 MElem/sec decimal128_dictionary/zstd_parquet_2 1.09 65.7±1.31ms 15.2 MElem/sec 1.00 60.2±1.72ms 16.6 MElem/sec decimal128_dictionary_1pct_cardinality/bloom_filter 1.06 29.0±1.63ms 34.4 MElem/sec 1.00 27.3±0.65ms 36.6 MElem/sec decimal128_dictionary_1pct_cardinality/cdc 1.00 28.9±0.32ms 34.6 MElem/sec 1.04 30.1±0.41ms 33.2 MElem/sec decimal128_dictionary_1pct_cardinality/default 1.00 26.4±1.09ms 37.8 MElem/sec 1.01 26.7±0.34ms 37.5 MElem/sec decimal128_dictionary_1pct_cardinality/number_distinct_values 1.00 40.1±0.64ms 24.9 MElem/sec 1.03 41.5±2.17ms 24.1 MElem/sec decimal128_dictionary_1pct_cardinality/parquet_2 1.00 26.1±1.13ms 38.4 MElem/sec 1.08 28.1±0.93ms 35.5 MElem/sec decimal128_dictionary_1pct_cardinality/zstd 1.00 27.6±0.17ms 36.2 MElem/sec 1.08 29.9±0.80ms 33.5 MElem/sec decimal128_dictionary_1pct_cardinality/zstd_parquet_2 1.00 27.4±0.14ms 36.5 MElem/sec 1.08 29.5±1.25ms 33.9 MElem/sec float64_dictionary/bloom_filter 1.00 19.9±0.77ms 50.2 MElem/sec 1.15 22.9±0.71ms 43.7 MElem/sec float64_dictionary/cdc 1.11 17.7±0.21ms 56.6 MElem/sec 1.00 15.9±0.63ms 63.0 MElem/sec float64_dictionary/default 1.00 13.0±0.28ms 76.8 MElem/sec 1.14 14.8±0.51ms 67.5 MElem/sec float64_dictionary/number_distinct_values 1.04 46.9±1.44ms 21.3 MElem/sec 1.00 44.9±2.80ms 22.3 MElem/sec float64_dictionary/parquet_2 1.00 12.6±0.43ms 79.6 MElem/sec 1.17 14.7±0.73ms 68.1 MElem/sec float64_dictionary/zstd 1.00 17.8±0.66ms 56.3 MElem/sec 1.14 20.3±0.90ms 49.2 MElem/sec float64_dictionary/zstd_parquet_2 1.00 17.3±0.22ms 57.8 MElem/sec 1.15 19.9±0.27ms 50.3 MElem/sec float64_dictionary_1pct_cardinality/bloom_filter 1.36 8.6±0.09ms 116.8 MElem/sec 1.00 6.3±0.51ms 158.9 MElem/sec float64_dictionary_1pct_cardinality/cdc 1.23 11.2±0.05ms 89.4 MElem/sec 1.00 9.1±0.64ms 110.0 MElem/sec float64_dictionary_1pct_cardinality/default 1.33 8.3±0.15ms 119.8 MElem/sec 1.00 6.3±2.52ms 159.2 MElem/sec float64_dictionary_1pct_cardinality/number_distinct_values 1.13 21.6±0.49ms 46.4 MElem/sec 1.00 19.1±0.82ms 52.2 MElem/sec float64_dictionary_1pct_cardinality/parquet_2 1.34 8.3±0.12ms 120.2 MElem/sec 1.00 6.2±1.66ms 161.4 MElem/sec float64_dictionary_1pct_cardinality/zstd 1.19 10.0±0.26ms 99.6 MElem/sec 1.00 8.4±3.14ms 118.9 MElem/sec float64_dictionary_1pct_cardinality/zstd_parquet_2 1.26 9.9±0.08ms 101.2 MElem/sec 1.00 7.9±2.14ms 127.0 MElem/sec int64_dictionary/bloom_filter 1.00 19.1±0.70ms 52.5 MElem/sec 1.12 21.4±1.05ms 46.7 MElem/sec int64_dictionary/cdc 1.14 16.7±0.22ms 59.7 MElem/sec 1.00 14.6±0.04ms 68.4 MElem/sec int64_dictionary/default 1.20 11.6±0.31ms 86.6 MElem/sec 1.00 9.7±0.34ms 103.5 MElem/sec int64_dictionary/number_distinct_values 1.02 45.7±1.56ms 21.9 MElem/sec 1.00 44.7±1.22ms 22.3 MElem/sec int64_dictionary/parquet_2 1.13 13.2±0.26ms 75.8 MElem/sec 1.00 11.7±0.42ms 85.8 MElem/sec int64_dictionary/zstd 1.14 13.8±0.17ms 72.4 MElem/sec 1.00 12.1±0.30ms 82.5 MElem/sec int64_dictionary/zstd_parquet_2 1.12 17.1±0.16ms 58.6 MElem/sec 1.00 15.2±0.34ms 65.7 MElem/sec int64_dictionary_1pct_cardinality/bloom_filter 1.37 7.3±0.07ms 136.3 MElem/sec 1.00 5.4±1.03ms 186.2 MElem/sec int64_dictionary_1pct_cardinality/cdc 1.26 10.0±0.29ms 100.1 MElem/sec 1.00 8.0±0.40ms 125.8 MElem/sec int64_dictionary_1pct_cardinality/default 1.30 7.0±0.05ms 142.0 MElem/sec 1.00 5.4±3.10ms 184.5 MElem/sec int64_dictionary_1pct_cardinality/number_distinct_values 1.09 20.4±0.23ms 49.1 MElem/sec 1.00 18.7±4.75ms 53.5 MElem/sec int64_dictionary_1pct_cardinality/parquet_2 1.41 7.0±0.05ms 141.9 MElem/sec 1.00 5.0±0.81ms 199.4 MElem/sec int64_dictionary_1pct_cardinality/zstd 1.16 8.8±0.09ms 113.6 MElem/sec 1.00 7.6±7.97ms 132.0 MElem/sec int64_dictionary_1pct_cardinality/zstd_parquet_2 1.32 8.7±0.06ms 115.2 MElem/sec 1.00 6.6±0.53ms 152.2 MElem/sec primitive_dictionary/bloom_filter 1.00 20.1±0.41ms 49.7 MElem/sec 1.12 22.6±0.15ms 44.2 MElem/sec primitive_dictionary/cdc 1.01 19.0±0.52ms 52.6 MElem/sec 1.00 18.8±0.23ms 53.2 MElem/sec primitive_dictionary/default 1.06 13.8±0.13ms 72.6 MElem/sec 1.00 13.0±0.23ms 76.7 MElem/sec primitive_dictionary/number_distinct_values 1.00 49.4±3.19ms 20.2 MElem/sec 1.01 49.8±0.95ms 20.1 MElem/sec primitive_dictionary/parquet_2 1.06 15.5±0.15ms 64.7 MElem/sec 1.00 14.6±0.07ms 68.5 MElem/sec primitive_dictionary/zstd 1.05 15.7±0.14ms 63.7 MElem/sec 1.00 15.0±0.20ms 66.7 MElem/sec primitive_dictionary/zstd_parquet_2 1.06 18.6±0.38ms 53.9 MElem/sec 1.00 17.5±0.23ms 57.1 MElem/sec primitive_dictionary_1pct_cardinality/bloom_filter 1.38 7.1±0.09ms 140.3 MElem/sec 1.00 5.2±0.48ms 193.3 MElem/sec primitive_dictionary_1pct_cardinality/cdc 1.27 9.8±0.11ms 101.9 MElem/sec 1.00 7.7±0.55ms 129.4 MElem/sec primitive_dictionary_1pct_cardinality/default 1.38 6.8±0.03ms 146.4 MElem/sec 1.00 5.0±1.22ms 201.3 MElem/sec primitive_dictionary_1pct_cardinality/number_distinct_values 1.11 19.7±0.39ms 50.7 MElem/sec 1.00 17.8±1.11ms 56.3 MElem/sec primitive_dictionary_1pct_cardinality/parquet_2 1.42 6.8±0.03ms 146.1 MElem/sec 1.00 4.8±1.47ms 206.8 MElem/sec primitive_dictionary_1pct_cardinality/zstd 1.28 8.5±0.21ms 117.1 MElem/sec 1.00 6.7±2.44ms 150.0 MElem/sec primitive_dictionary_1pct_cardinality/zstd_parquet_2 1.37 8.4±0.14ms 118.6 MElem/sec 1.00 6.2±0.51ms 162.0 MElem/sec string_dictionary/bloom_filter 1.05 131.1±9.54ms 7.6 MElem/sec 1.00 124.5±11.27ms 8.0 MElem/sec string_dictionary/cdc 1.12 85.8±2.70ms 11.7 MElem/sec 1.00 76.6±13.14ms 13.1 MElem/sec string_dictionary/default 1.00 83.6±0.86ms 12.0 MElem/sec 1.04 86.6±10.41ms 11.6 MElem/sec string_dictionary/number_distinct_values 1.16 173.6±12.44ms 5.8 MElem/sec 1.00 150.2±10.08ms 6.7 MElem/sec string_dictionary/parquet_2 1.33 78.6±8.97ms 12.7 MElem/sec 1.00 59.3±5.84ms 16.9 MElem/sec string_dictionary/zstd 1.17 205.2±4.64ms 4.9 MElem/sec 1.00 175.3±3.05ms 5.7 MElem/sec string_dictionary/zstd_parquet_2 1.00 174.7±6.25ms 5.7 MElem/sec 1.02 178.2±4.85ms 5.6 MElem/sec string_dictionary_1pct_cardinality/bloom_filter 1.03 12.8±0.09ms 78.4 MElem/sec 1.00 12.4±0.07ms 80.7 MElem/sec string_dictionary_1pct_cardinality/cdc 1.01 15.8±0.33ms 63.4 MElem/sec 1.00 15.6±0.34ms 64.0 MElem/sec string_dictionary_1pct_cardinality/default 1.02 12.4±0.08ms 80.6 MElem/sec 1.00 12.1±0.10ms 82.5 MElem/sec string_dictionary_1pct_cardinality/number_distinct_values 1.01 28.2±0.16ms 35.5 MElem/sec 1.00 27.8±0.60ms 35.9 MElem/sec string_dictionary_1pct_cardinality/parquet_2 1.03 12.4±0.06ms 80.4 MElem/sec 1.00 12.1±0.17ms 82.5 MElem/sec string_dictionary_1pct_cardinality/zstd 1.01 14.3±0.07ms 70.1 MElem/sec 1.00 14.1±0.11ms 70.7 MElem/sec string_dictionary_1pct_cardinality/zstd_parquet_2 1.00 14.1±0.29ms 71.1 MElem/sec 1.00 14.0±0.18ms 71.3 MElem/sec string_dictionary_low_cardinality_100/bloom_filter 1.55 13.4±0.14ms 74.6 MElem/sec 1.00 8.7±0.11ms 115.4 MElem/sec string_dictionary_low_cardinality_100/cdc 1.44 15.7±0.15ms 63.9 MElem/sec 1.00 10.9±0.08ms 92.1 MElem/sec string_dictionary_low_cardinality_100/default 1.56 13.2±0.17ms 75.5 MElem/sec 1.00 8.5±0.12ms 117.5 MElem/sec string_dictionary_low_cardinality_100/number_distinct_values 1.25 35.3±0.56ms 28.3 MElem/sec 1.00 28.2±0.13ms 35.5 MElem/sec string_dictionary_low_cardinality_100/parquet_2 1.54 13.2±0.09ms 75.9 MElem/sec 1.00 8.5±0.08ms 117.1 MElem/sec string_dictionary_low_cardinality_100/zstd 1.57 13.6±0.13ms 73.6 MElem/sec 1.00 8.7±0.05ms 115.6 MElem/sec string_dictionary_low_cardinality_100/zstd_parquet_2 1.56 13.4±0.09ms 74.5 MElem/sec 1.00 8.6±0.06ms 116.0 MElem/sec string_dictionary_low_cardinality_20/bloom_filter 1.47 13.1±0.14ms 76.1 MElem/sec 1.00 8.9±0.12ms 112.1 MElem/sec string_dictionary_low_cardinality_20/cdc 1.41 15.9±0.36ms 63.0 MElem/sec 1.00 11.3±0.21ms 88.5 MElem/sec string_dictionary_low_cardinality_20/default 1.55 13.6±0.27ms 73.8 MElem/sec 1.00 8.7±0.14ms 114.4 MElem/sec string_dictionary_low_cardinality_20/number_distinct_values 1.18 34.1±0.58ms 29.4 MElem/sec 1.00 28.8±0.58ms 34.7 MElem/sec string_dictionary_low_cardinality_20/parquet_2 1.65 14.5±0.66ms 69.1 MElem/sec 1.00 8.8±0.17ms 113.9 MElem/sec string_dictionary_low_cardinality_20/zstd 1.47 13.3±0.14ms 75.4 MElem/sec 1.00 9.0±0.22ms 111.0 MElem/sec string_dictionary_low_cardinality_20/zstd_parquet_2 1.50 13.4±0.24ms 74.5 MElem/sec 1.00 8.9±0.23ms 111.8 MElem/sec string_dictionary_low_cardinality_400/bloom_filter 1.59 13.4±0.15ms 74.5 MElem/sec 1.00 8.4±0.04ms 118.8 MElem/sec string_dictionary_low_cardinality_400/cdc 1.47 15.9±0.52ms 63.1 MElem/sec 1.00 10.8±0.15ms 92.8 MElem/sec string_dictionary_low_cardinality_400/default 1.66 13.8±0.22ms 72.6 MElem/sec 1.00 8.3±0.08ms 120.8 MElem/sec string_dictionary_low_cardinality_400/number_distinct_values 1.19 33.5±0.32ms 29.9 MElem/sec 1.00 28.2±0.33ms 35.5 MElem/sec string_dictionary_low_cardinality_400/parquet_2 1.58 13.1±0.08ms 76.3 MElem/sec 1.00 8.3±0.11ms 120.7 MElem/sec string_dictionary_low_cardinality_400/zstd 1.56 13.6±0.10ms 73.5 MElem/sec 1.00 8.7±0.10ms 114.6 MElem/sec string_dictionary_low_cardinality_400/zstd_parquet_2 1.58 13.7±0.22ms 73.0 MElem/sec 1.00 8.7±0.20ms 115.0 MElem/sec ``` </details> <details><summary>Per-runner information</summary> <details><summary>arrow_writer — shard 1/1</summary> **Node:** gk3-benchmark-cluster-nap-z3xgfn7r-37ade6db-hvv7 **Instance:** c4-highmem-16-lssd (12 vCPU / 65 GiB) **uname:** ```text Linux bench-c6122594490-3334-2lbxj 6.12.94+ #1 SMP Fri Aug 21 07:56:42 UTC 2026 x86_64 GNU/Linux ``` **BENCH_COMMAND:** ```sh cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer ``` <details><summary>CPU Details (lscpu)</summary> ```text Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 52 bits physical, 57 bits virtual Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: GenuineIntel Model name: Intel(R) Xeon(R) 6985P-C CPU @ 2.30GHz CPU family: 6 Model: 173 Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 1 Stepping: 1 CPU(s) scaling MHz: 175% CPU max MHz: 2300.0000 CPU min MHz: 2300.0000 BogoMIPS: 4600.00 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd arat avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid bus_lock_detect cldemote movdiri movdir64b fsrm md_clear serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities Hypervisor vendor: KVM Virtualization type: full L1d cache: 384 KiB (8 instances) L1i cache: 512 KiB (8 instances) L2 cache: 16 MiB (8 instances) L3 cache: 504 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` </details> <details><summary>Resource Usage</summary> **base (merge-base)** | Metric | Value | |--------|-------| | Wall time | 1055.2s | | Peak memory | 1.7 GiB | | Avg memory | 96.8 MiB | | CPU user | 1022.5s | | CPU sys | 28.7s | | Peak spill | 0 B | **branch** | Metric | Value | |--------|-------| | Wall time | 1065.3s | | Peak memory | 1.2 GiB | | Avg memory | 91.5 MiB | | CPU user | 1010.8s | | CPU sys | 49.4s | | Peak spill | 0 B | </details> </details> </details> --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner <!-- benchmark-request:6122594490:finish:5040d6ad9ef14fed268c095dfef35f20 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
