adriangbot commented on PR #11524: URL: https://github.com/apache/arrow-rs/pull/11524#issuecomment-6125674463
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11524#issuecomment-6124985721) **Target:** arrow_writer Comparing improve-parquet-dictionary-computation (b1f10c7ae74037b17d2c72eec0f4137ea0f00332) to 0706d31 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/0706d317a94098cec4b631031efcdf16f51ed1f5..b1f10c7ae74037b17d2c72eec0f4137ea0f00332) <details><summary>Run configuration</summary> ```yaml run benchmark arrow_writer env: BENCH_FILTER: dictionary resources: arch: amd64 shards: 1 ``` </details> <details><summary>Details</summary> ``` group base changed ----- ---- ------- bool_dictionary/bloom_filter 1.00 13.0±0.17ms 77.0 MElem/sec 1.00 13.0±0.15ms 77.1 MElem/sec bool_dictionary/cdc 1.01 10.7±0.25ms 93.1 MElem/sec 1.00 10.6±0.09ms 94.0 MElem/sec bool_dictionary/default 1.01 6.7±0.11ms 150.1 MElem/sec 1.00 6.6±0.11ms 151.7 MElem/sec bool_dictionary/number_distinct_values 1.03 18.7±0.20ms 53.4 MElem/sec 1.00 18.3±0.20ms 54.8 MElem/sec bool_dictionary/parquet_2 1.00 7.2±0.19ms 138.0 MElem/sec 1.00 7.2±0.11ms 138.0 MElem/sec bool_dictionary/zstd 1.00 6.7±0.08ms 148.6 MElem/sec 1.00 6.7±0.15ms 149.2 MElem/sec bool_dictionary/zstd_parquet_2 1.00 7.3±0.11ms 137.1 MElem/sec 1.00 7.3±0.05ms 137.4 MElem/sec decimal128_dictionary/bloom_filter 1.02 45.9±1.31ms 21.8 MElem/sec 1.00 44.8±0.89ms 22.3 MElem/sec decimal128_dictionary/cdc 1.00 37.1±0.38ms 27.0 MElem/sec 1.00 37.1±0.40ms 26.9 MElem/sec decimal128_dictionary/default 1.00 34.1±0.69ms 29.3 MElem/sec 1.00 34.1±0.70ms 29.3 MElem/sec decimal128_dictionary/number_distinct_values 1.00 65.3±1.25ms 15.3 MElem/sec 1.06 69.3±2.13ms 14.4 MElem/sec decimal128_dictionary/parquet_2 1.01 64.7±1.34ms 15.5 MElem/sec 1.00 64.0±1.26ms 15.6 MElem/sec decimal128_dictionary/zstd 1.03 39.0±1.28ms 25.7 MElem/sec 1.00 37.9±0.66ms 26.4 MElem/sec decimal128_dictionary/zstd_parquet_2 1.01 68.2±1.46ms 14.7 MElem/sec 1.00 67.3±1.12ms 14.9 MElem/sec decimal128_dictionary_1pct_cardinality/bloom_filter 1.00 26.6±0.49ms 37.6 MElem/sec 1.02 27.0±0.53ms 37.0 MElem/sec decimal128_dictionary_1pct_cardinality/cdc 1.00 29.0±0.30ms 34.5 MElem/sec 1.01 29.3±0.41ms 34.1 MElem/sec decimal128_dictionary_1pct_cardinality/default 1.00 25.9±0.37ms 38.6 MElem/sec 1.04 27.0±0.66ms 37.1 MElem/sec decimal128_dictionary_1pct_cardinality/number_distinct_values 1.00 41.0±0.75ms 24.4 MElem/sec 1.05 43.0±0.95ms 23.3 MElem/sec decimal128_dictionary_1pct_cardinality/parquet_2 1.00 25.9±0.34ms 38.6 MElem/sec 1.02 26.6±0.65ms 37.7 MElem/sec decimal128_dictionary_1pct_cardinality/zstd 1.00 27.9±0.38ms 35.9 MElem/sec 1.03 28.8±0.60ms 34.8 MElem/sec decimal128_dictionary_1pct_cardinality/zstd_parquet_2 1.00 28.1±0.45ms 35.6 MElem/sec 1.04 29.1±0.68ms 34.4 MElem/sec float64_dictionary/bloom_filter 1.00 19.6±0.08ms 50.9 MElem/sec 1.01 19.8±0.48ms 50.5 MElem/sec float64_dictionary/cdc 1.00 17.9±0.23ms 55.9 MElem/sec 1.00 17.8±0.48ms 56.2 MElem/sec float64_dictionary/default 1.00 12.5±0.09ms 80.0 MElem/sec 1.01 12.7±0.40ms 79.0 MElem/sec float64_dictionary/number_distinct_values 1.00 46.7±0.73ms 21.4 MElem/sec 1.01 47.2±0.69ms 21.2 MElem/sec float64_dictionary/parquet_2 1.02 12.6±0.13ms 79.7 MElem/sec 1.00 12.4±0.31ms 80.9 MElem/sec float64_dictionary/zstd 1.02 17.9±0.15ms 56.0 MElem/sec 1.00 17.4±0.20ms 57.4 MElem/sec float64_dictionary/zstd_parquet_2 1.01 17.4±0.31ms 57.4 MElem/sec 1.00 17.2±0.24ms 58.2 MElem/sec float64_dictionary_1pct_cardinality/bloom_filter 1.00 8.7±0.07ms 114.6 MElem/sec 1.00 8.7±0.19ms 114.9 MElem/sec float64_dictionary_1pct_cardinality/cdc 1.00 11.4±0.14ms 87.9 MElem/sec 1.00 11.4±0.09ms 87.8 MElem/sec float64_dictionary_1pct_cardinality/default 1.00 8.4±0.13ms 118.7 MElem/sec 1.00 8.4±0.12ms 119.0 MElem/sec float64_dictionary_1pct_cardinality/number_distinct_values 1.00 22.0±0.29ms 45.4 MElem/sec 1.00 22.0±0.56ms 45.4 MElem/sec float64_dictionary_1pct_cardinality/parquet_2 1.01 8.5±0.11ms 118.3 MElem/sec 1.00 8.4±0.12ms 119.1 MElem/sec float64_dictionary_1pct_cardinality/zstd 1.00 10.2±0.12ms 97.9 MElem/sec 1.00 10.3±0.27ms 97.4 MElem/sec float64_dictionary_1pct_cardinality/zstd_parquet_2 1.00 10.1±0.13ms 99.2 MElem/sec 1.00 10.1±0.07ms 99.0 MElem/sec int64_dictionary/bloom_filter 1.01 18.8±0.33ms 53.2 MElem/sec 1.00 18.7±0.21ms 53.5 MElem/sec int64_dictionary/cdc 1.00 16.6±0.18ms 60.4 MElem/sec 1.00 16.6±0.16ms 60.3 MElem/sec int64_dictionary/default 1.01 11.4±0.23ms 87.4 MElem/sec 1.00 11.3±0.11ms 88.4 MElem/sec int64_dictionary/number_distinct_values 1.03 46.7±1.30ms 21.4 MElem/sec 1.00 45.3±0.58ms 22.1 MElem/sec int64_dictionary/parquet_2 1.01 13.3±0.24ms 75.2 MElem/sec 1.00 13.2±0.17ms 75.9 MElem/sec int64_dictionary/zstd 1.02 14.0±0.15ms 71.3 MElem/sec 1.00 13.8±0.10ms 72.7 MElem/sec int64_dictionary/zstd_parquet_2 1.02 17.3±0.27ms 57.9 MElem/sec 1.00 17.0±0.22ms 59.0 MElem/sec int64_dictionary_1pct_cardinality/bloom_filter 1.03 7.5±0.13ms 133.6 MElem/sec 1.00 7.3±0.06ms 137.2 MElem/sec int64_dictionary_1pct_cardinality/cdc 1.00 10.1±0.10ms 98.7 MElem/sec 1.00 10.1±0.11ms 98.8 MElem/sec int64_dictionary_1pct_cardinality/default 1.01 7.2±0.05ms 139.8 MElem/sec 1.00 7.0±0.06ms 141.9 MElem/sec int64_dictionary_1pct_cardinality/number_distinct_values 1.01 20.8±0.23ms 48.2 MElem/sec 1.00 20.5±0.25ms 48.8 MElem/sec int64_dictionary_1pct_cardinality/parquet_2 1.01 7.2±0.08ms 138.6 MElem/sec 1.00 7.1±0.08ms 140.2 MElem/sec int64_dictionary_1pct_cardinality/zstd 1.02 9.0±0.12ms 111.1 MElem/sec 1.00 8.8±0.10ms 113.3 MElem/sec int64_dictionary_1pct_cardinality/zstd_parquet_2 1.01 8.8±0.08ms 113.3 MElem/sec 1.00 8.7±0.25ms 114.9 MElem/sec primitive_dictionary/bloom_filter 1.05 20.5±0.48ms 48.7 MElem/sec 1.00 19.6±0.12ms 51.0 MElem/sec primitive_dictionary/cdc 1.04 19.5±0.46ms 51.3 MElem/sec 1.00 18.8±0.11ms 53.3 MElem/sec primitive_dictionary/default 1.05 14.1±0.20ms 70.7 MElem/sec 1.00 13.5±0.11ms 74.3 MElem/sec primitive_dictionary/number_distinct_values 1.25 59.4±1.70ms 16.8 MElem/sec 1.00 47.6±0.84ms 21.0 MElem/sec primitive_dictionary/parquet_2 1.06 15.9±0.21ms 62.8 MElem/sec 1.00 15.1±0.16ms 66.3 MElem/sec primitive_dictionary/zstd 1.06 16.1±0.37ms 62.1 MElem/sec 1.00 15.2±0.08ms 65.7 MElem/sec primitive_dictionary/zstd_parquet_2 1.05 18.7±0.29ms 53.6 MElem/sec 1.00 17.8±0.11ms 56.3 MElem/sec primitive_dictionary_1pct_cardinality/bloom_filter 1.04 7.4±0.12ms 135.9 MElem/sec 1.00 7.1±0.06ms 141.4 MElem/sec primitive_dictionary_1pct_cardinality/cdc 1.04 10.1±0.16ms 99.1 MElem/sec 1.00 9.7±0.11ms 103.1 MElem/sec primitive_dictionary_1pct_cardinality/default 1.03 7.0±0.13ms 142.4 MElem/sec 1.00 6.8±0.10ms 146.6 MElem/sec primitive_dictionary_1pct_cardinality/number_distinct_values 1.06 20.9±0.56ms 47.9 MElem/sec 1.00 19.7±0.20ms 50.7 MElem/sec primitive_dictionary_1pct_cardinality/parquet_2 1.05 7.1±0.12ms 141.3 MElem/sec 1.00 6.8±0.07ms 147.9 MElem/sec primitive_dictionary_1pct_cardinality/zstd 1.06 8.8±0.15ms 113.1 MElem/sec 1.00 8.4±0.08ms 119.7 MElem/sec primitive_dictionary_1pct_cardinality/zstd_parquet_2 1.05 8.6±0.14ms 115.9 MElem/sec 1.00 8.2±0.08ms 121.3 MElem/sec string_dictionary/bloom_filter 1.00 123.7±1.87ms 8.1 MElem/sec 1.02 125.6±8.90ms 8.0 MElem/sec string_dictionary/cdc 1.02 64.4±1.42ms 15.5 MElem/sec 1.00 63.4±1.54ms 15.8 MElem/sec string_dictionary/default 1.00 94.1±11.54ms 10.6 MElem/sec 1.00 94.3±10.87ms 10.6 MElem/sec string_dictionary/number_distinct_values 1.00 140.3±3.04ms 7.1 MElem/sec 1.16 163.3±4.16ms 6.1 MElem/sec string_dictionary/parquet_2 1.04 81.0±10.80ms 12.4 MElem/sec 1.00 78.0±4.87ms 12.8 MElem/sec string_dictionary/zstd 1.07 204.3±2.54ms 4.9 MElem/sec 1.00 191.0±4.22ms 5.2 MElem/sec string_dictionary/zstd_parquet_2 1.00 174.5±2.74ms 5.7 MElem/sec 1.01 175.5±5.35ms 5.7 MElem/sec string_dictionary_1pct_cardinality/bloom_filter 1.06 13.0±0.12ms 76.7 MElem/sec 1.00 12.3±0.07ms 81.5 MElem/sec string_dictionary_1pct_cardinality/cdc 1.04 15.8±0.11ms 63.2 MElem/sec 1.00 15.3±0.23ms 65.5 MElem/sec string_dictionary_1pct_cardinality/default 1.06 12.8±0.15ms 78.3 MElem/sec 1.00 12.0±0.07ms 83.4 MElem/sec string_dictionary_1pct_cardinality/number_distinct_values 1.02 29.0±0.16ms 34.5 MElem/sec 1.00 28.4±0.38ms 35.2 MElem/sec string_dictionary_1pct_cardinality/parquet_2 1.05 12.7±0.10ms 78.9 MElem/sec 1.00 12.0±0.17ms 83.2 MElem/sec string_dictionary_1pct_cardinality/zstd 1.05 14.4±0.14ms 69.2 MElem/sec 1.00 13.7±0.14ms 72.9 MElem/sec string_dictionary_1pct_cardinality/zstd_parquet_2 1.06 14.4±0.09ms 69.4 MElem/sec 1.00 13.6±0.26ms 73.8 MElem/sec string_dictionary_low_cardinality_100/bloom_filter 1.09 13.7±0.13ms 72.8 MElem/sec 1.00 12.7±0.14ms 79.0 MElem/sec string_dictionary_low_cardinality_100/cdc 1.05 16.1±0.14ms 62.3 MElem/sec 1.00 15.3±0.23ms 65.5 MElem/sec string_dictionary_low_cardinality_100/default 1.08 13.6±0.14ms 73.8 MElem/sec 1.00 12.6±0.05ms 79.6 MElem/sec string_dictionary_low_cardinality_100/number_distinct_values 1.02 34.4±0.35ms 29.1 MElem/sec 1.00 33.7±0.53ms 29.7 MElem/sec string_dictionary_low_cardinality_100/parquet_2 1.09 13.6±0.26ms 73.6 MElem/sec 1.00 12.4±0.15ms 80.5 MElem/sec string_dictionary_low_cardinality_100/zstd 1.08 13.9±0.14ms 72.0 MElem/sec 1.00 12.9±0.10ms 77.6 MElem/sec string_dictionary_low_cardinality_100/zstd_parquet_2 1.07 13.8±0.13ms 72.4 MElem/sec 1.00 12.9±0.13ms 77.4 MElem/sec string_dictionary_low_cardinality_20/bloom_filter 1.08 13.7±0.14ms 73.1 MElem/sec 1.00 12.6±0.13ms 79.1 MElem/sec string_dictionary_low_cardinality_20/cdc 1.04 16.1±0.16ms 62.1 MElem/sec 1.00 15.5±0.48ms 64.3 MElem/sec string_dictionary_low_cardinality_20/default 1.08 13.3±0.14ms 75.2 MElem/sec 1.00 12.3±0.11ms 81.5 MElem/sec string_dictionary_low_cardinality_20/number_distinct_values 1.04 34.7±0.58ms 28.8 MElem/sec 1.00 33.4±0.56ms 30.0 MElem/sec string_dictionary_low_cardinality_20/parquet_2 1.08 13.5±0.15ms 73.8 MElem/sec 1.00 12.6±0.07ms 79.6 MElem/sec string_dictionary_low_cardinality_20/zstd 1.10 13.8±0.21ms 72.7 MElem/sec 1.00 12.5±0.18ms 80.1 MElem/sec string_dictionary_low_cardinality_20/zstd_parquet_2 1.04 13.6±0.13ms 73.3 MElem/sec 1.00 13.1±0.52ms 76.2 MElem/sec string_dictionary_low_cardinality_400/bloom_filter 1.08 13.7±0.48ms 72.8 MElem/sec 1.00 12.7±0.12ms 78.5 MElem/sec string_dictionary_low_cardinality_400/cdc 1.06 16.0±0.24ms 62.4 MElem/sec 1.00 15.2±0.20ms 65.9 MElem/sec string_dictionary_low_cardinality_400/default 1.08 13.5±0.18ms 73.9 MElem/sec 1.00 12.5±0.16ms 79.7 MElem/sec string_dictionary_low_cardinality_400/number_distinct_values 1.03 34.6±0.39ms 28.9 MElem/sec 1.00 33.7±0.57ms 29.7 MElem/sec string_dictionary_low_cardinality_400/parquet_2 1.07 13.5±0.14ms 74.2 MElem/sec 1.00 12.5±0.11ms 79.8 MElem/sec string_dictionary_low_cardinality_400/zstd 1.07 14.0±0.17ms 71.4 MElem/sec 1.00 13.0±0.06ms 76.7 MElem/sec string_dictionary_low_cardinality_400/zstd_parquet_2 1.07 13.9±0.13ms 71.9 MElem/sec 1.00 13.0±0.11ms 77.2 MElem/sec ``` </details> <details><summary>Per-runner information</summary> <details><summary>arrow_writer — shard 1/1</summary> **Node:** gk3-benchmark-cluster-nap-1xor34y1-8bf6c572-kfgd **Instance:** c4-highmem-16 (12 vCPU / 65 GiB) **uname:** ```text Linux bench-c6124985721-3339-wrwxd 6.12.94+ #1 SMP Fri Aug 21 07:56:42 UTC 2026 x86_64 GNU/Linux ``` **BENCH_COMMAND:** ```sh cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer ``` <details><summary>CPU Details (lscpu)</summary> ```text Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 52 bits physical, 57 bits virtual Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: GenuineIntel Model name: INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz CPU family: 6 Model: 207 Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 1 Stepping: 2 BogoMIPS: 4600.00 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd arat avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid cldemote movdiri movdir64b fsrm md_clear serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 arch_capabilities Hypervisor vendor: KVM Virtualization type: full L1d cache: 384 KiB (8 instances) L1i cache: 256 KiB (8 instances) L2 cache: 16 MiB (8 instances) L3 cache: 260 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` </details> <details><summary>Resource Usage</summary> **base (merge-base)** | Metric | Value | |--------|-------| | Wall time | 1050.2s | | Peak memory | 870.0 MiB | | Avg memory | 138.1 MiB | | CPU user | 1023.9s | | CPU sys | 20.0s | | Peak spill | 0 B | **branch** | Metric | Value | |--------|-------| | Wall time | 1045.2s | | Peak memory | 632.8 MiB | | Avg memory | 112.6 MiB | | CPU user | 1021.7s | | CPU sys | 20.4s | | Peak spill | 0 B | </details> </details> </details> --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner <!-- benchmark-request:6124985721:finish:a625143e056736c6e93cef509d3cbd09 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
