alamb commented on PR #6034: URL: https://github.com/apache/arrow-datafusion/pull/6034#issuecomment-1522341230
``` + echo '****** TPCH SF1 (Parquet) ******' ****** TPCH SF1 (Parquet) ****** + python3 /home/alamb/arrow-datafusion/benchmarks/compare.py /home/alamb/benchmarking/feature%2Fstream_groupby4/tpch_sf1_parquet_main.json /home/alamb/benchmarking/feature%2Fstream_groupby4/tpch_sf1_parqu\ et_branch.json ┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓ ┃ Query ┃ /home/alamb… ┃ /home/alamb… ┃ Change ┃ ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩ │ QQuery 1 │ 1440.81ms │ 1466.22ms │ no change │ │ QQuery 2 │ 382.39ms │ 404.11ms │ 1.06x slower │ │ QQuery 3 │ 543.05ms │ 552.98ms │ no change │ │ QQuery 4 │ 231.01ms │ 220.95ms │ no change │ │ QQuery 5 │ 693.54ms │ 687.44ms │ no change │ │ QQuery 6 │ 426.04ms │ 449.61ms │ 1.06x slower │ │ QQuery 7 │ 1185.19ms │ 1188.84ms │ no change │ │ QQuery 8 │ 702.14ms │ 709.04ms │ no change │ │ QQuery 9 │ 1300.47ms │ 1316.84ms │ no change │ │ QQuery 10 │ 773.33ms │ 785.76ms │ no change │ │ QQuery 11 │ 332.28ms │ 353.08ms │ 1.06x slower │ │ QQuery 12 │ 330.60ms │ 321.46ms │ no change │ │ QQuery 13 │ 1088.95ms │ 1150.66ms │ 1.06x slower │ │ QQuery 14 │ 426.84ms │ 438.99ms │ no change │ │ QQuery 15 │ 390.98ms │ 400.00ms │ no change │ │ QQuery 16 │ 328.41ms │ 351.25ms │ 1.07x slower │ │ QQuery 17 │ 2761.33ms │ 2798.95ms │ no change │ │ QQuery 18 │ 3650.22ms │ 3674.77ms │ no change │ │ QQuery 19 │ 724.28ms │ 776.02ms │ 1.07x slower │ │ QQuery 20 │ 1214.26ms │ 1322.29ms │ 1.09x slower │ │ QQuery 21 │ 1651.66ms │ 1685.36ms │ no change │ │ QQuery 22 │ 196.26ms │ 196.06ms │ no change │ └──────────────┴──────────────┴──────────────┴──────────────┘ + echo '****** TPCH SF1 (mem) ******' ****** TPCH SF1 (mem) ****** + python3 /home/alamb/arrow-datafusion/benchmarks/compare.py /home/alamb/benchmarking/feature%2Fstream_groupby4/tpch_sf1_mem_main.json /home/alamb/benchmarking/feature%2Fstream_groupby4/tpch_sf1_mem_branc\ h.json ┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Query ┃ -o ┃ -o ┃ Change ┃ ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ QQuery 1 │ 758.37ms │ 778.09ms │ no change │ │ QQuery 2 │ 277.40ms │ 309.41ms │ 1.12x slower │ │ QQuery 3 │ 173.75ms │ 188.35ms │ 1.08x slower │ │ QQuery 4 │ 113.69ms │ 110.27ms │ no change │ │ QQuery 5 │ 475.95ms │ 461.74ms │ no change │ │ QQuery 6 │ 36.47ms │ 37.38ms │ no change │ │ QQuery 7 │ 1083.01ms │ 1063.32ms │ no change │ │ QQuery 8 │ 259.31ms │ 246.75ms │ no change │ │ QQuery 9 │ 624.32ms │ 575.27ms │ +1.09x faster │ │ QQuery 10 │ 310.52ms │ 351.58ms │ 1.13x slower │ │ QQuery 11 │ 284.14ms │ 282.22ms │ no change │ │ QQuery 12 │ 148.13ms │ 145.27ms │ no change │ │ QQuery 13 │ 659.01ms │ 719.22ms │ 1.09x slower │ │ QQuery 14 │ 52.88ms │ 48.78ms │ +1.08x faster │ │ QQuery 15 │ 90.71ms │ 103.68ms │ 1.14x slower │ │ QQuery 16 │ 233.70ms │ 258.77ms │ 1.11x slower │ │ QQuery 17 │ 2403.54ms │ 2550.52ms │ 1.06x slower │ │ QQuery 18 │ 2964.21ms │ 3217.22ms │ 1.09x slower │ │ QQuery 19 │ 139.10ms │ 149.13ms │ 1.07x slower │ │ QQuery 20 │ 930.97ms │ 1042.61ms │ 1.12x slower │ │ QQuery 21 │ 1399.42ms │ 1408.19ms │ no change │ │ QQuery 22 │ 140.09ms │ 138.46ms │ no change │ └──────────────┴──────────────┴──────────────┴───────────────┘ ``` I am running with https://github.com/alamb/datafusion-benchmarking/blob/87ee101b70b15dd4529f124d65189b0fb87e09b7/bench.sh Running on a gcp machine `e2-standard-8`: ```shell cat /proc/cpuinfo ... processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 79 model name : Intel(R) Xeon(R) CPU @ 2.20GHz stepping : 0 microcode : 0xffffffff cpu MHz : 2200.164 cache size : 56320 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpui\ d tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adj\ ust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt arat md_clear arch_capabilities bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa mmio_stale_data retbleed bogomips : 4400.32 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management: ``` I am hoping to make the benchmarks easier to run / reproduce. I also plan to take another close look at this PR tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
