zhztheplayer opened a new pull request, #5887: URL: https://github.com/apache/incubator-gluten/pull/5887
The patch introduce [ClickBench](https://github.com/ClickHouse/ClickBench) benchmark into gluten-it. The benchmark can be triggered locally and is not yet added to CI. To trigger the benchmark, using the following command: ``` <gluten-it> queries-compare --local --preset=velox --benchmark-type=clickbench --error-on-memleak --off-heap-size=400g -s1.0 --threads=112 --shuffle-partitions=112 --iterations=1 --skip-data-gen --enable-history --enable-ui ``` So far 12 out of the total 44 queries are failing. An example report: ``` Test report: Summary: 31 out of 43 queries passed. | Query ID| Was Passed| Expected Row Count| Actual Row Count| Baseline Planning Time (Millis)| Planning Time (Millis)| Baseline Query Time (Millis)| Query Time (Millis)| Query Time Variation| | q1| true| 1| 1| 199| 26| 4747| 3892| 21.97%| | q2| true| 1| 1| 93| 34| 3532| 3820| -7.54%| | q3| true| 1| 1| 34| 15| 1487| 2089| -28.82%| | q4| true| 1| 1| 11| 14| 948| 1458| -34.98%| | q5| true| 1| 1| 25| 16| 3393| 4344| -21.89%| | q6| true| 1| 1| 13| 15| 3743| 5857| -36.09%| | q7| true| 1| 1| 69| 13| 1235| 744| 65.99%| | q8| true| 18| 18| 90| 22| 2333| 1833| 27.28%| | q9| true| 10| 10| 27| 16| 4025| 6011| -33.04%| | q10| true| 10| 10| 24| 17| 5077| 6826| -25.62%| | q11| true| 10| 10| 23| 22| 2132| 5024| -57.56%| | q12| true| 10| 10| 19| 23| 2233| 3089| -27.71%| | q13| true| 10| 10| 18| 22| 3275| 5229| -37.37%| | q14| true| 10| 10| 24| 27| 5805| 7579| -23.41%| | q15| true| 10| 10| 17| 19| 2824| 5292| -46.64%| | q16| true| 10| 10| 11| 21| 4645| 3118| 48.97%| | q17| true| 10| 10| 13| 20| 4053| 4931| -17.81%| | q18| true| 10| 10| 10| 13| 3147| 2376| 32.45%| | q20| true| 4| 4| 33| 17| 690| 1717| -59.81%| | q21| true| 1| 1| 30| 19| 2645| 1814| 45.81%| | q22| true| 10| 10| 48| 39| 2948| 910| 223.96%| | q23| true| 10| 10| 26| 27| 5227| 1928| 171.11%| | q26| true| 10| 10| 12| 28| 1433| 342| 319.01%| | q28| true| 25| 25| 43| 33| 3165| 1517| 108.64%| | q30| true| 1| 1| 115| 63| 1731| 1710| 1.23%| | q31| true| 10| 10| 20| 18| 2653| 2472| 7.32%| | q34| true| 10| 10| 10| 15| 7043| 6712| 4.93%| | q35| true| 10| 10| 14| 16| 6531| 6857| -4.75%| | q36| true| 10| 10| 19| 18| 2992| 3078| -2.79%| | q37| true| 10| 10| 29| 26| 1151| 647| 77.90%| | q38| true| 10| 10| 19| 20| 810| 438| 84.93%| Failed queries (a failed query with correct row count indicates value mismatches): | Query ID| Was Passed| Expected Row Count| Actual Row Count| Baseline Planning Time (Millis)| Planning Time (Millis)| Baseline Query Time (Millis)| Query Time (Millis)| Query Time Variation| | q19| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| | q24| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| | q25| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| | q27| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| | q29| false| 11| 11| 41| 55| 10925| 8379| 30.39%| | q32| false| 10| 10| 16| 17| 2895| 4213| -31.28%| | q33| false| 10| 10| 10| 15| 8111| 5057| 60.39%| | q39| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| | q40| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| | q41| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| | q42| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| | q43| false| N/A| N/A| N/A| N/A| N/A| N/A| N/A| Overall: | Query ID| Was Passed| Expected Row Count| Actual Row Count| Baseline Planning Time (Millis)| Planning Time (Millis)| Baseline Query Time (Millis)| Query Time (Millis)| Query Time Variation| | succeeded| true| 246| 246| 1138| 694| 97653| 103654| -5.79%| | all| true| N/A| N/A| N/A| N/A| N/A| N/A| N/A| ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
