milenkovicm commented on PR #2315: URL: https://github.com/apache/datafusion-ballista/pull/2315#issuecomment-5307467034
I've run this PR with AQE enabled (local MBP) ```bash cargo run --bin tpch -- benchmark ballista -p /Users/marko/TMP/tpch_data/tpch-data-sf10/ -f parquet -i 1 --port 50050 --host 127.0.0.1 -c datafusion.execution.target_partitions=14 ``` ``` Query 1 took 0.390 s and returned 4 rows Query 2 took 0.172 s and returned 100 rows Query 3 took 0.427 s and returned 10 rows Query 4 took 0.228 s and returned 5 rows Query 5 took 1.020 s and returned 5 rows Query 6 took 0.110 s and returned 1 rows Query 7 took 0.828 s and returned 4 rows Query 8 took 0.365 s and returned 2 rows Query 9 took 0.651 s and returned 168 rows Query 10 took 0.477 s and returned 20 rows Query 11 took 0.096 s and returned 0 rows Query 12 took 0.344 s and returned 2 rows Query 13 took 0.334 s and returned 46 rows Query 14 took 0.173 s and returned 1 rows Query 15 took 0.213 s and returned 1 rows Query 16 took 0.167 s and returned 27840 rows Query 17 took 0.779 s and returned 1 rows Query 18 took 1.294 s and returned 100 rows Query 19 took 0.320 s and returned 1 rows Query 20 took 0.336 s and returned 1804 rows Query 21 took 1.423 s and returned 100 rows Query 22 took 0.158 s and returned 7 rows Total time: 10.305 s ``` And with AQE disabled ```bash cargo run --bin tpch -- benchmark ballista -p /Users/marko/TMP/tpch_data/tpch-data-sf10/ -f parquet -i 1 --port 50050 --host 127.0.0.1 -c datafusion.execution.target_partitions=14 -c ballista.planner.adaptive.enabled=false ``` ``` Query 1 took 0.434 s and returned 4 rows Query 2 took 0.544 s and returned 100 rows Query 3 took 0.623 s and returned 10 rows Query 4 took 0.314 s and returned 5 rows Query 5 took 1.377 s and returned 5 rows Query 6 took 0.125 s and returned 1 rows Query 7 took 1.655 s and returned 4 rows Query 8 took 2.728 s and returned 2 rows Query 9 took 2.999 s and returned 168 rows Query 10 took 0.692 s and returned 20 rows Query 11 took 0.397 s and returned 0 rows Query 12 took 0.389 s and returned 2 rows Query 13 took 0.367 s and returned 46 rows Query 14 took 0.212 s and returned 1 rows Query 15 took 0.250 s and returned 1 rows Query 16 took 0.216 s and returned 27840 rows Query 17 took 1.763 s and returned 1 rows Query 18 took 1.563 s and returned 100 rows Query 19 took 0.392 s and returned 1 rows Query 20 took 0.400 s and returned 1804 rows Query 21 took 2.365 s and returned 100 rows Query 22 took 0.187 s and returned 7 rows Total time: 19.992 s ``` I remember when total time was like ~45 seconds, huge improvement in last few versions, very happy to see this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
