Omega359 commented on issue #13748: URL: https://github.com/apache/datafusion/issues/13748#issuecomment-3201739835
@alamb @ozankabak I don't think this issue should have been closed. The performance now is definitely better than it was as provided in the description above however it's still quite slow. I've updated the [sql_planner benchmark](https://gist.github.com/Omega359/0ad26d183ba7689b6c21b48182d61864) to showcase that planning is still exponentially slow based on the number of columns and unions in the query. The output I see is below: ``` physical_sorted_union_order_by_10 time: [12.089 ms 12.105 ms 12.125 ms] change: [+6.4475% +6.6567% +6.8895%] (p = 0.00 < 0.05) Performance has regressed. physical_sorted_union_order_by_50 time: [323.92 ms 324.75 ms 325.53 ms] change: [+6.5657% +6.8547% +7.1319%] (p = 0.00 < 0.05) Performance has regressed. Benchmarking physical_sorted_union_order_by_100: Warming up for 30.000 s Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 18.5s. physical_sorted_union_order_by_100 time: [1.7079 s 1.7143 s 1.7221 s] change: [+6.5417% +6.9412% +7.5811%] (p = 0.00 < 0.05) Performance has regressed. Found 1 outliers among 10 measurements (10.00%) 1 (10.00%) high mild Benchmarking physical_sorted_union_order_by_200: Warming up for 30.000 s Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 118.6s. physical_sorted_union_order_by_200 time: [10.271 s 10.321 s 10.400 s] change: [+4.3930% +4.9196% +5.7641%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 10 measurements (30.00%) 2 (20.00%) low mild 1 (10.00%) high severe Benchmarking physical_sorted_union_order_by_500: Warming up for 30.000 s Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 1350.0s. physical_sorted_union_order_by_500 time: [129.75 s 131.00 s 132.82 s] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
