peter-toth edited a comment on pull request #28885: URL: https://github.com/apache/spark/pull/28885#issuecomment-864199872
> > > I measured considerable improvement (~30%) using TPCDSQueryBenchmark on scaleFactor=5 data with the TPCDS Q14a, Q14b, Q23a, Q23b, Q47 and Q57 queries. > > > > > > Did you see any regressions? > > Let me rerun the benchmarks again and come back to you. The previous benchmark run was quite some time ago. As AQE has been enabled by default since that and now it works with most of those affected TPCDS queries, the improvement is only visible if I disable AQE. But, in that case it is still considerable: https://github.com/peter-toth/spark/blob/64c619e798c003e9b27ba534ae370db75b1cfa1c/sql/core/benchmarks/TPCDSQueryBenchmark-results.txt In some cases non-AQE became as good as AQE. UPDATE: I've included AQE and non-AQE results as well to see the improvement and catch regression. I don't see any regression. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
