nathanb9 commented on issue #17737: URL: https://github.com/apache/datafusion/issues/17737#issuecomment-4585282746
@neilconway Yep TPC-DS has lots of CTEs being inlined and seeing huge speed ups 1.5x-2.7x with materialization POC > could either be made configurable via SQL, or potentially done by the query optimizer. For DuckDB they use the SQL planner to mark all CTEs with multiple references to be materialized. Then they have optimizer rule which makes decision whether its good to keep it as materialized or inline it. I've outlined the work in the description https://github.com/apache/datafusion/pull/22551. Would appreciate thoughts on approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
