peter-toth edited a comment on issue #25644: [SPARK-28940][SQL] Subquery reuse across all subquery levels URL: https://github.com/apache/spark/pull/25644#issuecomment-538660898 After https://github.com/apache/spark/pull/26015 some of the new UTs showed and error and I realized that the new feature of that PR (exchange reuse across all subquery levels) conflicts with my implementation of subquery reuse across all subquery levels. This was because the `subqueryCache` that I added to the `QueryExecution` contained a cached subquery that was later removed by the new version of `ReuseExchange` rule. The subquery that was cached had a parent `Exchange` node that was replaced to a `ReusedExchangeExec` node. In the last commit I changed my subquery reuse implementation to something similar to exchange reuse implementation. I believe this new approach is a bit less efficient but the advantage is that there is no more need for `subqueryCache` in `QueryExecution` and overall the required change is smaller than before. cc @mgaido91 @maryannxue @gatorsmile
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
