peter-toth edited a comment on pull request #32298: URL: https://github.com/apache/spark/pull/32298#issuecomment-1075538393
>Is it still way better than running the scalar subqueries over the same table multiple times? If the scalar subqueries to be merged are quick then I'm not sure. > I'm more worried about the complexities (i.e., pattern matching cognitive overhead) with new plan nodes like CommonSubqueries and CommonSubqueriesExec. Many rules have been implemented as pattern matching, e.g., a rule that matches a Project is supposed to also match CommonSubqueries (in theory)? Hmm, `CommonScalarSubqueries` (and `CommonScalarSubqueriesExec`) can appear only as a root node, it is't exactly the same as `Project`. It never replaces Project. I think it is more similar to `WithCTE` in that regard. Can you give an example where we need to handle these new nodes as `Project`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
