cloud-fan commented on PR #49202: URL: https://github.com/apache/spark/pull/49202#issuecomment-2553206778
I'm very confused. When pushing predicates through Project, the problem we hit is we need to rewrite the attributes in the predicate with the corresponding expression in the Project. If the attribute appears more than once in the predicate, and the expression from Project is expensive, filter pushdown may make the query slower. The solution is to wrap the predicate with `With` expression, adding the expression from Project as the common expression, and replace attributes in predicate with the common expression reference. But this PR does some rewriting and I don't understand what it does. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
