peter-toth commented on PR #52149: URL: https://github.com/apache/spark/pull/52149#issuecomment-3238015158
Technically, I can create a new rule and limit the scope to batch eval expressions like Python UDFs, but I feel the new rule will share most of the logic with `CollapseProject`. Like checking if the producer is referenced at most once or if not then determining if it is cheap to inline the producer, as I don't want to move an expression that contains a Python UDF to the top project node in other cases. I agree that the name `CollapseProject` doesn't fully describe the current functionality after this PR, but do we surely want create a new rule for a very similar logic? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
