peter-toth commented on PR #41119: URL: https://github.com/apache/spark/pull/41119#issuecomment-1570112778
Sorry, I still haven't got time to review the PR thoroughly. (Maybe next week...) IMO it is a good idea to lazily evaluate expressions and cache the results runtime in case of any expresions that might be used more than once. In some cases it could achieve much better performance than any kind of static analysis we do currently... But the downside is that the generated code is bigger and so it can cause regressions too. So, probably I would add a new feature flag to allow switching between the current "static" and the suggested new "runtime" subexpression elimination of this PR. But others might have better ideas... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
