Kimahriman commented on pull request #34558: URL: https://github.com/apache/spark/pull/34558#issuecomment-1067900687
It's hard to say because when I tested this out on my production jobs (actually still actively using it), I had several other changes too. I'm not sure if there are any benchmarks involving HOFs? Though it's highly dependent on what the lambda function is, and honestly that's one of the main benefits, the lambda functions themselves can be codegen'd instead of eval'd. I also have a larger goal to support subexpression elimination inside lambda functions, because that's where I've found our biggest problem is. https://github.com/apache/spark/pull/34727 is also part of that goal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
