Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19082
@kiszk @gatorsmile @rednaxelafx Based on the following reasoning, I don't
think the cutting of whole-stage operators you may propose would be much
beneficial than the approach did in #18931:
1. If #18931 splits smaller functions. For the smaller operator functions,
JIT will try to inline it into larger compilation unit. This is based on the
information from @kiszk. Please correct me if it's not correct. So I don't
think we will loss the advantage of wholestage codegen. I'm doubt that it is
any different than not split it, if under your proposal.
2. For the bigger operator functions, it can't be inlined. But based on the
info provided by @kiszk, it won't get performance improvement to inline such
big functions.
Thus, even considering you can provide better cutting points, I don't think
it can get much benefit but incur much complexity into wholestage codegen.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
