Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19082
@kiszk Thanks for your summary. I had a few related discussions with
@rednaxelafx and @liancheng in the recent weeks. Vertical cuts like
https://github.com/apache/spark/pull/19082 is pretty promising for improving
the performance. Thus, I plan to review the implementation in the recent days.
At the beginning, I did not notice
https://github.com/apache/spark/pull/18931 is using `private` or `final`
attributes; otherwise, inlining does not benefit. It will be the same as the
whole-stage codegen is turned off. The current proposal in
https://github.com/apache/spark/pull/18931 is pretty limited due to the
parameter number limits and the others. Thus, we need a better design for
horizontal/ring cuts. The design is highly dependent on JVM internals and
whole-stage codegen algorithms. So far, we do not have anything significant to
share yet.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
