Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19082
>In the above case, 1. is slower than 2. due to the advantage of JIT
compilation.
>#18931 can cut the boundary of whole-stage operators. Hopefully, this
cutting makes the byte code size smaller than current aggressive embedding...
That is correct to me.
> #18931 is not what we want, although it can partially resolves some
issues. Simply disabling the whole-stage codegen after the cut might still
trigger the regression like Q66.
The regression of Q66 is explained in previous comment:
https://github.com/apache/spark/pull/19082#issuecomment-335340888. A hot method
which can't be compiled can cause regression. I think the split operator
functions are hot methods for sure, if they can't be compiled, disabling
wholestage codegen should not bring regreesion IMO.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
