Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19324
@juliuszsompolski Thanks for pinging me.
#18931 is an attempt to separate the consume function as it can as
possible. With long chain of any operators, you can have a long consume
function and fail JIT, this is the one reason it tries to split into functions
at the root of codegen support, instead of in some operators. I'd avoid to
duplicate the separate logic in all operators, IMO.
For the explicit delaying evaluation of projection, currently the strategy
I take is not going to split it. I guess that you mean the evaluation that can
be delayed by the compiler, I personally think it should not be a big impact
under the whole-stage codegen framework. The reason is those evaluation are
actually needed and can't be avoided in most of (if not all) cases. From the
benchmark we can see there is no negative impact even in the cases where no
long consume function exists.
Yeah, I think the simplifies for the use of `continue` is a good thing, if
it is possible.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
