ZhangHuiGui commented on issue #43036:
URL: https://github.com/apache/arrow/issues/43036#issuecomment-2222491633
> Another argument (more practical) in favor of making pre-allocation fully
determined by the types: during kernel dispatching we're looking at the lazily
computed `null_count` which might be `kUnknownNullCount` non-deterministically.
Calling `GetNullCount()` on all the inputs would be wasteful because it's
probably better to let the kernel use the pre-allocated validity bitmap to
compute the intersection together with its own loop.
Thanks for your patience in explaining this. I understand.
> @pitrou might feel like this is a risk worth taking and we should demand
that kernels handle non-determinism correctly.
It seems that there is no consensus on whether we should cancel this
dynamic allocation of validity-buffer risk. @pitrou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
