tustvold commented on PR #6155: URL: https://github.com/apache/arrow-rs/pull/6155#issuecomment-2278320966
I think this is an interesting angle but I do wonder if there is something fishy in what DataFusion is doing here. In particular, whilst lazily computing null masks will save cycles, array slicing is still a non-trivial operation. I wonder if the issue might be that the algorithm in DF is relying on slicing on a hot path where it would be better of tracking offsets and then using one of the selection kernels -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
