eeroel commented on issue #39906: URL: https://github.com/apache/arrow/issues/39906#issuecomment-1925398859
I did some more digging, and as far as I understand most of the time spent is not from the comparisons per se, but the overhead from these CallFunction invocations, e.g. https://github.com/apache/arrow/blob/22f2cfd1e1ebe49016b6d97c49f494287a98d02f/cpp/src/arrow/compute/expression_internal.h#L101 I don't really understand the Arrow internals very well, but I wonder if there would be a way to reduce this overhead? There seems to be a batch version of CallFunction so would the "right" approach be to batch the comparisons up somehow? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
