save-buffer commented on issue #13981: URL: https://github.com/apache/arrow/issues/13981#issuecomment-1229370796
Hi @js8544, the current kernels subsystem in Arrow currently struggles from having too many random small memory allocations, so this isn't very surprising at all. Profiling your benchmark seems to provide more support for this - there's a lot more allocation/freeing going on besides just allocating the output (such as allocating std::vectors, incrementing/decrementing the refcount on the Scalar, etc.). I wrote a document about fixing this system a while back, and I hope to get to implementing it soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
