alamb commented on PR #10020: URL: https://github.com/apache/arrow-rs/pull/10020#issuecomment-4673433834
> @alamb @adriangb These gains are amazing. Do you think we could reuse the infrastructure we recently added in #9846 to write benchmarks around memory usage here as well? > > That way, we can keep track of these degenerate use cases and make sure they do not regress. Sounds like a good idea to me However, note that arrow-rs doesn't actually have any different memory characteristics in the next release -- rather it now has the API hooks that can potentially reduce memory substntially (at the cost of spilling to some other storage) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
