tustvold commented on issue #6439: URL: https://github.com/apache/arrow-rs/issues/6439#issuecomment-2423683840
Builders are an interesting point, but come with two potential challenges to be aware of: * Builders tend to over-allocate to avoid repeatedly reallocating * Updating memory accounting per element is likely prohibitively expensive This means any "eager" memory computation is likely to work poorly, however, the lazy approach as currently implemented for arrays could actually work quite well. I could see a world where we remove lazy array memory tracking in favour of eager allocation tracking, and add lazy memory tracking for builders -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
