alamb commented on issue #7186: URL: https://github.com/apache/arrow-rs/issues/7186#issuecomment-4783620495
> also, if minify is for reduce memory, in case the buffer is shared, calling minify will just increase memory as it will require data copy (which is still valuable) I think this is the case for almost all kernels (they make a new array from an existing one, temporarily increasing memory usage) > also in minify some people might want to cleanup all non empty nulls and reduce the unused space but this specific option as oppose to only clean top level is different that only match for some data types. I was thinking minify would always create a new Array (with new allocations, etc) that consumes the minimum amount of new space (so for lists and other structured types it would avoid copying unused data for child arrays, as proposed in https://github.com/apache/arrow-rs/pull/9970). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
