tustvold commented on issue #4724: URL: https://github.com/apache/arrow-rs/issues/4724#issuecomment-1790458075
[get_array_memory_size](https://docs.rs/arrow-array/latest/arrow_array/array/trait.Array.html#tymethod.get_array_memory_size) returns the total capacity of all the buffers backing the array, e.g. https://github.com/apache/arrow-rs/blob/master/arrow-array/src/array/boolean_array.rs#L305C44-L305C66 If those buffers are slices of the same underlying allocation, as they will be in the above example, you will count that allocation multiple times. FWIW it is relatively trivial to rewrite an array if you so wish ``` let data = array.to_data(); let mutable = MutableArrayData::new(vec![&data], false, 0); mutable.extend(0, 0, array.len()); let compacted = make_array(mutable.freeze()); ``` But at least in the example provided by @v0y4g3r the issue might simply be one of accounting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
