maltzsama commented on issue #106: URL: https://github.com/apache/arrow-swift/issues/106#issuecomment-4020126516
Hey @willtemperley, great catch! You are completely right. There is no basis in the Apache Arrow memory specification for this arbitrary +8 byte overhead. It seems like a legacy oversight that slipped through and was mistakenly codified into the unit tests. In datasets with a large number of columns or frequent small buffer allocations, this undocumented padding just leads to unnecessary memory bloat. I have just opened a PR to remove this padding and strictly enforce the 64-byte alignment. I also updated the testPrimitiveArray unit test you pointed out so it now correctly asserts a capacity of 128 instead of 136. PR: https://github.com/apache/arrow-swift/pull/144 Thanks for documenting this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
