LuciferYang opened a new issue, #13359: URL: https://github.com/apache/gluten/issues/13359
`StdMemoryAllocator::reallocateAligned` handles shrink-to-fit with `std::realloc` on a block that came from `aligned_alloc`. realloc only guarantees fundamental alignment, so when it moves the block the result may not be aligned to the requested alignment. Arrow's own system allocator avoids realloc in `ReallocateAligned` for this reason. On macOS arm64 this happens in practice. With native libraries that log every misaligned shrink, `VeloxTPCHV1Suite` and `VeloxTPCHV1GlutenShuffleManagerSuite` produced 354 shrinks whose result was only 16, 32 or 48 bytes past a 64-byte boundary, mostly 16 KB to 64 KB shuffle buffers shrunk to 64 or 128 bytes. The tests still passed, and I could not find code in cpp/core or cpp/velox that depends on 64-byte alignment, so this is a violation of the Arrow MemoryPool contract rather than a known wrong result. On Linux with glibc the shrink happens in place, and with jemalloc moved blocks stay 64-byte aligned. This issue was written with the assistance of AI (Claude Opus). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
