singhpratech commented on issue #1297: URL: https://github.com/apache/arrow-go/issues/1297#issuecomment-5579337585
Thank you, that is the right correction, and I have applied it on my side. You are right that a Go-heap buffer cannot be shared with C persistently without pinning, and the binding does not try to: every buffer it hands to the Metal side is pinned with runtime.Pinner for the life of the handle, and its own page-aligned allocator is C memory (posix_memalign), outside the Go heap. What I missed when filing is that memory/mallocator already does what I was asking for: NewMallocatorWithAlignment(16384) gives libc memory at a chosen alignment, no build tag needed. I checked it after your comment: twenty 10M-element int64 arrays built with it are all page aligned, and the binding imports them without a copy, plain and under GOEXPERIMENT=cgocheck2. The binding's docs now point at mallocator as arrow-go's own route, and its test suite covers it. So the only thing left in this report is documentation: NewGoAllocator makes no alignment promise (its buffers land at offset 0 or 8192 within a 16 KiB page), and the memory package does not say which allocator to use for a C handoff or what alignment each guarantees. If a short paragraph in the memory package doc, pointing at mallocator and NewMallocatorWithAlignment for buffers that C will keep, would be welcome, I am glad to open a pull request; otherwise please feel free to close this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
