This is an automated email from the ASF dual-hosted git repository.
zeroshade pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-go.git
The following commit(s) were added to refs/heads/main by this push:
new 281776d9 docs(arrow/memory): document allocator alignment and which
allocator to use when C retains buffers (#1302)
281776d9 is described below
commit 281776d9292c5229c48c336bca3af4d339369cce
Author: singhpratech <[email protected]>
AuthorDate: Tue Sep 8 15:18:36 2026 -0400
docs(arrow/memory): document allocator alignment and which allocator to use
when C retains buffers (#1302)
### Rationale for this change
Nothing in the `memory` package said what alignment an allocator
guarantees, or which allocator to
use for buffers that C will keep after the cgo call. The `cdata` export
comment named only the
`CgoArrowAllocator`, which needs the Arrow C++ library, while
`mallocator` covers the same need
without it. Discussed in #1297.
### What changes are included in this PR?
Two sections in the `memory` package comment: the 64-byte alignment
`GoAllocator` and
`mallocator.NewMallocator` provide and `NewMallocatorWithAlignment` for
anything larger; and the
cgo pointer rule with the two allocators that return C memory. Doc
comments on `GoAllocator` and
`NewGoAllocator`. The `ExportArrowRecordBatch` comment now names
`mallocator.Mallocator` alongside
`CgoArrowAllocator` and points at the package doc.
### Are these changes tested?
Documentation only; `go vet` and `go build ./...` pass and `go doc
./arrow/memory` renders the new sections.
### Are there any user-facing changes?
Documentation only.
Closes #1297.
---
arrow/cdata/interface.go | 9 ++++++---
arrow/memory/doc.go | 22 ++++++++++++++++++++++
arrow/memory/go_allocator.go | 5 +++++
3 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/arrow/cdata/interface.go b/arrow/cdata/interface.go
index 03a9e7b0..6641f267 100644
--- a/arrow/cdata/interface.go
+++ b/arrow/cdata/interface.go
@@ -228,9 +228,12 @@ func ExportArrowSchema(schema *arrow.Schema, out
*CArrowSchema) {
// stay valid. This is only true until the CGO call returns, at which point
the garbage collector
// is free to move things around again. As a result, if the function you're
calling is going to
// hold onto the pointers or otherwise continue to reference the memory
*after* the call returns,
-// you should use the CgoArrowAllocator rather than the GoAllocator (or
DefaultAllocator) so that
-// the memory which is allocated for the record batch in the first place is
allocated in C,
-// not by the Go runtime and is therefore not subject to the Garbage
collection.
+// you should build the record with an allocator that returns C memory rather
than the GoAllocator
+// (or DefaultAllocator), so that the memory which is allocated for the record
batch in the first
+// place is allocated in C, not by the Go runtime, and is therefore not
subject to the Garbage
+// collection: mallocator.Mallocator (libc malloc, no C++ dependency) or, when
the Arrow C++
+// library is linked, the CgoArrowAllocator behind the 'ccalloc' build tag.
See the memory
+// package documentation for the alignment each allocator guarantees.
//
// The release function on the populated CArrowArray will properly decrease
the reference counts,
// and release the memory if the record has already been released. But since
this must be explicitly
diff --git a/arrow/memory/doc.go b/arrow/memory/doc.go
index 20a28e4e..e8d11e4f 100644
--- a/arrow/memory/doc.go
+++ b/arrow/memory/doc.go
@@ -18,5 +18,27 @@
Package memory provides support for allocating and manipulating memory at a
low level.
The build tag 'mallocator' will switch the default allocator to one backed by
libc malloc. This also requires CGO.
+
+# Alignment
+
+[GoAllocator] (the [DefaultAllocator] unless the 'mallocator' build tag is
set) returns
+buffers whose start is rounded up to a multiple of 64 bytes, and the
mallocator package's
+[github.com/apache/arrow-go/v18/arrow/memory/mallocator.NewMallocator] does
the same for
+libc memory. Neither promises more than 64. A consumer that needs a larger
alignment (a
+page-aligned buffer for a GPU runtime, mmap or DMA, for example) should
allocate with
+[github.com/apache/arrow-go/v18/arrow/memory/mallocator.NewMallocatorWithAlignment],
+which takes any power of two.
+
+# Sharing buffers with C
+
+Memory from [GoAllocator] lives on the Go heap. Under the cgo pointer rules
+(https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers) C code may use such a
pointer only
+for the duration of the cgo call it was passed to; it must not keep it,
because the
+garbage collector may move or free the memory. Buffers that C will hold on to,
such as
+arrays exported over the C Data Interface with the cdata package and retained
by the
+consumer, should therefore be allocated with an allocator that returns C
memory:
+[github.com/apache/arrow-go/v18/arrow/memory/mallocator.Mallocator] (libc
malloc, no
+C++ dependency, requires cgo) or, when the Arrow C++ library is linked, the
+CgoArrowAllocator behind the 'ccalloc' build tag.
*/
package memory
diff --git a/arrow/memory/go_allocator.go b/arrow/memory/go_allocator.go
index 6ccf5f75..b121f087 100644
--- a/arrow/memory/go_allocator.go
+++ b/arrow/memory/go_allocator.go
@@ -16,8 +16,13 @@
package memory
+// GoAllocator is an Allocator backed by the Go heap. Buffers it returns start
+// at a 64-byte boundary; Free is a no-op because the garbage collector
reclaims
+// the memory. Its buffers must not be retained by C code beyond a single cgo
+// call; see the package documentation for allocators that return C memory.
type GoAllocator struct{}
+// NewGoAllocator returns a GoAllocator.
func NewGoAllocator() *GoAllocator { return &GoAllocator{} }
func (a *GoAllocator) Allocate(size int) []byte {
