datbth commented on issue #43791:
URL: https://github.com/apache/arrow/issues/43791#issuecomment-2306651784
Many thanks for the quick answer!
I'm trying to form an understanding (thanks to your explanation).
1. Arrow has its own "reference counter" and is controlled by `Retain` and
`Release`.
1. After `Release()` decrements Arrow's reference counter to 0:
1. It marks slices and buffers as nil.
1. This also effectively reduces the Go's reference counter of
those slices and buffers.
3. It calls `mem.Free()`
1. The default `GoAllocator.Free()` does nothing, but other
allocators can perform their memory cleanup here.
2. When Go GC runs
1. Because of 1.i.a.a, it can clean up the "released" slices and
buffers.
2. Even when `Release` was not called, taking your `reader` example, if
the reader and the record no longer have any Go reference, Go GC can still
clean up the memory.
3. Because of 1.i.a, and the pattern that `Next()` releases current record
before fetching the next one, it is vital to call `Retain()` if we are using
the previous record after calling new `Next()`
4. So for the case of `GoAllocator`
* Missing `Retain` or extraneous `Release` -> can lead to nil pointers
(as mentioned in 3).
* Extraneous `Retain` or missing `Release` -> Go GC can still clean up,
albeit later (as mentioned in 2.ii).
Regarding the pattern or documentation, I found these:
* `RecordBuilder.NewRecord()` [is clearly
documented](https://github.com/apache/arrow/blob/main/go/arrow/array/record.go#L329),
which is nice.
* But ipc `Reader.Record()` [doesn't mention
that](https://github.com/apache/arrow/blob/main/go/arrow/ipc/reader.go#L262),
which leads to my confusion.
* Also, the documentation is not available at the interface level, but I
have to look for the implementations.
* But now that I know the mechanism of `Retain`/`Release` and the pattern of
`Next`, I will probably try to look into the implementation of those specific
`Next` and `Record`.
Thanks again!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
