zeroshade commented on issue #588: URL: https://github.com/apache/arrow-adbc/issues/588#issuecomment-2856389747
Go's garbage collector is a concurrent mark-and-sweep. Rather than maintaining refcounts, the GC goroutine will periodically look at the current heap size (with various settings configurable via env vars) and then trace the live heap objects to mark things that are reachable, and then do a sweep to clean up anything that is unreachable. You can see it all explained [here](https://tip.golang.org/doc/gc-guide). Rather than being "signaled" there's just a goroutine that concurrently runs (and is scheduled by the runtime) doing GC work when there is work to do (or isn't scheduled/is off when there's no GC-related work to do). The cost of this is based on Go-owned heap memory. While the Go drivers, when building the shared-library objects, utilize an allocator that utilizes C-owned memory to pass data through the C interface (i.e. the go runtime doesn't know anything about the Arrow data or anything we construct with the Allocator, and therefore the Go GC doesn't scan it). This is to ensure it's safe to pass the memory through the FFI boundaries without fighting the Go garbage collector, or building up a large memory base that it has to manage (as the lifetime of the memory will be managed by the consumer calling the `release` callback. While there's definitely some questions surrounding multiple Go-based drivers in the same process that need to be addressed (the same issue would arise with any drivers that require protobuf/gRPC, or Rust async runtimes), I don't see any larger issue beyond this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
