I've been working to track this down for 2 days now and I'm just taking a
long shot to see if anyone might have a new idea for me.
My cgo-based bindings library seems to have unbounded RSS memory growth,
which I have been able to reduce to the smallest benchmark test and even
pinpoint the exact call, but the reason behind it still eludes me.
The context is that I have a struct in C++ that will store a const char*
for the last exception that was thrown, as a strdup() copy which gets
cleaned up properly each time.
typedef struct _HandleContext {
HandleId handle;
const char* last_error;
} _HandleContext;
const char* getLastError(_HandleContext* ctx);
On the Go side, I have a function for lastError() to return the last error
value
func (c *Config) lastError() error {
err := C.getLastError(c.ptr)
if err == nil {
return nil
}
e := C.GoString(err)
if e == "" {
return nil
}
runtime.KeepAlive(c)
// return nil // <- would result in no memory growth
return errors.New(e) // <- results in memory growth
}
What I am seeing in my benchmark test is that the RSS grows something like
20MB a second, yet the GODEBUG=gctrace=1 and the pprof memory profile don't
reflect this memory usage at all, aside from showing a hotspot (in pprof)
being the GoString() call:
gc 4 @4.039s 0%: 0.006+0.14+0.003 ms clock,
0.024+0.10/0.039/0.070+0.014 ms cpu, 4->4->0 MB, 5 MB goal, 4 P
gc 5 @6.857s 0%: 0.003+0.20+0.004 ms clock,
0.015+0.069/0.025/0.15+0.016 ms cpu, 4->4->0 MB, 5 MB goal, 4 P
...
gc 26 @69.498s 0%: 0.005+0.12+0.003 ms clock,
0.021+0.10/0.044/0.093+0.014 ms cpu, 4->4->0 MB, 5 MB goal, 4 P
// 800MB RSS usage
gc 27 @71.824s 0%: 0.006+2.2+0.003 ms clock,
0.025+0.063/0.058/0.11+0.014 ms cpu, 4->4->0 MB, 5 MB goal, 4 P
(pprof) top10
Showing nodes accounting for 46083.69kB, 100% of 46083.69kB total
Showing top 10 nodes out of 19
flat flat% sum% cum cum%
30722.34kB 66.67% 66.67% 30722.34kB 66.67% <...>._Cfunc_GoStringN (inline)
7168.11kB 15.55% 82.22% 7168.11kB 15.55% errors.New (inline)
3073.16kB 6.67% 88.89% 46083.69kB 100% <...>.testLeak
1536.02kB 3.33% 92.22% 1536.02kB 3.33%
<...>.(*DisplayTransform).SetInputColorSpace.func1
1024.02kB 2.22% 94.44% 1024.02kB 2.22% <...>.(*Config).NumViews.func1
1024.02kB 2.22% 96.67% 1024.02kB 2.22% <...>.(*Config).View.func1
512.01kB 1.11% 97.78% 512.01kB 1.11%
<...>.(*DisplayTransform).SetView.func1
512.01kB 1.11% 98.89% 512.01kB 1.11% <...>._Cfunc_GoString (inline)
512.01kB 1.11% 100% 512.01kB 1.11% <...>.newProcessor (inline)
0 0% 100% 512.01kB 1.11%
<...>.(*Config).ColorSpaceNameByIndex
Regardless of whether I ignore the error return value in my test, it grows.
If I return nil instead of errors.New(e), it will stay around 20MB RSS.
I MUST be doing something stupid, but I can't see any reason for the memory
growth based on returning this string wrapped in an error. At first I
thought I was leaking in C/C++ but it a led to this one factor on the Go
side. Any tips would help greatly, since I have tried debug GC output,
pprof reports, valgrind, address sanitizer, and refactoring the entire
memory management of my C bindings layer.
Justin
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CAPGFgA20ai9u9Gy8Zk_XTq5ZK9bz_oo0ZrZs5ruvEwfwK3ukqA%40mail.gmail.com.
