I'd put all that C-calling code in ONE goroution, with runtime.LockOSThread, and send commands to this thread through a channel, with a receiver chan for response, too. This way only one goroutine needs LockOSThread, and only that pays the memory penalty. If it becomes the bottleneck, you can easily ramp up several such "C-facing-worker-goroutines".
kongfam...@gmail.com a következőt írta (2020. október 25., vasárnap, 15:39:31 UTC+1): > Thanks Ian > > I found out that some libraries use thread-local storage > I understood that it should be called runtime.LockOSThread() in my go > routine because each C function in for-loop can run on different thread > > I have one more question. > If I called runtime.LockOSThread(), should I also call > runtime.UnlockOSThread()? > When I called runtime.UnlockOSThread() in the end of go routine, heap > usage increased. > I think thread-local storage's usage is accumulating because OS thread > isn't terminated > Bacause of the above problem, I don't call runtime.UnlockOSThread() in the > end of go routine. > > 2020년 10월 25일 일요일 오전 2시 53분 42초 UTC+9에 Ian Lance Taylor님이 작성: > >> On Sat, Oct 24, 2020 at 10:30 AM ByeongJoon Gong >> <kongfam...@gmail.com> wrote: >> > >> > I'm developing speech-to-text inference server with cpp libraries >> (wav2letter) >> > >> > according to runtime docs ( >> https://golang.org/src/runtime/proc.go?s=108905:108924#L3803), >> > >> > "A goroutine should call LockOSThread before calling OS services or >> non-Go library functions that depend on per-thread state" >> > >> > The current server process as fallow >> > >> > func requestHandler(stream grpc....) { >> > # runtime.LockOSThread() >> > var decoder unsafe.Pointer >> > init := false >> > >> > defer func() { >> > if decoder != nil { >> > C.resetDecoder(decoder) >> > } >> > } >> > for { >> > in, err := stream.Recv(); // grpc streaming >> > if err != nil { >> > break; >> > } >> > >> > if init == true { >> > C.decode(decoder, someTransformFunc(in.audio)) >> > } else { >> > decoder = C.getDecoder() >> > init = true >> > } >> > } >> > } >> > >> > Even if I don't call runtime.LockOSThread(), it works. >> > but memory leak occured when repeated. >> > ( VmData keep increasing on /proc/{pid}/status ) >> > >> > If I call runtime.LockOSThread(), it works good and no memory leak. >> > >> > why does a memory leak occur only when runtime.LockOSThread() is not >> called? >> >> We don't know because we don't know how the C code works. If it uses >> per-thread state, then you need to call runtime.LockOSThread in Go. >> But we don't know what it does. You'll need to look at the C code. >> >> Ian >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4f82b68e-ba91-44fa-bde3-a3bac67c2197n%40googlegroups.com.
