RE: [go-nuts] Memory leak when calling highly parallel goroutines

['Rob Archibald' via golang-nuts]

Thanks for your help on this! I really appreciate it. I've been pulling my hair 
out for weeks on this issue, so I truly appreciate any help you can offer.

The way this runs in my production app is similar to how it works in the 
example app I provided. A request is sent to a web server and then up to 1000+ 
http requests are made. Data is pulled, parsed, and ultimately added to a 
database. Each of those requests are done in goroutines and each request 
finishes. But, the memory is never cleaned up. That's why I created this the 
way that I did. I'm aware that if the goroutines never finish or if I create 
them forever, it'll grow infinitely, but that's not what I'm doing. Requests 
come in and are processed and completed, but they're never cleaned up.

As I just mentioned in the Github issue you referenced, it looks like this 
example isn't actually breaking in Linux after all so I'll put together one 
that does break on Linux too. 

Blessings,
Rob Archibald
CTO, EndFirst LLC
r...@robarchibald.com


-Original Message-
From: Ian Lance Taylor [mailto:i...@golang.org] 
Sent: Wednesday, October 25, 2017 11:03 AM
To: Rob Archibald
Cc: golang-nuts
Subject: Re: [go-nuts] Memory leak when calling highly parallel goroutines

On Wed, Oct 25, 2017 at 10:21 AM, Rob Archibald  wrote:
>
> It is a memory leak because the memory continues to grow for as long 
> as the loop continues to run. If you change the code example I gave to 
> an infinite loop you'll see that it grows infinitely. I had it stop at 
> 1000 for demonstration purposes because each scan request that my 
> production app gets typically kicks off about 1000 goroutines.
>
> I posted to the issue you linked to as well. Thanks for letting me 
> know about that.
>
> I'm measuring memory from the system perspective. Windows and Linux 
> both report gigs of memory used on my production app before it crashes 
> due to out of memory. This is a show-stopper issue for me. If Go can't 
> release memory from a simple goroutine when it exits, I'll have to 
> rewrite using something else.

Sorry, I didn't mean to suggest that you should add your program to that issue. 
 I only meant to point to that issue as describing a known problem with 
programs that start a very large number of goroutines and then go back to just 
using a small number of goroutines.  If your program continues to use an 
increasing amount of memory over time, then it is a different problem.

When you say "measuring memory from the system perspective," what precisely do 
you mean?  It really matters, as different system measurements report different 
things, and none of them really correspond to how Go manages its heap.  Or, to 
put it another way, what is the real problem?

I tried running your program.  For me the virtual memory size as measured by 
the ps program goes up to 42176 (42M) and then stops.
This is on GNU/Linux, though, not Windows.

You say that you are describing a memory leak because memory continues to grow 
for as long as the loop continues to run.  Since the loop is starting new 
goroutines more quickly than they can finish, that is not surprising.  If you 
want to control memory usage with a loop like that, you need to limit the 
number of goroutines you start in parallel.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Memory leak when calling highly parallel goroutines

['Rob Archibald' via golang-nuts]

Thanks Ian,
It is a memory leak because the memory continues to grow for as long as the 
loop continues to run. If you change the code example I gave to an infinite 
loop you'll see that it grows infinitely. I had it stop at 1000 for 
demonstration purposes because each scan request that my production app gets 
typically kicks off about 1000 goroutines. 

I posted to the issue you linked to as well. Thanks for letting me know about 
that. 

I'm measuring memory from the system perspective. Windows and Linux both report 
gigs of memory used on my production app before it crashes due to out of 
memory. This is a show-stopper issue for me. If Go can't release memory from a 
simple goroutine when it exits, I'll have to rewrite using something else. 

Blessings,
Rob Archibald
CTO, EndFirst LLC
r...@robarchibald.com

> On Oct 25, 2017, at 6:50 AM, Ian Lance Taylor  wrote:
> 
>> On Wed, Oct 25, 2017 at 12:06 AM,   wrote:
>> 
>> I've got a nasty memory leak. It appears that Go won't reclaim memory when
>> goroutines are called in a highly parallel fashion.
>> 
>> To illustrate the issue I've written a simple example below. When I run this
>> on my Windows system, I get 800MB to 1 GB of memory consumption.
> 
> How are you measuring memory consumption?
> 
> A memory leak implies that memory continues growing without bound.
> That is not what you seem to be reporting.  What you are reporting is
> high memory usage in a steady state.  It's certainly true that if you
> create a large number of goroutines in parallel your program will need
> a bunch of memory to support those goroutines.  And once a program
> requires a large amount of memory, it will hold on to that memory for
> a while, on the assumption that it is likely to happen again.  Over
> time unneeded memory will be released back to the system, where "over
> time" means several minutes.
> 
> That said, there is a current issue: the G structure that represents a
> goroutine is never released.  So if your program starts a large number
> of goroutines at one time, and then never does that again, you will be
> left with some memory that is never released.  This is
> https://golang.org/issue/9869.
> 
> Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.