Hi,
I used something like the following to generate a memprof for 100 minutes
func main() {
flag.Parse()
if *cpuprofile != "" {
f, err := os.Create(*cpuprofile)
if err != nil {
fmt.Println("could not create CPU profile: ", err)
}
defer f.Close() // error handling omitted for example
if err := pprof.StartCPUProfile(f); err != nil {
fmt.Print("could not start CPU profile: ", err)
}
defer pprof.StopCPUProfile()
}
timeout := time.After(100 * time.Minute)
A_chan := make(chan bool)
B_chan := make(chan bool)
go util.A(A_chan)
go util.B(B_chan)
(..Rest of the code..)
for {
select {
case <-A_chan:
continue
case <-B_chan:
continue
case <-timeout:
break
}
break
}
if *memprofile != "" {
count = count + 1
fmt.Println("Generating Mem Profile:")
fmt.Print(count)
f, err := os.Create(*memprofile)
if err != nil {
fmt.Println("could not create memory profile: ", err)
}
defer f.Close() // error handling omitted for example
runtime.GC() // get up-to-date statistics
if err := pprof.WriteHeapProfile(f); err != nil {
fmt.Println("could not write memory profile: ", err)
}
}
}
/Desktop/memprof:go tool pprof --alloc_space main mem3.prof
Fetched 1 source profiles out of 2
File: main
Build ID: 99b8f2b91a4e037cf4a622aa32f2c1866764e7eb
Type: alloc_space
Time: Mar 22, 2020 at 7:02pm (IST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 17818.11MB, 87.65% of 20329.62MB total
<<<<<<<<<<<<<<<
Dropped 445 nodes (cum <= 101.65MB)
Showing top 10 nodes out of 58
flat flat% sum% cum cum%
11999.09MB 59.02% 59.02% 19800.37MB 97.40% home/nsaboo/project/aws.Events
1675.69MB 8.24% 67.27% 1911.69MB 9.40%
home/nsaboo/project/utils.Unflatten
627.21MB 3.09% 70.35% 1475.10MB 7.26% encoding/json.mapEncoder.encode
626.76MB 3.08% 73.43% 626.76MB 3.08% encoding/json.(*Decoder).refill
611.95MB 3.01% 76.44% 4557.69MB 22.42% home/nsaboo/project/lib.format
569.97MB 2.80% 79.25% 569.97MB 2.80% os.(*File).WriteString
558.95MB 2.75% 82.00% 2034.05MB 10.01% encoding/json.Marshal
447.51MB 2.20% 84.20% 447.51MB 2.20% reflect.copyVal
356.10MB 1.75% 85.95% 432.28MB 2.13% compress/flate.NewWriter
344.88MB 1.70% 87.65% 590.38MB 2.90% reflect.Value.MapKeys
1)Is this the correct way of getting a memory profile?
2)I ran the service for 100 minutes on a machine with 8GB memory. How am I
seeing memory accounting for around 17GB?
3)I understand 'flat' means memory occupied within that method, but how
come it shot up more than the available memory? Hence, asking if the above
process is the correct way of gathering the memory profile.
Thanks,
Nitish
On Fri, Mar 13, 2020 at 6:22 PM Michael Jones <michael.jo...@gmail.com>
wrote:
> hi. get the time at the start, check the elapsed time in your infinite
> loop, and trigger the write/exit after a minute, 10 minutes, 100 minutes,
> ...
>
> On Fri, Mar 13, 2020 at 5:45 AM Nitish Saboo <nitish.sabo...@gmail.com>
> wrote:
>
>> Hi Michael,
>>
>> Thanks for your response.
>>
>> That code looks wrong. I see the end but not the start. Look here and
>> copy carefully:
>>
>> >>Since I did not want cpu profiling I omitted the start of the code and
>> just added memory profiling part.
>>
>> Call at end, on way out.
>>
>> >>Oh yes, I missed that.I have to call memory profiling code at the end
>> on the way out.But the thing is that it runs as a service in infinite for
>> loop.
>>
>> func main() {
>> flag.Parse()
>> if *cpuprofile != "" {
>> f, err := os.Create(*cpuprofile)
>> if err != nil {
>> fmt.Println("could not create CPU profile: ", err)
>> }
>> defer f.Close() // error handling omitted for example
>> if err := pprof.StartCPUProfile(f); err != nil {
>> fmt.Print("could not start CPU profile: ", err)
>> }
>> defer pprof.StopCPUProfile()
>> }
>>
>> A_chan := make(chan bool)
>> B_chan := make(chan bool)
>> go util.A(A_chan)
>> go util.B(B_chan)
>> (..Rest of the code..)
>>
>> for {
>> select {
>> case <-A_chan:
>> continue
>> case <-B_chan:
>> continue
>>
>> }
>> }
>>
>> }
>>
>> What would be the correct way to add the memprofile code changes, since
>> it is running in an infinite for loop ?
>>
>> Also, as shared by others above, there are no promises about how soon the
>> dead allocations go away, The speed gets faster and faster version to
>> version, and is impressive indeed now, so old versions are not the best to
>> use, ubt even so, if the allocation feels small to th GC the urgency to
>> free it will be low. You need to loop in allocating and see if the memory
>> grows and grows.
>>
>> >> Yes, got it.I will try using the latest version of Go and check the
>> behavior.
>>
>> Thanks,
>> Nitish
>>
>> On Fri, Mar 13, 2020 at 6:20 AM Michael Jones <michael.jo...@gmail.com>
>> wrote:
>>
>>> That code looks wrong. I see the end but not the start. Look here and
>>> copy carefully:
>>> https://golang.org/pkg/runtime/pprof/
>>>
>>> Call at end, on way out.
>>>
>>> Also, as shared by others above, there are no promises about how soon
>>> the dead allocations go away, The speed gets faster and faster version to
>>> version, and is impressive indeed now, so old versions are not the best to
>>> use, ubt even so, if the allocation feels small to th GC the urgency to
>>> free it will be low. You need to loop in allocating and see if the memory
>>> grows and grows.
>>>
>>> On Thu, Mar 12, 2020 at 9:22 AM Nitish Saboo <nitish.sabo...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have compiled my Go binary against go version 'go1.7 linux/amd64'.
>>>> I added the following code change in the main function to get the
>>>> memory profiling of my service
>>>>
>>>> var memprofile = flag.String("memprofile", "", "write memory profile to
>>>> `file`")
>>>>
>>>> func main() {
>>>> flag.Parse()
>>>> if *memprofile != "" {
>>>> f, err := os.Create(*memprofile)
>>>> if err != nil {
>>>> fmt.Println("could not create memory profile: ", err)
>>>> }
>>>> defer f.Close() // error handling omitted for example
>>>> runtime.GC() // get up-to-date statistics
>>>> if err := pprof.WriteHeapProfile(f); err != nil {
>>>> fmt.Println("could not write memory profile: ", err)
>>>> }
>>>> }
>>>> ..
>>>> ..
>>>> (Rest code to follow)
>>>>
>>>> I ran the binary with the following command:
>>>>
>>>> nsaboo@ubuntu:./main -memprofile=mem.prof
>>>>
>>>> After running the service for couple of minutes, I stopped it and got
>>>> the file 'mem.prof'
>>>>
>>>> 1)mem.prof contains the following:
>>>>
>>>> nsaboo@ubuntu:~/Desktop/memprof$ vim mem.prof
>>>>
>>>> heap profile: 0: 0 [0: 0] @ heap/1048576
>>>>
>>>> # runtime.MemStats
>>>> # Alloc = 761184
>>>> # TotalAlloc = 1160960
>>>> # Sys = 3149824
>>>> # Lookups = 10
>>>> # Mallocs = 8358
>>>> # Frees = 1981
>>>> # HeapAlloc = 761184
>>>> # HeapSys = 1802240
>>>> # HeapIdle = 499712
>>>> # HeapInuse = 1302528
>>>> # HeapReleased = 0
>>>> # HeapObjects = 6377
>>>> # Stack = 294912 / 294912
>>>> # MSpan = 22560 / 32768
>>>> # MCache = 2400 / 16384
>>>> # BuckHashSys = 2727
>>>> # NextGC = 4194304
>>>> # PauseNs = [752083 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>> 0]
>>>> # NumGC = 1
>>>> # DebugGC = false
>>>>
>>>> 2)When I tried to open the file using the following command, it just
>>>> goes into interactive mode and shows nothing
>>>>
>>>> a)Output from go version go1.7 linux/amd64 for mem.prof
>>>>
>>>> nsaboo@ubuntu:~/Desktop/memprof$ go tool pprof mem.prof
>>>> Entering interactive mode (type "help" for commands)
>>>> (pprof) top
>>>> profile is empty
>>>> (pprof)
>>>>
>>>> b)Output from go version go1.12.4 linux/amd64 for mem.prof
>>>>
>>>> nsaboo@ubuntu:~/Desktop/memprof$ go tool pprof mem.prof
>>>> Type: space
>>>> No samples were found with the default sample value type.
>>>> Try "sample_index" command to analyze different sample values.
>>>> Entering interactive mode (type "help" for commands, "o" for options)
>>>> (pprof) o
>>>> call_tree = false
>>>> compact_labels = true
>>>> cumulative = flat //: [cum | flat]
>>>> divide_by = 1
>>>> drop_negative = false
>>>> edgefraction = 0.001
>>>> focus = ""
>>>> granularity = functions //: [addresses |
>>>> filefunctions | files | functions | lines]
>>>> hide = ""
>>>> ignore = ""
>>>> mean = false
>>>> nodecount = -1 //: default
>>>> nodefraction = 0.005
>>>> noinlines = false
>>>> normalize = false
>>>> output = ""
>>>> prune_from = ""
>>>> relative_percentages = false
>>>> sample_index = space //: [objects | space]
>>>> show = ""
>>>> show_from = ""
>>>> tagfocus = ""
>>>> taghide = ""
>>>> tagignore = ""
>>>> tagshow = ""
>>>> trim = true
>>>> trim_path = ""
>>>> unit = minimum
>>>> (pprof) space
>>>> (pprof) sample_index
>>>> (pprof) top
>>>> Showing nodes accounting for 0, 0% of 0 total
>>>> flat flat% sum% cum cum%
>>>>
>>>>
>>>> 3)Please let me know if it is this the correct way of getting the
>>>> memory profiling ?
>>>>
>>>> 4)Can we deduce something from this memory stats that points us to
>>>> increase in memory usage?
>>>>
>>>> 5)I am just thinking out loud, since I am using go1.7, can that be the
>>>> reason for the issue of increase in memory usage that might get fixed with
>>>> latest go versions ?
>>>>
>>>> Thanks,
>>>> Nitish
>>>>
>>>> On Tue, Mar 10, 2020 at 6:56 AM Jake Montgomery <jake6...@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> On Monday, March 9, 2020 at 1:37:00 PM UTC-4, Nitish Saboo wrote:
>>>>>>
>>>>>> Hi Jake,
>>>>>>
>>>>>> The memory usage remains constant when the rest of the service is
>>>>>> running.Only when LoadPatternDB() method is called within the service,
>>>>>> Memory Consumption increases which actually should not happen.
>>>>>> I am assuming if there is a memory leak while calling this method
>>>>>> because the memory usage then becomes constant after getting increased
>>>>>> and
>>>>>> then further increases on next call.
>>>>>>
>>>>>
>>>>> Its possible that I am not fully understanding, perhaps a language
>>>>> problem. But from what you have written above I still don't see that this
>>>>> means you definitely have a memory leak. To test for that you would need
>>>>> to *continuously
>>>>> *call LoadPatternDB() and monitor memory for a *considerable time*.
>>>>> If it eventually stabilizes to a constant range then there is no leak,
>>>>> just
>>>>> normal Go-GC variation. If it never stops climbing, and eventually
>>>>> consumes
>>>>> all the memory, then it would probably be a leak. Just because it goes up
>>>>> after one call, or a few calls doe not mean there is a leak.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "golang-nuts" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/golang-nuts/f897fdb1-8968-4435-9fe9-02e167e09a36%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/golang-nuts/f897fdb1-8968-4435-9fe9-02e167e09a36%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/golang-nuts/CALjMrq6DC98p4M4V2QCbQFTcsL1PtOWELvg8MEcMYj9EM9ui_A%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/golang-nuts/CALjMrq6DC98p4M4V2QCbQFTcsL1PtOWELvg8MEcMYj9EM9ui_A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>
>>>
>>> --
>>>
>>> *Michael T. jonesmichael.jo...@gmail.com <michael.jo...@gmail.com>*
>>>
>>
>
> --
>
> *Michael T. jonesmichael.jo...@gmail.com <michael.jo...@gmail.com>*
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALjMrq7_d8s%3DmS5WVgV9K1m5VCBUoep2mitvX4o%3D%2BHVqf1APmQ%40mail.gmail.com.
