Hi Tamas,
1) There is no such option '--heap_inuse'. We have an -*-inuse_space*
option. Is this what you are talking about?
nsaboo@ubuntu:~/Desktop/memprof$ go tool pprof --inuse_space main mem3.prof
Fetched 1 source profiles out of 2
File: main
Build ID: 99b8f2b91a4e037cf4a622aa32f2c1866764e7eb
Type: inuse_space
Time: Mar 22, 2020 at 6:32am (PDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) o
call_tree = false
compact_labels = true
cumulative = flat //: [cum | flat]
divide_by = 1
drop_negative = false
edgefraction = 0.001
focus = ""
granularity = filefunctions //: [addresses |
filefunctions | files | functions | lines]
hide = ""
ignore = ""
mean = false
nodecount = -1 //: default
nodefraction = 0.005
noinlines = false
normalize = false
output = ""
prune_from = ""
relative_percentages = false
sample_index = inuse_space //: [alloc_objects |
alloc_space | inuse_objects | inuse_space]
show = ""
show_from = ""
tagfocus = ""
taghide = ""
tagignore = ""
tagshow = ""
trim = true
trim_path = ""
unit = minimum
2) When I don't pass the flag it defaults to *--inuse_space*:
File: main
Build ID: 99b8f2b91a4e037cf4a622aa32f2c1866764e7eb
Type: inuse_space
Time: Mar 22, 2020 at 6:32am (PDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 3303.21kB, 100% of 3303.21kB total
Showing top 10 nodes out of 28
flat flat% sum% cum cum%
1762.94kB 53.37% 53.37% 1762.94kB 53.37% runtime/pprof.StartCPUProfile
516.01kB 15.62% 68.99% 516.01kB 15.62% runtime/pprof.(*profMap).lookup
512.19kB 15.51% 84.50% 512.19kB 15.51% runtime.malg
512.08kB 15.50% 100% 512.08kB 15.50%
crypto/x509/pkix.(*Name).FillFromRDNSequence
0 0% 100% 512.08kB 15.50% crypto/tls.(*Conn).Handshake
0 0% 100% 512.08kB 15.50%
crypto/tls.(*Conn).clientHandshake
0 0% 100% 512.08kB 15.50%
crypto/tls.(*Conn).verifyServerCertificate
0 0% 100% 512.08kB 15.50%
crypto/tls.(*clientHandshakeState).doFullHandshake
0 0% 100% 512.08kB 15.50%
crypto/tls.(*clientHandshakeState).handshake
0 0% 100% 512.08kB 15.50%
crypto/x509.(*CertPool).AppendCertsFromPEM
Please correct me if I am missing something here.
Thanks,
Nitish
On Wed, Mar 25, 2020 at 12:16 AM Tamás Gulácsi <tgulacs...@gmail.com> wrote:
> You've requested the total allocated space (--alloc_space), not only the
> heap used (--heap_inuse, or no flag).
> So that 17GiB is the total allocated size, does NOT include the released!
>
> 2020. március 24., kedd 15:16:46 UTC+1 időpontban Nitish Saboo a
> következőt írta:
>>
>> Hi,
>>
>> I have already gone through those links. They helped me to gather the mem
>> profile and while analyzing the data(as given in those links) I have come
>> across the following issue:
>>
>> While I was running the service for 100 minutes the 'top' command output
>> was showing Mem% as 11.1. There was no increase in mem usage since I had
>> not called 'LoadPatternDB()' method. I have 8GB of memory on the node where
>> I am running the service. My issue is :
>>
>>
>> - Why is it showing memory accounting for around 17GB? 11.1 % of 8GB
>> is .88GB and my node is only of 8GB. I feel the way I gathered the mem
>> profiling was not correct ..is it ?
>>
>> Please let me know where am I going wrong?
>>
>> Thanks,
>> Nitish
>>
>> On Tue, Mar 24, 2020 at 5:32 PM Nitish Saboo <nitish...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> >>There is no root analysis available in Go. Read the paper I linked to.
>>>
>>> Sorry I did not get you. Which paper are you referring to?
>>>
>>> While I was running the service for 100 minutes the 'top' command output
>>> was showing Mem% as 11.1. There was no increase in mem usage since I had
>>> not called 'LoadPatternDB()' method.I have 8GB of memory on the node where
>>> I am running the service. My issue is :
>>>
>>>
>>> - Why is it showing memory accounting for around 17GB? 11.1 % of
>>> 8GB is .88GB and my node is only of 8GB. I feel the way I gathered the
>>> mem
>>> profiling was not correct ..is it ?
>>>
>>> Please advise me what am I missing?
>>>
>>> Thanks,
>>> Nitish
>>>
>>> On Tue, Mar 24, 2020 at 1:28 AM Robert Engels <ren...@ix.netcom.com>
>>> wrote:
>>>
>>>> Yes. You have a leak in your Go code. It shows you the object types
>>>> that are taking up all of the space. There is no root analysis available in
>>>> Go. Read the paper I linked to.
>>>>
>>>> On Mar 23, 2020, at 9:12 AM, Nitish Saboo <nitish...@gmail.com> wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I used something like the following to generate a memprof for 100
>>>> minutes
>>>>
>>>> func main() {
>>>> flag.Parse()
>>>> if *cpuprofile != "" {
>>>> f, err := os.Create(*cpuprofile)
>>>> if err != nil {
>>>> fmt.Println("could not create CPU profile: ", err)
>>>> }
>>>> defer f.Close() // error handling omitted for example
>>>> if err := pprof.StartCPUProfile(f); err != nil {
>>>> fmt.Print("could not start CPU profile: ", err)
>>>> }
>>>> defer pprof.StopCPUProfile()
>>>> }
>>>> timeout := time.After(100 * time.Minute)
>>>> A_chan := make(chan bool)
>>>> B_chan := make(chan bool)
>>>> go util.A(A_chan)
>>>> go util.B(B_chan)
>>>> (..Rest of the code..)
>>>>
>>>> for {
>>>> select {
>>>> case <-A_chan:
>>>> continue
>>>> case <-B_chan:
>>>> continue
>>>> case <-timeout:
>>>> break
>>>> }
>>>> break
>>>> }
>>>>
>>>> if *memprofile != "" {
>>>> count = count + 1
>>>> fmt.Println("Generating Mem Profile:")
>>>> fmt.Print(count)
>>>> f, err := os.Create(*memprofile)
>>>> if err != nil {
>>>> fmt.Println("could not create memory profile: ", err)
>>>> }
>>>> defer f.Close() // error handling omitted for example
>>>> runtime.GC() // get up-to-date statistics
>>>> if err := pprof.WriteHeapProfile(f); err != nil {
>>>> fmt.Println("could not write memory profile: ", err)
>>>> }
>>>>
>>>> }
>>>> }
>>>>
>>>> /Desktop/memprof:go tool pprof --alloc_space main mem3.prof
>>>> Fetched 1 source profiles out of 2
>>>> File: main
>>>> Build ID: 99b8f2b91a4e037cf4a622aa32f2c1866764e7eb
>>>> Type: alloc_space
>>>> Time: Mar 22, 2020 at 7:02pm (IST)
>>>> Entering interactive mode (type "help" for commands, "o" for options)
>>>> (pprof) top
>>>> Showing nodes accounting for 17818.11MB, 87.65% of 20329.62MB total
>>>> <<<<<<<<<<<<<<<
>>>> Dropped 445 nodes (cum <= 101.65MB)
>>>> Showing top 10 nodes out of 58
>>>> flat flat% sum% cum cum%
>>>> 11999.09MB 59.02% 59.02% 19800.37MB 97.40%
>>>> home/nsaboo/project/aws.Events
>>>> 1675.69MB 8.24% 67.27% 1911.69MB 9.40%
>>>> home/nsaboo/project/utils.Unflatten
>>>> 627.21MB 3.09% 70.35% 1475.10MB 7.26%
>>>> encoding/json.mapEncoder.encode
>>>> 626.76MB 3.08% 73.43% 626.76MB 3.08%
>>>> encoding/json.(*Decoder).refill
>>>> 611.95MB 3.01% 76.44% 4557.69MB 22.42%
>>>> home/nsaboo/project/lib.format
>>>> 569.97MB 2.80% 79.25% 569.97MB 2.80% os.(*File).WriteString
>>>> 558.95MB 2.75% 82.00% 2034.05MB 10.01% encoding/json.Marshal
>>>> 447.51MB 2.20% 84.20% 447.51MB 2.20% reflect.copyVal
>>>> 356.10MB 1.75% 85.95% 432.28MB 2.13% compress/flate.NewWriter
>>>> 344.88MB 1.70% 87.65% 590.38MB 2.90% reflect.Value.MapKeys
>>>>
>>>> 1)Is this the correct way of getting a memory profile?
>>>>
>>>> 2)I ran the service for 100 minutes on a machine with 8GB memory. How
>>>> am I seeing memory accounting for around 17GB?
>>>>
>>>> 3)I understand 'flat' means memory occupied within that method, but how
>>>> come it shot up more than the available memory? Hence, asking if the above
>>>> process is the correct way of gathering the memory profile.
>>>>
>>>> Thanks,
>>>> Nitish
>>>>
>>>> On Fri, Mar 13, 2020 at 6:22 PM Michael Jones <michae...@gmail.com>
>>>> wrote:
>>>>
>>>>> hi. get the time at the start, check the elapsed time in your infinite
>>>>> loop, and trigger the write/exit after a minute, 10 minutes, 100 minutes,
>>>>> ...
>>>>>
>>>>> On Fri, Mar 13, 2020 at 5:45 AM Nitish Saboo <nitish...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> Thanks for your response.
>>>>>>
>>>>>> That code looks wrong. I see the end but not the start. Look here and
>>>>>> copy carefully:
>>>>>>
>>>>>> >>Since I did not want cpu profiling I omitted the start of the code
>>>>>> and just added memory profiling part.
>>>>>>
>>>>>> Call at end, on way out.
>>>>>>
>>>>>> >>Oh yes, I missed that.I have to call memory profiling code at the
>>>>>> end on the way out.But the thing is that it runs as a service in infinite
>>>>>> for loop.
>>>>>>
>>>>>> func main() {
>>>>>> flag.Parse()
>>>>>> if *cpuprofile != "" {
>>>>>> f, err := os.Create(*cpuprofile)
>>>>>> if err != nil {
>>>>>> fmt.Println("could not create CPU profile: ", err)
>>>>>> }
>>>>>> defer f.Close() // error handling omitted for example
>>>>>> if err := pprof.StartCPUProfile(f); err != nil {
>>>>>> fmt.Print("could not start CPU profile: ", err)
>>>>>> }
>>>>>> defer pprof.StopCPUProfile()
>>>>>> }
>>>>>>
>>>>>> A_chan := make(chan bool)
>>>>>> B_chan := make(chan bool)
>>>>>> go util.A(A_chan)
>>>>>> go util.B(B_chan)
>>>>>> (..Rest of the code..)
>>>>>>
>>>>>> for {
>>>>>> select {
>>>>>> case <-A_chan:
>>>>>> continue
>>>>>> case <-B_chan:
>>>>>> continue
>>>>>>
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> What would be the correct way to add the memprofile code changes,
>>>>>> since it is running in an infinite for loop ?
>>>>>>
>>>>>> Also, as shared by others above, there are no promises about how soon
>>>>>> the dead allocations go away, The speed gets faster and faster version to
>>>>>> version, and is impressive indeed now, so old versions are not the best
>>>>>> to
>>>>>> use, ubt even so, if the allocation feels small to th GC the urgency to
>>>>>> free it will be low. You need to loop in allocating and see if the memory
>>>>>> grows and grows.
>>>>>>
>>>>>> >> Yes, got it.I will try using the latest version of Go and check
>>>>>> the behavior.
>>>>>>
>>>>>> Thanks,
>>>>>> Nitish
>>>>>>
>>>>>> On Fri, Mar 13, 2020 at 6:20 AM Michael Jones <michae...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> That code looks wrong. I see the end but not the start. Look here
>>>>>>> and copy carefully:
>>>>>>> https://golang.org/pkg/runtime/pprof/
>>>>>>>
>>>>>>> Call at end, on way out.
>>>>>>>
>>>>>>> Also, as shared by others above, there are no promises about how
>>>>>>> soon the dead allocations go away, The speed gets faster and faster
>>>>>>> version
>>>>>>> to version, and is impressive indeed now, so old versions are not the
>>>>>>> best
>>>>>>> to use, ubt even so, if the allocation feels small to th GC the urgency
>>>>>>> to
>>>>>>> free it will be low. You need to loop in allocating and see if the
>>>>>>> memory
>>>>>>> grows and grows.
>>>>>>>
>>>>>>> On Thu, Mar 12, 2020 at 9:22 AM Nitish Saboo <nitish...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have compiled my Go binary against go version 'go1.7 linux/amd64'.
>>>>>>>> I added the following code change in the main function to get the
>>>>>>>> memory profiling of my service
>>>>>>>>
>>>>>>>> var memprofile = flag.String("memprofile", "", "write memory
>>>>>>>> profile to `file`")
>>>>>>>>
>>>>>>>> func main() {
>>>>>>>> flag.Parse()
>>>>>>>> if *memprofile != "" {
>>>>>>>> f, err := os.Create(*memprofile)
>>>>>>>> if err != nil {
>>>>>>>> fmt.Println("could not create memory profile: ", err)
>>>>>>>> }
>>>>>>>> defer f.Close() // error handling omitted for example
>>>>>>>> runtime.GC() // get up-to-date statistics
>>>>>>>> if err := pprof.WriteHeapProfile(f); err != nil {
>>>>>>>> fmt.Println("could not write memory profile: ", err)
>>>>>>>> }
>>>>>>>> }
>>>>>>>> ..
>>>>>>>> ..
>>>>>>>> (Rest code to follow)
>>>>>>>>
>>>>>>>> I ran the binary with the following command:
>>>>>>>>
>>>>>>>> nsaboo@ubuntu:./main -memprofile=mem.prof
>>>>>>>>
>>>>>>>> After running the service for couple of minutes, I stopped it and
>>>>>>>> got the file 'mem.prof'
>>>>>>>>
>>>>>>>> 1)mem.prof contains the following:
>>>>>>>>
>>>>>>>> nsaboo@ubuntu:~/Desktop/memprof$ vim mem.prof
>>>>>>>>
>>>>>>>> heap profile: 0: 0 [0: 0] @ heap/1048576
>>>>>>>>
>>>>>>>> # runtime.MemStats
>>>>>>>> # Alloc = 761184
>>>>>>>> # TotalAlloc = 1160960
>>>>>>>> # Sys = 3149824
>>>>>>>> # Lookups = 10
>>>>>>>> # Mallocs = 8358
>>>>>>>> # Frees = 1981
>>>>>>>> # HeapAlloc = 761184
>>>>>>>> # HeapSys = 1802240
>>>>>>>> # HeapIdle = 499712
>>>>>>>> # HeapInuse = 1302528
>>>>>>>> # HeapReleased = 0
>>>>>>>> # HeapObjects = 6377
>>>>>>>> # Stack = 294912 / 294912
>>>>>>>> # MSpan = 22560 / 32768
>>>>>>>> # MCache = 2400 / 16384
>>>>>>>> # BuckHashSys = 2727
>>>>>>>> # NextGC = 4194304
>>>>>>>> # PauseNs = [752083 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>>>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>>>>>> 0 0 0
>>>>>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>>>>>> 0 0 0
>>>>>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>>>>>> 0 0 0
>>>>>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>>>>>> 0 0 0
>>>>>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>>>>>> 0 0 0
>>>>>>>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
>>>>>>>> 0 0 0
>>>>>>>> 0 0 0]
>>>>>>>> # NumGC = 1
>>>>>>>> # DebugGC = false
>>>>>>>>
>>>>>>>> 2)When I tried to open the file using the following command, it
>>>>>>>> just goes into interactive mode and shows nothing
>>>>>>>>
>>>>>>>> a)Output from go version go1.7 linux/amd64 for mem.prof
>>>>>>>>
>>>>>>>> nsaboo@ubuntu:~/Desktop/memprof$ go tool pprof mem.prof
>>>>>>>> Entering interactive mode (type "help" for commands)
>>>>>>>> (pprof) top
>>>>>>>> profile is empty
>>>>>>>> (pprof)
>>>>>>>>
>>>>>>>> b)Output from go version go1.12.4 linux/amd64 for mem.prof
>>>>>>>>
>>>>>>>> nsaboo@ubuntu:~/Desktop/memprof$ go tool pprof mem.prof
>>>>>>>> Type: space
>>>>>>>> No samples were found with the default sample value type.
>>>>>>>> Try "sample_index" command to analyze different sample values.
>>>>>>>> Entering interactive mode (type "help" for commands, "o" for
>>>>>>>> options)
>>>>>>>> (pprof) o
>>>>>>>> call_tree = false
>>>>>>>> compact_labels = true
>>>>>>>> cumulative = flat //: [cum | flat]
>>>>>>>> divide_by = 1
>>>>>>>> drop_negative = false
>>>>>>>> edgefraction = 0.001
>>>>>>>> focus = ""
>>>>>>>> granularity = functions //: [addresses |
>>>>>>>> filefunctions | files | functions | lines]
>>>>>>>> hide = ""
>>>>>>>> ignore = ""
>>>>>>>> mean = false
>>>>>>>> nodecount = -1 //: default
>>>>>>>> nodefraction = 0.005
>>>>>>>> noinlines = false
>>>>>>>> normalize = false
>>>>>>>> output = ""
>>>>>>>> prune_from = ""
>>>>>>>> relative_percentages = false
>>>>>>>> sample_index = space //: [objects |
>>>>>>>> space]
>>>>>>>> show = ""
>>>>>>>> show_from = ""
>>>>>>>> tagfocus = ""
>>>>>>>> taghide = ""
>>>>>>>> tagignore = ""
>>>>>>>> tagshow = ""
>>>>>>>> trim = true
>>>>>>>> trim_path = ""
>>>>>>>> unit = minimum
>>>>>>>> (pprof) space
>>>>>>>> (pprof) sample_index
>>>>>>>> (pprof) top
>>>>>>>> Showing nodes accounting for 0, 0% of 0 total
>>>>>>>> flat flat% sum% cum cum%
>>>>>>>>
>>>>>>>>
>>>>>>>> 3)Please let me know if it is this the correct way of getting the
>>>>>>>> memory profiling ?
>>>>>>>>
>>>>>>>> 4)Can we deduce something from this memory stats that points us to
>>>>>>>> increase in memory usage?
>>>>>>>>
>>>>>>>> 5)I am just thinking out loud, since I am using go1.7, can that be
>>>>>>>> the reason for the issue of increase in memory usage that might get
>>>>>>>> fixed
>>>>>>>> with latest go versions ?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Nitish
>>>>>>>>
>>>>>>>> On Tue, Mar 10, 2020 at 6:56 AM Jake Montgomery <jake...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Monday, March 9, 2020 at 1:37:00 PM UTC-4, Nitish Saboo wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Jake,
>>>>>>>>>>
>>>>>>>>>> The memory usage remains constant when the rest of the service is
>>>>>>>>>> running.Only when LoadPatternDB() method is called within the
>>>>>>>>>> service,
>>>>>>>>>> Memory Consumption increases which actually should not happen.
>>>>>>>>>> I am assuming if there is a memory leak while calling this
>>>>>>>>>> method because the memory usage then becomes constant after getting
>>>>>>>>>> increased and then further increases on next call.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Its possible that I am not fully understanding, perhaps a language
>>>>>>>>> problem. But from what you have written above I still don't see that
>>>>>>>>> this
>>>>>>>>> means you definitely have a memory leak. To test for that you would
>>>>>>>>> need to *continuously
>>>>>>>>> *call LoadPatternDB() and monitor memory for a *considerable time*.
>>>>>>>>> If it eventually stabilizes to a constant range then there is no
>>>>>>>>> leak, just
>>>>>>>>> normal Go-GC variation. If it never stops climbing, and eventually
>>>>>>>>> consumes
>>>>>>>>> all the memory, then it would probably be a leak. Just because it
>>>>>>>>> goes up
>>>>>>>>> after one call, or a few calls doe not mean there is a leak.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "golang-nuts" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to golan...@googlegroups.com.
>>>>>>>>> To view this discussion on the web visit
>>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/f897fdb1-8968-4435-9fe9-02e167e09a36%40googlegroups.com
>>>>>>>>> <https://groups.google.com/d/msgid/golang-nuts/f897fdb1-8968-4435-9fe9-02e167e09a36%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "golang-nuts" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to golan...@googlegroups.com.
>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/CALjMrq6DC98p4M4V2QCbQFTcsL1PtOWELvg8MEcMYj9EM9ui_A%40mail.gmail.com
>>>>>>>> <https://groups.google.com/d/msgid/golang-nuts/CALjMrq6DC98p4M4V2QCbQFTcsL1PtOWELvg8MEcMYj9EM9ui_A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>> .
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Michael T. jonesmichae...@gmail.com*
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Michael T. jonesmichae...@gmail.com*
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to golan...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/golang-nuts/CALjMrq7_d8s%3DmS5WVgV9K1m5VCBUoep2mitvX4o%3D%2BHVqf1APmQ%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/golang-nuts/CALjMrq7_d8s%3DmS5WVgV9K1m5VCBUoep2mitvX4o%3D%2BHVqf1APmQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/6455e855-3f1f-4836-ab58-2256e97f7eef%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/6455e855-3f1f-4836-ab58-2256e97f7eef%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALjMrq40q9U59xQQDja4tkgXmJTbT0G9PKzzjWL51A-wJw8h5g%40mail.gmail.com.
