(Sorry for the typo. Where ever I mentioned alloc -> heap, I mean all -> heap)
On Wednesday, May 26, 2021 at 3:01:21 PM UTC+5:30 varun...@gmail.com wrote: > Thanks @Ian for the clarification. > > The BySize arrays gives the number of active objects in a size class but > it does not give the total memory allocated for a size class. E.g, for 1k > size class, if 100K was the total size allocated and there are only 10 > objects in it, it does not tell the memory consumption of these 10 objects. > > I partially got what I am looking from the core dump. Taking core dump of > the process and analysing it with viewcore shows the follows: > > (viewcore) breakdown > all 11055939584 100.00% > text 28172288 0.25% > readonly 7422455808 67.14% > data 3137536 0.03% > bss 1497206784 13.54% (grab bag, includes OS thread > stacks, ...) > heap 2037923840 <(203)%20792-3840> 18.43% > in use spans 1340497920 12.12% > alloc 1306139856 11.81% > live 269335312 2.44% > garbage 1036804544 9.38% > free 19920896 0.18% > round 14437168 0.13% > manual spans 424214528 3.84% (Go stacks) > alloc 420808704 3.81% > free 3405824 0.03% > free spans 273211392 2.47% > retained 256065536 2.32% (kept for reuse by Go) > released 17145856 0.16% (given back to the OS) > ptr bitmap 65011712 0.59% > span table 2031616 0.02% > (viewcore) > > Cross referencing the output with this comment: > https://github.com/golang/go/issues/32284#issuecomment-497851388, alloc > -> heap -> in use spans seems to be HeapInUse. In the alloc of 1306139856 > bytes, there seems to be 1036804544 garbage. > > I have a follow-up question on this: > > For the memory profile taken around the same time as core dump, the total > memory seems to be 688M > (pprof) top 10 > Showing nodes accounting for 592.62MB, 86.06% of 688.58MB total > Dropped 178 nodes (cum <= 3.44MB) > > alloc -> heap -> in use spans -> live only accounts for 269M. Is the rest > ~420M in the profile coming from StackInUse (alloc -> heap -> manual spans > )? > > Is it always the case that StackInUse is shown in heap profile ? > > Thanks, > Varun > > On Wednesday, May 26, 2021 at 1:24:32 AM UTC+5:30 Ian Lance Taylor wrote: > >> On Tue, May 25, 2021 at 6:53 AM varun...@gmail.com <varun...@gmail.com> >> wrote: >> > >> > This statement is a bit confusing to me regarding HeapInuse: "which >> includes space allocated to hold objects that do not yet exist or that have >> been released by the garbage collector" >> > >> > According to memstats documentation, HeapInUse -> Includes all spans >> that have at least one object in them. >> > >> > So, why should it include space for objects that do not exist any more >> or that has been released by garbage collector. Should it not go to >> HeapIdle? >> >> A typical span can contain a large number of objects. Each one of >> those objects may be allocated or not. If a span contains at least >> one allocated object, the total size of the span will be counted as >> part of HeapInUse. >> >> >> > Is it the case of internal fragmentation. For example, if a span is >> shared by 2 objects and one object is freed - then, the entire span is >> accounted under HeapInUse. But since there is free space in span, it can be >> used by new objects and it can not be returned to HeapIdle as there is one >> object in the span. When pprof runs, it only uses the live objects and >> therefore, it does not account for the free space in spans while HeapInUse >> will account for it. >> >> Yes. >> >> >> > If this is the case, then how can we know the percentage of utilisation >> across a size class. >> >> That seems like a different question. But you can find the answer to >> that question by looking at the BySize array. >> >> Ian >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ac625f40-ae41-4e40-9d9a-9d4aa5e62af6n%40googlegroups.com.
