Re: [vpp-dev] naive questions on VPP memory usage ( does it ever come down )
Thanks Dave and Damjan for quick inputs. This helps in understanding the memory mgmt in vpp. Inline to your questions. 1. Can you capture "show memory main-heapsh” before and after? We are currently in fdio.1810 and i am not seeing this option. 2. One more quick question. With this model of memory mgmt, the traditional linux tools like pmap/smap will not give a clear view of exact memory usage and if any leaks are there. I have tried memory-trace on feature along with "show memory verbose" However, I am not able to map a particular allocation to a particular deallocation to see if a memory leak is present or not. "show memory verbose" It captures only the program counter of allocation/deallocations, but not the pointer at which the allocation /deallocation happened. Due to this, I am not able to find if there is a leak. Is there any trick to map allocation/deallocations one-on-one to find leaks from "show memory" output. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15971): https://lists.fd.io/g/vpp-dev/message/15971 Mute This Topic: https://lists.fd.io/mt/72699704/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] naive questions on VPP memory usage ( does it ever come down )
Can you capture "show memory main-heapsh” before and after? Will show you exact number of pages mapped. I.e. vpp# show memory main-heap Thread 0 vpp_main virtual memory start 0x7f13c03ae000, size 1048640k, 262160 pages, page size 4k numa 1: 45220 pages, 180880k not mapped: 216940 pages, 867760k total: 1.00G, used: 43.19M, free: 980.88M, trimmable: 978.81M This means that 45220 pages (180MB) out of 262160 (1G of 4K pages) is mapped. New page is dynamically allocated by kernel 1st time we write a byte into its memory address. In VPP page is never freed, even if we free all allocations which hold that page, kernel still thinks it is used by us and VPP memory allocator will reuse it. Effect of this is that if you allocate 800Mb out of heap, touch at least one byte at each page, and then you free that allocation then freed memory will be available for other allocations, but from kernel perspective 800MB of pages will stay in use for a VPP lifetime ... — Damjan > On 1 Apr 2020, at 14:58, Satya Murthy wrote: > > Few questions on VPP memory usage. > > 1. Using pmap -p i am collecting the total memory usage of vpp > process at the beginning of my test. ( it is X KB ) > 2. I ran test for few hours which will obviously have lot of pool_get/pool_put > 3. Collected the same same pmap output and the memory usage grown to X+Y > 4. Cleared all my sessions which ideally should have cleared all my > allocations. > 5. I still see the memory usage being stuck at X+Y > > I see that only pool_free is doing unmap of the memory, whereas pool_put is > not doing so. > Could this be the reason why I am seeing memory not coming down. > > Thanks & Regards, > Satish > > > -- > Thanks & Regards, > Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15969): https://lists.fd.io/g/vpp-dev/message/15969 Mute This Topic: https://lists.fd.io/mt/72699704/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] naive questions on VPP memory usage ( does it ever come down )
Could be. In real life, objects allocated from pools have widely variable lifetimes. It would be virtually impossible to unmap less than an entire pool. If your application would tolerate something akin garbage collection, I suppose you could create a fresh pool with a fresh set of indices, and free the original. Personally, I would never go in this direction. HTH... Dave From: vpp-dev@lists.fd.io On Behalf Of Satya Murthy Sent: Wednesday, April 1, 2020 8:59 AM To: vpp-dev@lists.fd.io Subject: [vpp-dev] naive questions on VPP memory usage ( does it ever come down ) Few questions on VPP memory usage. 1. Using pmap -p i am collecting the total memory usage of vpp process at the beginning of my test. ( it is X KB ) 2. I ran test for few hours which will obviously have lot of pool_get/pool_put 3. Collected the same same pmap output and the memory usage grown to X+Y 4. Cleared all my sessions which ideally should have cleared all my allocations. 5. I still see the memory usage being stuck at X+Y I see that only pool_free is doing unmap of the memory, whereas pool_put is not doing so. Could this be the reason why I am seeing memory not coming down. Thanks & Regards, Satish -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15968): https://lists.fd.io/g/vpp-dev/message/15968 Mute This Topic: https://lists.fd.io/mt/72699704/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] naive questions on VPP memory usage ( does it ever come down )
Few questions on VPP memory usage. 1. Using pmap -p i am collecting the total memory usage of vpp process at the beginning of my test. ( it is X KB ) 2. I ran test for few hours which will obviously have lot of pool_get/pool_put 3. Collected the same same pmap output and the memory usage grown to X+Y 4. Cleared all my sessions which ideally should have cleared all my allocations. 5. I still see the memory usage being stuck at X+Y I see that only pool_free is doing unmap of the memory, whereas pool_put is not doing so. Could this be the reason why I am seeing memory not coming down. Thanks & Regards, Satish -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15967): https://lists.fd.io/g/vpp-dev/message/15967 Mute This Topic: https://lists.fd.io/mt/72699704/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
