Re: [vpp-dev] naive questions on VPP memory usage ( does it ever come down )

[Satya Murthy]

Thanks Dave and Damjan for quick inputs.
This helps in understanding the memory mgmt in vpp.

Inline to your questions.
1. Can you capture "show memory main-heapsh” before and after?
We are currently in fdio.1810 and i am not seeing this option.

2. One more quick question.
With this model of memory mgmt, the traditional linux tools like pmap/smap will 
not give a clear view of exact memory usage and if any leaks are there.
I have tried memory-trace on feature along with "show memory verbose"

However, I am not able to map a particular allocation to a particular 
deallocation to see if a memory leak is present or not.
"show memory verbose" It captures only the program counter of 
allocation/deallocations, but not the pointer at which the allocation 
/deallocation happened.
Due to this, I am not able to find if there is a leak.

Is there any trick to map allocation/deallocations one-on-one to find leaks 
from "show memory" output.

--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15971): https://lists.fd.io/g/vpp-dev/message/15971
Mute This Topic: https://lists.fd.io/mt/72699704/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Coverity run FAILED as of 2020-04-01 14:00:25 UTC

[Noreply Jenkins]

Coverity run failed today.

Current number of outstanding issues are 5
Newly detected: 0
Eliminated: 0
More details can be found at  
https://scan.coverity.com/projects/fd-io-vpp/view_defects
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15970): https://lists.fd.io/g/vpp-dev/message/15970
Mute This Topic: https://lists.fd.io/mt/72701000/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] naive questions on VPP memory usage ( does it ever come down )

[Damjan Marion via Lists.Fd.Io]

Can you capture "show memory main-heapsh” before and after?

Will show you exact number of pages mapped.
I.e.

vpp# show memory main-heap
Thread 0 vpp_main
  virtual memory start 0x7f13c03ae000, size 1048640k, 262160 pages, page size 4k
numa 1: 45220 pages, 180880k
not mapped: 216940 pages, 867760k
  total: 1.00G, used: 43.19M, free: 980.88M, trimmable: 978.81M

This means that 45220 pages (180MB) out of 262160 (1G of 4K pages) is mapped.
New page is dynamically allocated by kernel 1st time we write a byte into its 
memory address.
In VPP page is never freed, even if we free all allocations which hold that 
page, kernel still thinks it is used by us
and VPP memory allocator will reuse it.

Effect of this is that if you allocate 800Mb out of heap, touch at least one 
byte at each page, and then you free that allocation then 
freed memory will be available for other allocations, but from kernel 
perspective 800MB of pages will stay in use for a VPP lifetime ...

— 
Damjan




> On 1 Apr 2020, at 14:58, Satya Murthy  wrote:
> 
> Few questions on VPP memory usage.
> 
> 1. Using pmap -p  i am collecting the total memory usage of vpp 
> process at the beginning of my test. ( it is X KB )
> 2. I ran test for few hours which will obviously have lot of pool_get/pool_put
> 3. Collected the same same pmap output and the memory usage grown to X+Y 
> 4. Cleared all my sessions which ideally should have cleared all my 
> allocations.
> 5. I still see the memory usage being stuck at X+Y
> 
> I see that only pool_free is doing unmap of the memory, whereas pool_put is 
> not doing so.
> Could this be the reason why I am seeing memory not coming down.
> 
> Thanks & Regards,
> Satish
> 
> 
> -- 
> Thanks & Regards,
> Murthy 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15969): https://lists.fd.io/g/vpp-dev/message/15969
Mute This Topic: https://lists.fd.io/mt/72699704/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] naive questions on VPP memory usage ( does it ever come down )

[Dave Barach via Lists.Fd.Io]

Could be. In real life, objects allocated from pools have widely variable 
lifetimes. It would be virtually impossible to unmap less than an entire pool.

If your application would tolerate something akin garbage collection, I suppose 
you could create a fresh pool with a fresh set of indices, and free the 
original.

Personally, I would never go in this direction.

HTH... Dave

From: vpp-dev@lists.fd.io  On Behalf Of Satya Murthy
Sent: Wednesday, April 1, 2020 8:59 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] naive questions on VPP memory usage ( does it ever come down 
)

Few questions on VPP memory usage.

1. Using pmap -p  i am collecting the total memory usage of vpp 
process at the beginning of my test. ( it is X KB )
2. I ran test for few hours which will obviously have lot of pool_get/pool_put
3. Collected the same same pmap output and the memory usage grown to X+Y
4. Cleared all my sessions which ideally should have cleared all my allocations.
5. I still see the memory usage being stuck at X+Y

I see that only pool_free is doing unmap of the memory, whereas pool_put is not 
doing so.
Could this be the reason why I am seeing memory not coming down.

Thanks & Regards,
Satish


--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15968): https://lists.fd.io/g/vpp-dev/message/15968
Mute This Topic: https://lists.fd.io/mt/72699704/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] naive questions on VPP memory usage ( does it ever come down )

[Satya Murthy]

Few questions on VPP memory usage.

1. Using pmap -p  i am collecting the total memory usage of vpp 
process at the beginning of my test. ( it is X KB )
2. I ran test for few hours which will obviously have lot of pool_get/pool_put
3. Collected the same same pmap output and the memory usage grown to X+Y
4. Cleared all my sessions which ideally should have cleared all my allocations.
5. I still see the memory usage being stuck at X+Y

I see that only pool_free is doing unmap of the memory, whereas pool_put is not 
doing so.
Could this be the reason why I am seeing memory not coming down.

Thanks & Regards,
Satish

--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15967): https://lists.fd.io/g/vpp-dev/message/15967
Mute This Topic: https://lists.fd.io/mt/72699704/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Spike in core VPP VSZ

[Damjan Marion via Lists.Fd.Io]

Not sure about 19.08 but i just tried on latest master.
Without dpdk core size is is 1.29G uncompressed and 1.31G with dpdk plugin 
loaded.

— 
Damjan

> On 1 Apr 2020, at 14:09, siddarth rai  wrote:
> 
> Hi,
> I am using VPP version v19.08.1
> I have noticed that the virtual memory size for core VPP has increased a lot 
> since the last VPP I used(v 18.01)
> 
> Earlier it used to be ~2 G, but now its up to ~16 G. This is causing the VPP 
> core-dump files to bloat up.
> When I say core VPP I mean VPP without any plugins. 
> 
> Can anyone tell what changes caused this increase in VPP VSZ ? Is there any 
> way to reduce it ?
> 
> I am attaching pmap output of VPP started with default configuration
> 
> https://pastebin.com/0nKwFyHF 
> 
> Regards,
> Siddarth
> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15966): https://lists.fd.io/g/vpp-dev/message/15966
Mute This Topic: https://lists.fd.io/mt/72698301/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Spike in core VPP VSZ

[siddarth rai]

Hi,
I am using VPP version v19.08.1
I have noticed that the virtual memory size for core VPP has increased a
lot since the last VPP I used(v 18.01)

Earlier it used to be ~2 G, but now its up to ~16 G. This is causing the
VPP core-dump files to bloat up.
When I say core VPP I mean VPP *without* any plugins.

Can anyone tell what changes caused this increase in VPP VSZ ? Is there any
way to reduce it ?

I am attaching pmap output of VPP started with default configuration

https://pastebin.com/0nKwFyHF

Regards,
Siddarth
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15965): https://lists.fd.io/g/vpp-dev/message/15965
Mute This Topic: https://lists.fd.io/mt/72698301/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CSIT - VM VHOST tests with rxq>1 failing [VPP-1853]

[Peter Mikus via Lists.Fd.Io]

Hello vpp-dev,

We found issues running CSIT VM VHOST tests. I opened VPP-1853 for tracking.

In case of any questions please contact @csit-dev.

Thank you.

Peter Mikus
Engineer – Software
Cisco Systems Limited
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15964): https://lists.fd.io/g/vpp-dev/message/15964
Mute This Topic: https://lists.fd.io/mt/72695885/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CSIT - VM VHOST tests with qemu mrg_rxbuf=off [VPP-1854]

[Peter Mikus via Lists.Fd.Io]

Hello vpp-dev,

We found issues running CSIT VM VHOST tests. I opened VPP-1854 for tracking.

In case of any questions please contact @csit-dev.

Thank you.

Peter Mikus
Engineer – Software
Cisco Systems Limited
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15963): https://lists.fd.io/g/vpp-dev/message/15963
Mute This Topic: https://lists.fd.io/mt/72695880/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-