[vpp-dev] feature arc node is NOT hitting
Hi VPP experts, We are facing some strange issue in the feature arc framwork. We have two nodes inserted in "device-input" feature arc, for example, "feature1" and "feature2". We have enabled these features on two interfaces available. "show interface ifc-1 features" and "show interface ifc-2 features" "device-input" feature1 feature2 Above command o/p showing that both features are enabled in the device-input feature arc for both interfaces. However, in the packet processing, we are observing that for one interface, only feature2 node is getting hit, but the packets are NOT going via feature1 node. Any inputs on why this could be happening (or) any hints on how to debug this further on this. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20631): https://lists.fd.io/g/vpp-dev/message/20631 Mute This Topic: https://lists.fd.io/mt/87699901/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP crashing on Cetnos8 server
Hi Ashish / VPP-experts, We are also seeing the same crash with fdio 21.06 version. vec_resize_allocate_memory + 0x285 vlib_put_next_frame + 0xbd Our main-heap size is set to 2G. Is this a known issue (or) any fix that is available for this. Any inputs will be helpful. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20501): https://lists.fd.io/g/vpp-dev/message/20501 Mute This Topic: https://lists.fd.io/mt/83553727/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP 20.05 problems related to memory allocations -- possible memory leak?
Hi Klemant/Elias/Vpp-Experts, We are also seeing the same crash with fdio 21.06 version. vec_resize_allocate_memory + 0x285 vlib_put_next_frame + 0xbd Our main-heap size is set to 2G. Is this a known issue (or) any fix that is available for this. Any inputs will be helpful. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20500): https://lists.fd.io/g/vpp-dev/message/20500 Mute This Topic: https://lists.fd.io/mt/77479819/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Does memif zero-copy work ?
Thanks MJ for the quick reply. Will try this and check. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20360): https://lists.fd.io/g/vpp-dev/message/20360 Mute This Topic: https://lists.fd.io/mt/86509719/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Does memif zero-copy work ?
Hi VPP experts, We are using memif for communication between an APP and VPP. We recently tried moving to zero-copy memif based on the example icmp_responder-zero-copy-slave. This example shows the code for the APP being in SLAVE mode. So, to go along with this, we made VPP as memif-MASTER and APP as SLAVE. But, the VPP memif create interface code is restricting the VPP to be memif-SLAVE when we use Zero-copy. So, few questions of interest are: - Does VPP latest release support zero-copy-memif with VPP being MASTER ? - Is there any example zero-copy code for APP side being in MASTER Please let us know. Current example only shows APP being SLAVE for zero-copy. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20358): https://lists.fd.io/g/vpp-dev/message/20358 Mute This Topic: https://lists.fd.io/mt/86509719/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Are INPUT and PROCESS nodes considered for CPU util calculations?
Hi VPP Experts, We have an issue at hand, where we are seeing non-uniform CPU utilizations showing up for workers from "show threads". We are doing lot of work as part of some timer node, which periodically does maintenance of flows. However, this maintenance activity, which is run as part of this INPUT node ( timer node), is not considered into CPU utilizations. Basically, this timer node does not have any vector as its input. So, is this load not considered in the worker CPU utilization ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20337): https://lists.fd.io/g/vpp-dev/message/20337 Mute This Topic: https://lists.fd.io/mt/86414816/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] vlib_get_plugin_symbol returning null after upgrading to 21.06 version
This works fine for me where the code is fully written by us. But, I have few functions which are kind of generated routines from a third party library, where we do not have control to add this __clib_export. Is there any way to work around this case? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20227): https://lists.fd.io/g/vpp-dev/message/20227 Mute This Topic: https://lists.fd.io/mt/85945383/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] vlib_get_plugin_symbol returning null after upgrading to 21.06 version
Thanks Mohammed and Dave for the quick inputs. Adding __clib_export solved my issue. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20226): https://lists.fd.io/g/vpp-dev/message/20226 Mute This Topic: https://lists.fd.io/mt/85945383/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] vlib_get_plugin_symbol returning null after upgrading to 21.06 version
Hi, We are trying to upgrade to fdio.2106 version from a previous version. After upgrading to 21.06 version with our custom plugins, we are seeing that vlib_get_plugin_symbol() to resolve a symbol from another plugin is always returning null. This was working fine in the earlier version. I cross checked that load_one_plugin() is successful in loading all the plugins, before we call the vlib_get_plugin_symbol(). Any pointers on what could be happening here ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20220): https://lists.fd.io/g/vpp-dev/message/20220 Mute This Topic: https://lists.fd.io/mt/85945383/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Making ACL plugin thread safe
Hi VPP-experts, We are exploring ways to make the ACL plugin thread safe and need your inputs on the same. As of today, most of the ACL apis (acl-add/del and acl-lookup-context-add/del) take thread barrier lock, due to which synchronization is automatically achieved for the workers, who are trying to access the ACL plugin for lookups. To eliminate this thread barrier lock, we are thinking few changes to begin with. 1) Make acl-list pools and acl-lookup-context pools as fixed length pools (to the max values that we can expect), so that element pointers will not change during resizing and multiple threads can do pool_put and pool_get at the same time with basic lock at the pool level (only during pool_get and pool_put). 2) acl_lookup_hash is a bihash. So, I assume multiple readers can access it when writer is trying to modify. Basically, its already a thread safe data structure and does not need any further changes. Anything else we need to worry about to make this thread safe ? Appreciate any inputs on this. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20211): https://lists.fd.io/g/vpp-dev/message/20211 Mute This Topic: https://lists.fd.io/mt/85919320/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] memif_tx_burst() is failing with INVALID_ARG after migrating to fdio-21.06 version
Hi, We had our software based on 20.05 vpp version. Recently, we moved to fdio-2106 version and could compile it successfully. However, memif_tx_burst() function is failing to send message to VPP with error code "INVALID_ARGUMENT". This was working in 20.05 version without any issues. I see that Jakub Grajciar changed in this area quite a bit. One of the change is: commit 1421748e3cd98d7355b1a1db283803a571569927 Author: Jakub Grajciar Date: Thu Jan 14 13:23:48 2021 +0100 libmemif: set data offset for memif buffer Update descriptor offset based on data pointer in memif_buffer_t. Slave only, master will not modify the descriptor. Type: feature Are these changes not backward compatible ? What changes we need to make to get this working in 21.06 version. Jakub Grajciar, Appreciate If you can give your inputs on this. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20178): https://lists.fd.io/g/vpp-dev/message/20178 Mute This Topic: https://lists.fd.io/mt/85787566/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP plugins in C++
Hi , Is it possible to develop a vpp plugin in C++ (or) does VPP framework pose any restrictions ? Appreciate your inputs on this. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20051): https://lists.fd.io/g/vpp-dev/message/20051 Mute This Topic: https://lists.fd.io/mt/85275658/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] ARP learning via workers and thread barrier locking
Hi All, As per my current understanding of the code for ARP learning: 1. ARP is learned on a VPP worker 2. Send to main thread via rpc, by adding to the rcp queue 3. main thread picks up this from rpc queue 4. main thread takes thread barrier lock and updates the ARP table In the step4, we are taking a thread barrier lock for ARP learing ( might be due to the ARP module being non-thread-safe). We are observing that this is resulting into worker threads being locked up for few milli/microseconds, causing some tail drops. Since, ARP learning is quite a common activity in the network, isnt this causing issue for VPP in general. We are seeing tail drops in VPP workers during ARP spikes. Any ideas on how to handle this (or) making ARP learning as lock-less, so that it can be done from any worker ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20017): https://lists.fd.io/g/vpp-dev/message/20017 Mute This Topic: https://lists.fd.io/mt/85132512/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Issues adding ACL with binary_api
vl_api_acl_add_replace_reply_t_handler: *73* : ACL index: 0 *73* pointing to the error code VNET_API_ERROR_INVALID_ARGUMENT. It gives some clue. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20016): https://lists.fd.io/g/vpp-dev/message/20016 Mute This Topic: https://lists.fd.io/mt/85093357/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] vpp main thread crashed at mspace_put
Thanks Sudhir for the quick inputs. Will check if we have any leaks. Also, as a side question, How are you finding leaks in VPP ? Are you using address sanitizer (or) something else? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19833): https://lists.fd.io/g/vpp-dev/message/19833 Mute This Topic: https://lists.fd.io/mt/81600282/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP getting hanged after consecutive VAPI requests
Hi Chinmaya, We are also facing similar issue and want to check with you if you are able to find a fix for this. Appreciate any inputs regarding the same. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19831): https://lists.fd.io/g/vpp-dev/message/19831 Mute This Topic: https://lists.fd.io/mt/75986184/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] vpp main thread crashed at mspace_put
Hi Sudhir, Were you able to find a solution to this problem. We are also facing similar issue. Any inputs would be helpful. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19830): https://lists.fd.io/g/vpp-dev/message/19830 Mute This Topic: https://lists.fd.io/mt/81600282/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] : Worker Thread Deadlock Detected from vl_api_clnt_node
Thanks Rajith for quick reply. At this point of time, we do not have a luxury of upgrading to 20.09 in a quick time. Hi Dave, Do you have any specific commit in mind that can help us to eliminate this crash. Really appreciate your inputs on this. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19733): https://lists.fd.io/g/vpp-dev/message/19733 Mute This Topic: https://lists.fd.io/mt/78681829/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] : Worker Thread Deadlock Detected from vl_api_clnt_node
Hi Rajith / Dave, We are on fdio.2005 version and see this same crash when we are doing packet tracing. Is there any specific patch/commit that improves the situation of this locking. If so, Can you please let us know the commit info. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19730): https://lists.fd.io/g/vpp-dev/message/19730 Mute This Topic: https://lists.fd.io/mt/78681829/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] side effects of not using vlib_get_next_frame and put_frame in packet processing
Hi ,
A basic doubt on the VPP packet processing loop.
Typically, we see the packet processing loop as:
while(n_left_from > 0)
{
get_next_frame
quad-loop
single-loop
put_next_frame
}
>From the code, I could not fully understand if it is mandatory to have the
>outerloop or not.
Even when I do directly do the quad-loop and single-loop and not use the
get-next and put-next-frames, it is working fine.
What is the side effect of not using get-next-frame and put-next-frame macros
and avoiding the outer loop.
Any inputs on this would be really helpful.
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19017): https://lists.fd.io/g/vpp-dev/message/19017
Mute This Topic: https://lists.fd.io/mt/81599954/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Bug in vl_socket_read_ready() function
Hi , While using AddressSanitizer with VPP, it detected use-after-poison in vl_socket_read_ready() function. Closely looking at the code, it seems that this is happening due to the following code. if (vec_len (msg_buffer) > msgbuf_len) /* There are some fragments left. Shrink the msg_buffer to simplify logic. */ vec_delete (msg_buffer, msgbuf_len, 0); < we are deleting the socket_main.input_buffer, which is initialized in api_init, but not again after this delete. So, when we try to use this buffer again, its detected by libASAN. I see that this change is done as part of following commit. -- commit 6a6af6ea1a77b5818e717047b5d01251ef6d024a Author: Vratko Polak Date: Mon Oct 7 14:52:53 2019 +0200 api: comment, simplify and fix api socket read The function vl_socket_read_ready did contain some comments already, but as they stated, the logic has to be tricky to cover multiple cases. - Can someone please comment, if I am missing anything here ? I see that this code is present even in the latest stable code base. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18714): https://lists.fd.io/g/vpp-dev/message/18714 Mute This Topic: https://lists.fd.io/mt/80528036/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Has anyone able to run AddressSanitizer with VPP, successfully ?
Thanks Ben for the info. My code is based on fdio.2005 The commit you mentioned to exclude is not present in this repo. Do you see any issues with fdio.2005 version for ASAN to run clean ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18642): https://lists.fd.io/g/vpp-dev/message/18642 Mute This Topic: https://lists.fd.io/mt/80279607/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Has anyone able to run AddressSanitizer with VPP, successfully ?
Hi All, Has anyone able to run the AddressSanitizer with basic VPP-APIs being run on VPP. As soon a VPP-API message lands onto the VPP, it is crashing with ASAN detecting an error. Due to this, we are not able to use ASAN for any other functional testing. The ASAN error is happening inside VPP infra and hence not able to get any hints on why this would happen. ==17226==ERROR: AddressSanitizer: use-after-poison on address 0x7fffc519aa5f at pc 0x76e9d67d bp 0x7fffb70318b0 sp 0x7fffb7031058 WRITE of size 61 at 0x7fffc519aa5f thread T0 (vpp_main) #0 0x76e9d67c (/lib64/libasan.so.5+0x9367c) #1 0x76b90682 in vl_socket_read_ready /fdio.2005/src/vlibmemory/socket_api.c:240 #2 0x73358cf6 in linux_epoll_input_inline /fdio.2005/src/vlib/unix/input.c:322 #3 0x7335959c in linux_epoll_input /fdio.2005/src/vlib/unix/input.c:372 #4 0x7326b690 in dispatch_node /fdio.2005/src/vlib/main.c:1271 #5 0x7327073b in vlib_main_or_worker_loop /fdio.2005/src/vlib/main.c:1958 #6 0x732724f7 in vlib_main_loop /fdio.2005/src/vlib/main.c:2141 #7 0x73273bdb in vlib_main /fdio.2005/src/vlib/main.c:2387 #8 0x7335e22b in thread0 /fdio.2005/src/vlib/unix/main.c:673 #9 0x727010b7 (/fdio.2005/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.20.05.1+0xc90b7) -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18633): https://lists.fd.io/g/vpp-dev/message/18633 Mute This Topic: https://lists.fd.io/mt/80279607/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP crash after enabling address sanitizer
Hi Ben, Thanks for the quick response. I don't see a possibility of this getting called from any other thread. As seen in the ASAN error o/p , its happening in the main_thread only always. WRITE of size 61 at 0x7fffc519aa5f thread T0 ( *vpp_main* ) Any idea what could be happening here. This issue is making the ASAN not useable with VPP. Please share your views on any workarounds for this. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18601): https://lists.fd.io/g/vpp-dev/message/18601 Mute This Topic: https://lists.fd.io/mt/80099837/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP crash after enabling address sanitizer
Hi,
Tried enabling Google address sanitizer in VPP by looking at the following Wiki.
https://fd.io/docs/vpp/master/troubleshooting/sanitizer.html
After this, we are seeing that ASAN is detecting errors in VPP-API infra code
and crashing.
One of the crash is as below.
==17226==ERROR: AddressSanitizer: use-after-poison on address 0x7fffc519aa5f at
pc 0x76e9d67d bp 0x7fffb70318b0 sp 0x7fffb7031058
WRITE of size 61 at 0x7fffc519aa5f thread T0 (vpp_main)
#0 0x76e9d67c (/lib64/libasan.so.5+0x9367c)
#1 0x76b90682 in vl_socket_read_ready
/fdio.2005/src/vlibmemory/socket_api.c:240
#2 0x73358cf6 in linux_epoll_input_inline
/fdio.2005/src/vlib/unix/input.c:322
#3 0x7335959c in linux_epoll_input /fdio.2005/src/vlib/unix/input.c:372
#4 0x7326b690 in dispatch_node /fdio.2005/src/vlib/main.c:1271
#5 0x7327073b in vlib_main_or_worker_loop /fdio.2005/src/vlib/main.c:1958
#6 0x732724f7 in vlib_main_loop /fdio.2005/src/vlib/main.c:2141
#7 0x73273bdb in vlib_main /fdio.2005/src/vlib/main.c:2387
#8 0x7335e22b in thread0 /fdio.2005/src/vlib/unix/main.c:673
#9 0x727010b7
(/fdio.2005/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.20.05.1+0xc90b7)
The problematic is pointing to
clib_error_t *
vl_socket_read_ready (clib_file_t * uf)
{
...
/* Ignore unprocessed_input for now, n describes input_buffer for now. */
*n = read (uf->file_descriptor, socket_main.input_buffer, vec_len
(socket_main.input_buffer)); << This is the line ASAN
is pointing to.*
}
Is this a known issue (or) anything else we need to change to get it going with
ASAN build.
Please let us know.
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18590): https://lists.fd.io/g/vpp-dev/message/18590
Mute This Topic: https://lists.fd.io/mt/80099837/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Packet corruption over memif ( with libmemif )
Hi, We are facing a strange issue in packet transfer over memif channel. We have a memif connection setup with VPP as master and App as Client. When the App sends a 64 messages continuously to VPP, the response from VPP is getting corrupted. We are doing coloring of the message bytes on both sides and we could see that some stomping of memory is happening at lower layers. One main point here is: Each packet that is getting transferred over memif is 64K of size. We have made memif interface MTU as 65535 to accomodate this. When we reduce each message size, then the crash is NOT happening within 64 messages. So, basically, we are suspecting that some ring memory is getting congested due to which this issue is happening. Fyi, following is the memif connection details for this. interface memif0/0 remote-name "App" remote-interface "app-conn1" socket-id 0 id 0 mode ip flags admin-up connected listener-fd 46 conn-fd 45 num-s2m-rings 1 num-m2s-rings 1 buffer-size 0 num-regions 2 region 0 size 524544 fd 47 region 1 size 268435456 fd 48 master-to-slave ring 0: region 0 offset 262272 ring-size 16384 int-fd 50 head 16486 tail 102 flags 0x interrupts 100 slave-to-master ring 0: region 0 offset 0 ring-size 16384 int-fd 49 head 204 tail 204 flags 0x0001 interrupts 0 Any hints on what could be happening here and how to debug this problem. Really Appreciate any help here, as we are stuck with this problem for the last two weeks and not able to debug what the issue is. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18250): https://lists.fd.io/g/vpp-dev/message/18250 Mute This Topic: https://lists.fd.io/mt/78708204/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] ip full reassembly - is_custom_app field broken ?
Hi Klement, Do you have the similar changes for ip6_full_reassembly.c as well. If so, Can you pls pass on the diffs. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17967): https://lists.fd.io/g/vpp-dev/message/17967 Mute This Topic: https://lists.fd.io/mt/76705450/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] 2 MB of message over memif channel
Thanks Damjan / MJ for the quick inputs. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17966): https://lists.fd.io/g/vpp-dev/message/17966 Mute This Topic: https://lists.fd.io/mt/78074016/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP committers: VPP PTL vote
+1 -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17573): https://lists.fd.io/g/vpp-dev/message/17573 Mute This Topic: https://lists.fd.io/mt/77123394/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] ip full reassembly - is_custom_app field broken ?
Thanks a lot Klement. Will give it a try with this patch and get back to you. (Hopefully , by early next week ). -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17365): https://lists.fd.io/g/vpp-dev/message/17365 Mute This Topic: https://lists.fd.io/mt/76705450/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] ip full reassembly - is_custom_app field broken ?
Ok. I think, required changes would be more complex / involved than I initially imagined. Can you please make these changes. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17346): https://lists.fd.io/g/vpp-dev/message/17346 Mute This Topic: https://lists.fd.io/mt/76705450/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] ip full reassembly - is_custom_app field broken ?
Thanks Klement for the quick response. I can make the changes you suggested. But, one major doubt I have is on the HANDOFF scenario. Let's say, as part of custom-reasm-node, if the packet is decided to be handed off to another thread, then the next_node is ALWAYS getting set as "ip4-full-reassembly-handoff". Shouldn't it go to custom-reasm-node ? This also needs to be decided based on "is_custom_app" , right ? Please let me know your inputs. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17344): https://lists.fd.io/g/vpp-dev/message/17344 Mute This Topic: https://lists.fd.io/mt/76705450/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] ip full reassembly - is_custom_app field broken ?
[Edited Message Follows]
Hi,
Looking at the ip4 full reassembly graph nodes for the purpose of punting the
fragments and getting the reassembled packets from our custom graph node.
However, from the code it seems that, is_custom_app flag is effectively
disabled.
I see that the node function is always getting called with is_custom_app =
false.
VLIB_NODE_FN (ip4_full_reass_node) (vlib_main_t * vm,
vlib_node_runtime_t * node,
vlib_frame_t * frame)
{
return ip4_full_reass_inline (vm, node, frame, false /* is_feature */ ,
false /* is_custom_app */ ); always called with FALSE
}
We also observed that sv_reassembly has custom_app functionality. But, we need
fully reassembled packet, and hence looking at full_reassembly functionality.
Is there a way to use this is_custom_app flag functionality in full_reassembly
?
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17340): https://lists.fd.io/g/vpp-dev/message/17340
Mute This Topic: https://lists.fd.io/mt/76705450/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] ip full reassembly - is_custom_app field broken ?
Hi,
Looking at the ip4 full reassembly graph nodes for the purpose of punting the
fragments and getting the reassembled packets from our custom graph node.
However, from the code it seems that, is_custom_app flag is effectively
disabled.
I see that the node function is always getting called with is_custom_app =
false.
VLIB_NODE_FN (ip4_full_reass_node) (vlib_main_t * vm,
vlib_node_runtime_t * node,
vlib_frame_t * frame)
{
return ip4_full_reass_inline (vm, node, frame, false /* is_feature */ ,
false /* is_custom_app */ ); always called with FALSE
}
Is there a way to use this is_custom_app flag functionality ?
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17340): https://lists.fd.io/g/vpp-dev/message/17340
Mute This Topic: https://lists.fd.io/mt/76705450/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Fib entry add programatically
Hi, Is there an easier way to add a static route programatically from a VPP plugin. I know that this can be done easily via vppctl CLI or via vppapi infrastructure. However, if we want to achieve the same from a VPP plugin, is there a way we can use CLI/vppapi infra directly rather than calling the internal functions that these will call ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17282): https://lists.fd.io/g/vpp-dev/message/17282 Mute This Topic: https://lists.fd.io/mt/76390006/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] ARP resolution from non-connected IP
Neale /Ben, We already have a static route to reach the peer. Even after that, we are getting this error. And this error is as per the VPP's ARP code. It's not allowing the ARP-REQUEST from a non-connected source IP. If we dont know allow this, I am not 100% sure how multihop-BGP peering kind of scenarios really work. In MH-BGP, the peer will be running on a loopback address and will try to reach a peer loopback. In this scenario, the ARP request can have the BGP-loopback address as the source address. How will it really work ? Any inputs pls. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17277): https://lists.fd.io/g/vpp-dev/message/17277 Mute This Topic: https://lists.fd.io/mt/76285682/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] ARP resolution from non-connected IP
Hi Dave, Thanks for the quick inputs. I tried enabling the proxy-arp in both the following ways. But, still I am seeing the drop as below. CLIs tried: = set ip arp proxy 203.0.113.2 - 203.0.113.4 set interface proxy-arp intfc1 enable trace command o/p indicating the ARP drop: == 05:36:30:174070: arp-input request, type ethernet/IP4, address size 6/4 be:bb:1d:60:20:81/1.1.1.1 -> 00:00:00:00:00:00/203.0.113.2 05:36:30:174079: error-drop arp-input: IP4 source address not local to subnet Anything wrong I am doing here ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17261): https://lists.fd.io/g/vpp-dev/message/17261 Mute This Topic: https://lists.fd.io/mt/76285682/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] ARP resolution from non-connected IP
Hi, Have a query on the ARP resolution. We have a Router and VPP-box connected. We are trying to do some peering from router’s loopback IP. This loopback IP is not in the connected subnet-range. Due to this, router is initiating an ARP request with src as non-connected-IP in the payload. VPP is dropping this ARP request saying “IP4 source address not local to subnet”. In linux and in cisco routers, we have options to allow ARP requests from non-connected subnets. Is there any workaround in VPP to allow this ARP request. Please let us know. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17259): https://lists.fd.io/g/vpp-dev/message/17259 Mute This Topic: https://lists.fd.io/mt/76285682/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Query on Feature arc for tapping all IP packets after reassembly
Thanks a lot Matt for the quick inputs. We are interested in packets destined for our interfaces only. Hence, I will try with ip4-local. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17215): https://lists.fd.io/g/vpp-dev/message/17215 Mute This Topic: https://lists.fd.io/mt/76144894/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Query on Feature arc for tapping all IP packets after reassembly
Hi,
We have a query on one of the requirements we have.
1. We would like to tap all the ip4 packets into our custom graph node.
2. But, we want to tap the ip packets only after reassembly is completed, if
fragments are received.
I was looking at ip-local feature arc, if this works for our requirement or
not. But, I am not 100% sure, if the reassembly restriction will be met by
registering our feature in ip4-local feature arc as below.
VNET_FEATURE_INIT (our_custom_feature, static) =
{
.arc_name = "ip4-local",
.node_name = "our-custom-node",
.runs_before = VNET_FEATURES("ip4-local-end-of-arc"),
};
If this does not work, can you please let us know any pointers on how to
achieve this.
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17195): https://lists.fd.io/g/vpp-dev/message/17195
Mute This Topic: https://lists.fd.io/mt/76144894/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Crash in ACL code while applying rulematch
Hi Andrew, We could not try the same test with the change you suggested. We will try it for sure in this week. Regarding our scenario for example: we have 4 workers. All the 4 workers can create ACLs and also can process data. During high load of data and create ACLs, we are seeing this crash. There are NO delete of ACLs done in the test. As I mentioned earlier, after putting a lock around the "add acl" and "rulematch acl" code, issue got solved. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16995): https://lists.fd.io/g/vpp-dev/message/16995 Mute This Topic: https://lists.fd.io/mt/75527176/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Crash in ACL code while applying rulematch
Hi Andrew, Thanks for your suggestions on this. We did few more tests today, and here are some observations. Please let us know your thoughts on this. 1) We added a lock on the datapath also while doing rulematch using acl apis. The same lock is getting acquired while doing acl adds from other threads. With this, we are NOT seeing the crash. So, one question we have is: Even when the main thread does control plane operations on ACLs ( like add / dels), the other workers will not be able to safely use the ACL apis to do rule matching ? Taking this lock in the datapath is surely not desirable, as it will hit performance. Please let us know your thoughts on this. 2) When we observed the crash earlier, we were doing only ACL adds and pumping data So, the contention is between adding new ACLs and using the ACLs during rulematch. There are delete of ACLs at all. This is one more observation we want to share with you, if that can give some hints for the crash. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16989): https://lists.fd.io/g/vpp-dev/message/16989 Mute This Topic: https://lists.fd.io/mt/75527176/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Crash in ACL code while applying rulematch
Hi Andrew, Thanks for the quick response. All the activity that we are performing with ACLs, like , add and delete of ACLs from multiple workers, we are doing by taking a lock. This will ensure that no two workers will be able to operate on ACLs at the same time. Is there any specific restriction that ACL operations needs to be performed from main thread only ? Is this restriction coming due to thread synchronization or some data structures / heaps. Please let us know. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16975): https://lists.fd.io/g/vpp-dev/message/16975 Mute This Topic: https://lists.fd.io/mt/75527176/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Crash in ACL code while applying rulematch
Hi , We are seeing a crash when we are trying to do ACL rulematch within a lookup context. #0 0x7f7786498ca0 in multi_acl_match_get_applied_ace_index (am=0x7f77866caa00 , is_ip6=0, match=0x7f7791fff610) at src/plugins/acl/public_inlines.h:625 #1 0x7f7786498f27 in hash_multi_acl_match_5tuple (p_acl_main=0x7f77866caa00 , lc_index=6463, pkt_5tuple=0x7f7791fff610, is_ip6=0, action=0x7f7791fff6b4 "", acl_pos_p=0x7f7791fff6a0, acl_match_p=0x7f7791fff6a4, rule_match_p=0x7f7791fff6a8, trace_bitmap=0x7f7791fff6ac) at src/plugins/acl/public_inlines.h:652 #2 0x7f778649912a in acl_plugin_match_5tuple_inline (p_acl_main=0x7f77866caa00 , lc_index=6463, pkt_5tuple=0x7f7791fff610, is_ip6=0, r_action=0x7f7791fff6b4 "", r_acl_pos_p=0x7f7791fff6a0, r_acl_match_p=0x7f7791fff6a4, r_rule_match_p=0x7f7791fff6a8, trace_bitmap=0x7f7791fff6ac) at src/plugins/acl/public_inlines.h:690 The code in public_inlines.h:625 corresponds to applied_hash_ace_entry_t *pae = vec_elt_at_index ((*applied_hash_aces), curr_index); Here the curr_index is 0 and (*applied_hash_aces)[0] is 0x0. Due to this, we are getting SIGSEGV. The lookup context and the ACL rules within that seems to be fine. What could be the reason this *applied_hash_aces can go wrong. One more thing is this crash is happening only when we add acl rules and lookup contexts from multiple workers. When we move to single worker, this crash is not happening. Though we are adding/deleting the ACLs/lookup contexts from different workers, we are protecting them using sping locks. Any help on this please. We are struggling to find what's the reason for this crash as this is happening only at high number of ACLs / calls. The acl rules are around 10K. Please let us know of any hints to look around. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16973): https://lists.fd.io/g/vpp-dev/message/16973 Mute This Topic: https://lists.fd.io/mt/75527176/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Query on Performance with pools
Hi , We are seeing following observations when we do performance tests with our plugin/graphnodes. 1. When we have 1 subscriber session, our custom-graph-node takes 1.35e1 cycles ( 60 vec/call ) 2. When we have 200 subscriber sessions, the same custom-graph-node takes 1.2e2 cycles ( same 60 vec/call ). The difference between 1.35e1 and 1.2e2 seems pretty high, and, with more subscribers in picture, this will degrade performance a lot. Also, this clearly shows that our graph node implementation is not doing enough prefetching. Our psuedo-code is something like this: Quad loop for buffers: 1.prefetech buffer headers/ buffer data 2.get session-top object from pool 3.get session-leaf-1 from session-top object 4. get session-leaf-2 from session-leaf-1 object We are only doing prefetching in step1 alone. How we can do pre-fetching in the case of steps2,3,4 ? When we have objects in a pool, which are scattered across, how to take leverage of data-prefetching. Any pointers / hints on how to handle this please. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16905): https://lists.fd.io/g/vpp-dev/message/16905 Mute This Topic: https://lists.fd.io/mt/75356233/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Query on Inner packet Fragmentation and Reassembly
Thanks a lot Klement for this quick info. This will serve our purpose. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16853): https://lists.fd.io/g/vpp-dev/message/16853 Mute This Topic: https://lists.fd.io/mt/75234196/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Query on Inner packet Fragmentation and Reassembly
Hi , We have a use case, where we receive packets in a tunnel, and the inner packet may be fragments. If we want to reassemble the inner fragments and get one single packet, does VPP already have a framework that has this functionality. If it's already there, we can make use of it. I saw MAP plugin, but I am not able to see the place where it reassembles ipv4 fragments and outputs one single packet. Any inputs/examples please. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16850): https://lists.fd.io/g/vpp-dev/message/16850 Mute This Topic: https://lists.fd.io/mt/75234196/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Crash in vlib_add_trace with multi worker mode
Hi , We are seeing a crash while doing add_trace for a vlib_buffer in our graph node. #0 0x74ee0feb in raise () from /lib64/libc.so.6 #1 0x74ecb5c1 in abort () from /lib64/libc.so.6 #2 0x0040831c in os_panic () at /fdio/src/fdio.1810/src/vpp/vnet/main.c:368 #3 0x75f28f2f in debugger () at /fdio/src/fdio.1810/src/vppinfra/error.c:84 #4 0x75f2936a in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7fffb2083920 "%s:%d (%s) assertion `%s' fails") at /fdio/src/fdio.1810/src/vppinfra/error.c:143 #5 0x7fffb2035d79 in vlib_validate_trace (tm=0x7fffbaccfd58, b=0x2aaab447d940) at /fdio/src/fdio.1810/src/vlib/trace_funcs.h:53 #6 0x7fffb2035ec8 in vlib_add_trace (vm=0x7fffbaccfb40, r=0x7fffbb430f40, b=0x2aaab447d940, n_data_bytes=36) The assert is pointing to following line in the code. ASSERT (!pool_is_free_index (tm->trace_buffer_pool, vlib_buffer_get_trace_index (b))); One specific point with this crash is: This crash is happening only when we have multiple workers in VPP. Following is the scenario this crash is happening. 1. Packet comes to worker-1 from nic card 2. graph node on worker-1 hands off the packet to worker-2 3. In worker-2, while processing the packet, we are trying to add the trace using vlib_add_trace and this crash occurs. The trace buffer within vlib_buffer_t, is it specific to worker ? If so, what happens, when the buffer gets hand off to another worker ? Can this cause the above crash ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16613): https://lists.fd.io/g/vpp-dev/message/16613 Mute This Topic: https://lists.fd.io/mt/74625018/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] vlib buffer getting overwritten over memif
[Edited Message Follows] Hi , Seeing some strange issue when using memif to transfer some messages from client to VPP. Each of the message that we want to transfer from client to VPP is around 64 KB. When the client sends messages ( each of size 64 KB) to VPP in quick succession, we are seeing that the buffer is getting overwritten and hence VPP handing of the message is going haywire. Couple of questions in this regard:, 1) When client sends message to VPP using memif, the vlib_buffer_t corresponding to this will still point to the shared memory only ? (or) does memif-input node copies separate vlib_buffer_t. If it;s pointing to the shared memory, then, if client modifies the shared memory during this time, the vlib-buffer will get impacted, right ? 2) I am also not sure , if our ring sizes are ok to handle messages of size 64KB . Here is the show memif output. Can you please let us know if the sizes seem fine. interface memif0/1 remote-name "Client" remote-interface "memif_conn" socket-id 0 id 1 mode ip flags admin-up connected listener-fd 40 conn-fd 41 num-s2m-rings 1 num-m2s-rings 1 buffer-size 0 num-regions 2 region 0 size 65792 fd 44 region 1 size 264241152 fd 47 master-to-slave ring 0: region 0 offset 32896 ring-size 2048 int-fd 53 head 5349 tail 3301 flags 0x interrupts 3098 slave-to-master ring 0: region 0 offset 0 ring-size 2048 int-fd 50 head 6602 tail 6602 flags 0x0001 interrupts 0 3) One more important point in this issue is: we are seeing this issue only when we are doing thread handoff of the received message from one worker to another worker. When we handoff a message that is received from memif to another worker, do we need to clone the buffer ? Basically, is there anything that is localized to per worker in this scenario ? Any inputs on this would really help us. Thanks, -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16471): https://lists.fd.io/g/vpp-dev/message/16471 Mute This Topic: https://lists.fd.io/mt/74397409/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] vlib buffer getting overwritten over memif
Hi , Seeing some strange issue when using memif to transfer some messages from client to VPP. Each of the message that we want to transfer from client to VPP is around 64 KB. When the client sends messages ( each of size 64 KB) to VPP in quick succession, we are seeing that the buffer is getting overwritten and hence VPP handing of the message is going haywire. Couple of questions in this regard:, 1) When client sends message to VPP using memif, the vlib_buffer_t corresponding to this will still point to the shared memory only ? (or) does memif-input node copies separate vlib_buffer_t. If it;s pointing to the shared memory, then, if client modifies the shared memory during this time, the vlib-buffer will get impacted, right ? 2) I am also not sure , if our ring sizes are ok to handle messages of size 64KB . Here is the show memif output. Can you please let us know if the sizes seem fine. interface memif0/1 remote-name "Client" remote-interface "memif_conn" socket-id 0 id 1 mode ip flags admin-up connected listener-fd 40 conn-fd 41 num-s2m-rings 1 num-m2s-rings 1 buffer-size 0 num-regions 2 region 0 size 65792 fd 44 region 1 size 264241152 fd 47 master-to-slave ring 0: region 0 offset 32896 ring-size 2048 int-fd 53 head 5349 tail 3301 flags 0x interrupts 3098 slave-to-master ring 0: region 0 offset 0 ring-size 2048 int-fd 50 head 6602 tail 6602 flags 0x0001 interrupts 0 Any inputs on this would really help us. Thanks, -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16471): https://lists.fd.io/g/vpp-dev/message/16471 Mute This Topic: https://lists.fd.io/mt/74397409/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] can the pointer of a used-pool-element change before it's put back ?
Hi, We are seeing some issue in our plugin that seems to caused by the change of pointer for a pool element. The scenario is as below. Can you please let us know , if this can really occur. 1. We have multiple workers 2. We have one global pool of custom-structures ( this is a non-fixed pool) 3. This global pool is protected by a lock for addition and deletion. 4. However, it is not protected for reading. So, all workers refer read entries from this pool by using pool_elt_at_index() 5. All the workers keep on adding and deleting elements in this pool by taking lock. 6. What we are seeing is, the pointer returned by pool_elt_at_index(global_pool, index) is getting changed in between and causing some issue in our logic. Couple of questions we have is : 1 ) If a pool element is getting used (not-free-element), Until that element is put back again into the pool, can we assume that the address of the element remain same and will not get chagned by pool resizing. 2) Having pools that spans across threads will cause any issues like this while reading the elements ( though we protect the pool for add/del of elements using a spin lock) ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16254): https://lists.fd.io/g/vpp-dev/message/16254 Mute This Topic: https://lists.fd.io/mt/74034826/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] accessing pool entries in gdb
Having some issue while accessing entries of a pool in GDB. I have a pool of some structures. custom_struct *pool; This custom_struct has alignment with 64 byte cache_line. now, I have added 3 entries in this pool. The code seems to be working fine in adding/deleting/traversing this pool using pool_elt_at_index. However, If i use gdb to view the elements using pool[0], pool[1] and pool[2], they are giving some invalid entries. The first entry pool[0] seem to be fine, but the next entry onwards are showing invalid entries in GDB. Is there anything wrong i am doing in gdb while traversing the pools ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16232): https://lists.fd.io/g/vpp-dev/message/16232 Mute This Topic: https://lists.fd.io/mt/73982726/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Storing vlib buffer index for later processing
Hi , We are having a scenario to support as below and we would like to know what we are doing here is correct or not. 1. Our graph node receives a frame with TWO buffers 2. Graph node decides to process FIRST buffer and enqueues the packet to a next-node. 3. Graph node decides to store the SECOND buffer index to process it after 10 sec. Hence, it stores the second buffer index to a vector for later processing. 4. Graph node returns a value of 2 ( Is this correct ? or do we need to return 1, since one buffer we want to consume at a later point of time ?) 5. After 10 sec, we process the vector of buffer indices and process them to send to another node. 6. While sending to another node, we create a new frame and add this buffer index to that. Basically, when a graph node want to consume a buffer for a later point of time, is there anything specific we need to do ? Or is it just enough to store the buffer index for a later use. Any inputs on this would help us in our design/implementation. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16096): https://lists.fd.io/g/vpp-dev/message/16096 Mute This Topic: https://lists.fd.io/mt/73086166/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] exporting plugin functions/symbols to global space
Hi, I remember seeing some questions regarding this in fdio mailing list, but not able to find it now. We have a plugin which has 10 functions that are needed by 3 other plugins ( which are owned by us ). As of now, we are doing vlib_get_plugin_symbol() and getting the symbols. But , I remember there is a way each plugin can export the functions to global space, so that other plugins dont need to do dlsym and get it directly from global symbol table. Is there any framework /macros in VPP to export all or specific symbols to global level from a plugin, so that all the other plugins can access it without doing dlsym. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16065): https://lists.fd.io/g/vpp-dev/message/16065 Mute This Topic: https://lists.fd.io/mt/73008778/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] query on pool memory usage
Hi, Is there anyway we can get info what are all vectors and pools each plugin is using and their corresponding memory usage. We are chasing some memory leak issue and if VPP has a builtin way of getting this info, we want to take leverage of it. If VPP does not have any built-in way, do you guys think, its better to go through each pool and print pool_bytes and entries in use/free to track this. Is this the way to go (or) any other tricks in this area ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16034): https://lists.fd.io/g/vpp-dev/message/16034 Mute This Topic: https://lists.fd.io/mt/72873560/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] naive questions on VPP memory usage ( does it ever come down )
Thanks Dave and Damjan for quick inputs. This helps in understanding the memory mgmt in vpp. Inline to your questions. 1. Can you capture "show memory main-heapsh” before and after? We are currently in fdio.1810 and i am not seeing this option. 2. One more quick question. With this model of memory mgmt, the traditional linux tools like pmap/smap will not give a clear view of exact memory usage and if any leaks are there. I have tried memory-trace on feature along with "show memory verbose" However, I am not able to map a particular allocation to a particular deallocation to see if a memory leak is present or not. "show memory verbose" It captures only the program counter of allocation/deallocations, but not the pointer at which the allocation /deallocation happened. Due to this, I am not able to find if there is a leak. Is there any trick to map allocation/deallocations one-on-one to find leaks from "show memory" output. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15971): https://lists.fd.io/g/vpp-dev/message/15971 Mute This Topic: https://lists.fd.io/mt/72699704/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] naive questions on VPP memory usage ( does it ever come down )
Few questions on VPP memory usage. 1. Using pmap -p i am collecting the total memory usage of vpp process at the beginning of my test. ( it is X KB ) 2. I ran test for few hours which will obviously have lot of pool_get/pool_put 3. Collected the same same pmap output and the memory usage grown to X+Y 4. Cleared all my sessions which ideally should have cleared all my allocations. 5. I still see the memory usage being stuck at X+Y I see that only pool_free is doing unmap of the memory, whereas pool_put is not doing so. Could this be the reason why I am seeing memory not coming down. Thanks & Regards, Satish -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15967): https://lists.fd.io/g/vpp-dev/message/15967 Mute This Topic: https://lists.fd.io/mt/72699704/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Creating a new frame using set of buffers
Hi, We have a module that need to receive the packets and buffer some of the packets until some duration and process them after a specific duration and send them out. To achieve this, we are currently noting down all the buffer indices in a frame that needs to be buffered and storing them in a vector and not enqueuing these buffers to any next node. However, after a certain duration, while processing these set of buffer indices, we want to create a frame out of these buffers and want to enqueue to different set of nodes using enqueue_x4 or similar function. How to create a new frame and add buffers to the new frame ? Is there any example that we can refer for this. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15926): https://lists.fd.io/g/vpp-dev/message/15926 Mute This Topic: https://lists.fd.io/mt/72645551/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] How to get source node of a buffer
Thanks Neale for the quick inputs. I am planning to add src_node index in the buffer metadata ( opaque2) for this purpose. Hoping that this helps in our debugging. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15850): https://lists.fd.io/g/vpp-dev/message/15850 Mute This Topic: https://lists.fd.io/mt/72511630/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] How to get source node of a buffer
Hi , Is there any way to find the source node of a buffer. Basically, I want to know from which node this buffer came. I understand that each graph node shall have a design which is independant of the source node. However, the source node information may be useful in case of debugging some crashes while processing a buffer. Any inputs on this pls. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15846): https://lists.fd.io/g/vpp-dev/message/15846 Mute This Topic: https://lists.fd.io/mt/72511630/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Is there any Linux FD to poll for VCL message
Hi Florin, Over the weekend, I went through the document that you mentioned and it gave me a good overview. Thanks for pointing to that doc. However, my task of integrating mqs->epfd into our main dispatch loop still seems to be needing more finer details of the code. With this respect, I have following few queries. It would be great if you can help with this. If this path poses more effort/risk in integration, we may want to switch to host interface approach ( which has its own issues though :( ) 1) Terminology mapping. In the VCL application framework, is the following mapping a correct understanding (with some open items) ? TCP-Application => app with a specific client index Worker => thread in the tcp-application Session => tcp end point mq in worker => mapped to ? mq_evt_conn => mapped to ? 2) Howcome vppcom_epoll_create is creating a new session. Bit confusing here. 3) Is the TCP server listen_fd also can be integrated into our application dispatch loop ? 4) How to get hold of mqs->epfd to put into our dispatcher? I see that the vcl_worker_t is inside vcl_private.c and .h which are not accessible by the application directly. Also, i dont see a direct need of having vcl_test_worker* framework in our app. Hope this is fine. 5) why can't the TCP application directly use vcl_worker_t data structure ? 6) Is there a sample tcp server example that has the server implementation with linux epoll instead of vppcom_epoll system. 7) Do you think this approach of integration will be more complex and hence do you suggest us moving to LDP (or) host interface approach ? Please let us know. Thanks in advance for your time. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15788): https://lists.fd.io/g/vpp-dev/message/15788 Mute This Topic: https://lists.fd.io/mt/71899986/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Is there any Linux FD to poll for VCL message
Hi Florin, Thanks a lot for the detailed explanation. This kind of gives an overview of this area, which really helps in our integration. Just one more question: We are planning to remove the concept of vcl worker in our worker, as our app is a single threaded app and will not be multi-threaded at any point of time in future as well. Hope this is doable and does not pose any restrictions in any of the VCL s/w layers. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15780): https://lists.fd.io/g/vpp-dev/message/15780 Mute This Topic: https://lists.fd.io/mt/71899986/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Is there any Linux FD to poll for VCL message
Hi Florin,
Thank you very much for the inputs.
These are very difficult to understand unless we go through the code in detail.
Today, Whole day, I was trying to follow your instructions and get this working
by looking at the code as well.
However, I am not fully successful.
Before going further, I would like to get my understanding clear, so that it
will form basis for my debugging.
Here are couple of questions:
1)
==
The message queue used between vpp and a vcl worker can do both mutex/condvar
and eventfd notifications. The former is the default but you can switch to
eventfds by adding to vcl.conf "use-mq-eventfd”. You can then use
vppcom_worker_mqs_epfd to retrieve a vcl worker's epoll fd (it’s an epoll fd
for historic reasons) which you should be able to nest into your own linux
epoll fd.
==
The message queues between VPP and VCL can be present either in "shared memory"
(or) "memfd segments".
For eventfd to work, the queues need to be present in the "memfd segments".
Is this correct understanding ?
2)
==
Note that you’ll also need to force memfd segments for vpp’s message queues,
i.e., session { evt_qs_memfd_seg }, and use the socket transport for binary
api, i.e., in vpp’s startup.conf add "socksvr { /path/to/api.sock }" and in
vcl.conf "api-socket-name /path/to/api.sock”.
==
I didnt understand the reason for moving the binary api to sockets.
Is this due to shm/memfd wont be used at the same time ?
3)
In a nut shell:
VCL-APP VPP
VCL-APP <---Binary Api via LinuxDomain
Sockets--> VPP
We will have two api clients with this model. One is shared memory client and
other is a socket client.
Is my understanding correct ?
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#15771): https://lists.fd.io/g/vpp-dev/message/15771
Mute This Topic: https://lists.fd.io/mt/71899986/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Is there any Linux FD to poll for VCL message
Hi , We have a TCP application trying integrate with VPP-VCL framework. Our application has its own dispatch loop with epoll and we would like to know if VCL framework has any linux fd ( like an eventfd for the entire svm message queue ) that we can add into our epoll to poll for VCL session messages. Once we get an asynchronous indication that a message has arrived in the VCL svm message queue, we can call vppcom_epoll_wait() function to read the messages for sessions and handle them accordingly. Any inputs on how we can achieve this? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15750): https://lists.fd.io/g/vpp-dev/message/15750 Mute This Topic: https://lists.fd.io/mt/71899986/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP with FRR Bring-up - tap interface enable causing crash
which fdio release version you are using? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15712): https://lists.fd.io/g/vpp-dev/message/15712 Mute This Topic: https://lists.fd.io/mt/71738703/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Trying my luck with BGP peering
Hi , I think, this topic has been discussed in few of the earlier questions, but still I could not find a one that gave a workable solution in totality. We are trying to write a BGP application which hosts BGP peering sessions, using VPP as a dataplane entity. We tried following few options with issues mentioned below. I am also attaching an image that we are trying to achieve. 1) Tried to clone VPP sandbox code for vpp router plugin. But, vppsb repo does not seem to be available in the following path anymore. https://gerrit.fd.io/r/vppsb Is there a place where I can get this code ? Or Is this obsoleted one ? 2) If vppsb is obsoleted, is there an alternative that works well for this. We tried punting approach by doing a. set punt tcp b. ip punt redirect add rx Interface/7/0.100 via 1.1.1.1 tapcli-0 However, here, we are seeing an ARP being sent from vpp-veth-ifc to host-veth-interface and it is not able to resolve the ARP. The redirect is getting triggered from VPP, but it is not reaching the host due to the ARP issue. Is there anything that we need to do for this ? Also, in this approach, the reverse path from vpp-host to the external box, how will it work ? The punt redirect config that we added, will it work for reverse path as well ? Please help us here as we are kind of stuck with what approach we need to take ? ( we want to avoid the VPP-TCP-stack for the time being, as it will be more effort to integrate our BGP app with VPP-TCP-stack,i.e VCL framework ). -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15703): https://lists.fd.io/g/vpp-dev/message/15703 Mute This Topic: https://lists.fd.io/mt/71831881/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] how to catch packets on a specific TCP port
Hi, We are looking for a way to get all the packets on a particular TCP port to reach our plugin. For UDP ports, we could see that udp_register_dst_port() can be used to get packets on a specific UDP port. But, for TCP, we could not see any registration function like this. Any pointers on this pls ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15690): https://lists.fd.io/g/vpp-dev/message/15690 Mute This Topic: https://lists.fd.io/mt/71746978/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Can i increase the size of vlib buffer opaque2
We are currently using opaque2 which has 10 uint32. Can i increase this size to 30 uint32s. What kind of impact/restrictions we have for this opaque2 metadata sizes. Please let us know. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15676): https://lists.fd.io/g/vpp-dev/message/15676 Mute This Topic: https://lists.fd.io/mt/71703548/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VCL client connect error
Hi,
We are seeing following error when we try to connect to VPP via VCL test client.
Is this a known issue?
startup file that we are using on VPP:
unix {
nodaemon
log /tmp/vpp.log
full-coredump
cli-listen /run/vpp/cli.sock
# gid vpp
}
#api-segment {
# gid vpp
#}
Error:
==
./vcl_test_client 127.0.0.1 12344
VCL<1273>: using default heapsize 268435456 (0x1000)
VCL<1273>: allocated VCL heap = 0x7fe8a141f010, size 268435456 (0x1000)
VCL<1273>: using default configuration.
vppcom_connect_to_vpp:577: vcl: VCL<1273>: app (vcl_test_client) connecting
to VPP api (/vpe-api)...
vl_map_shmem:637: region init fail
connect_to_vlib_internal:410: vl_client_api map rv -2
vppcom_connect_to_vpp:583: VCL<1273>: app (vcl_test_client) connect failed!
vppcom_app_create:724: VCL<1273>: ERROR: couldn't connect to VPP!
ERROR when calling vppcom_app_create(): Connection refused
ERROR: vppcom_app_create() failed (errno = 111)!
Any inputs on this please ?
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#15429): https://lists.fd.io/g/vpp-dev/message/15429
Mute This Topic: https://lists.fd.io/mt/71351013/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] two VPP api clients from same process
Hi, Have a question on number of API clients that we can have a from a client process. Can't we have more than one VPP-API-Clients from a single process ? >From the code, I observed that we have api_main in the client code which is >used for communication towards VPP, which may restrict us to have only one api >client from a process. Is this understanding correct ? Also, any enhancements done to have more than one client from same process. Please let us know. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15357): https://lists.fd.io/g/vpp-dev/message/15357 Mute This Topic: https://lists.fd.io/mt/71023411/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Is VppCom suitable for this scenario
Hi Florin, Thank you very much for quick inputs. I have gone through your youtube video from kubecon and it cleared lot of my doubts. You presented it in a very clear manner. As you rightly pointed out, VppCom will be a overhead for our use case. All we need is just a shared memory communication to send and receive bigger messages. Memif was not a candidate for this, since it will pose message size restrictions upto 64K. In this case, what framework we can use to send/recv messages from VPP workers across shared memory. Can we use SVM queues directly and get the message into our custom VPP plugin and process it ( in case of VPP receiving message from control plane app ) Any example code that already does this ? If so, can you please point this to us. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15073): https://lists.fd.io/g/vpp-dev/message/15073 Mute This Topic: https://lists.fd.io/mt/69461619/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Is VppCom suitable for this scenario
Hi , Have one basic doubt on applicability of VppCom library for a use case that we have as below. Use Case with following requirements: 1. control plane app needs to communicate with different VPP worker threads 2. control plane app may need to send messages to vpp workers with message size that can span upto a max size of 1 MB. 3. control plane app needs to have different VppCom channels with each worker For the above scenario, is VppCom a suitable infrastructure ? Using memif causes max size limit at 64KB. Hence, we are thinking about alternatives. Please share your inputs on this. ( Also, is there any documentation on VppCom library ? ) -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15056): https://lists.fd.io/g/vpp-dev/message/15056 Mute This Topic: https://lists.fd.io/mt/69461619/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP Crash while programming DPO from a VPP Worker
Hi Neale, Thanks a lot for the inputs. It solved our issue. As a side question: Is there any mechanism for the main thread to send the result to the worker thread? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15003): https://lists.fd.io/g/vpp-dev/message/15003 Mute This Topic: https://lists.fd.io/mt/69328025/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP Crash while programming DPO from a VPP Worker
Hi,
We are seeing a VPP crash (ASSERT) with following stack trace, when we try to
create a DPO from within a VPP worker.
The below crash clear shows that DPO creation is restricted only to the main
thread, since the ASSERT is making sure that the thread id is 0.
* Backtrace of "Thread workers 1" (0xd11ca700)
* Frame 00: /lib64/libc.so.6(gsignal+0x10b) [0x7f3b860e1feb]
* Frame 01: /lib64/libc.so.6(abort+0x12b) [0x7f3b860cc5c1]
* Frame 02: /usr/bin/vpp() [0x407e83]
* Frame 03:
/lib64/libvlib.so.18.10(vlib_worker_thread_barrier_sync_int+0x2c8)
[0x7f3b877d6298]
* Frame 04: /lib64/libvnet.so.18.10(dpo_stack+0x3fe) [0x7f3b881176be]
* Frame 05: /lib64/libvnet.so.18.10(+0x5134ad) [0x7f3b8811a4ad]
* Frame 06: /lib64/libvnet.so.18.10(load_balance_multipath_update+0xe8)
[0x7f3b8811cb48]
* Frame 07: /lib64/libvnet.so.18.10(fib_entry_src_mk_lb+0x13e)
[0x7f3b880ff4ee]
* Frame 08: /lib64/libvnet.so.18.10(fib_entry_src_action_install+0xc5)
[0x7f3b880ff7d5]
* Frame 09: /lib64/libvnet.so.18.10(fib_entry_create_special+0x39)
[0x7f3b880fd129]
* Frame 10: /lib64/libvnet.so.18.10(fib_table_entry_special_dpo_add+0xd5)
[0x7f3b880f4215]
void
vlib_worker_thread_barrier_sync_int (vlib_main_t * vm)
{
ASSERT (vlib_get_thread_index () == 0); < Crash is happening here
}
So, now the question is:
Can't we create any DPO from a worker thread ?
If not, is there any work around to do this.
Please let us know any inputs on this.
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#14994): https://lists.fd.io/g/vpp-dev/message/14994
Mute This Topic: https://lists.fd.io/mt/69328025/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] vector size issue with Multi threaded VPP
Retrying my luck once again. Hi, We are facing a strange problem when we moved from single worker to multi worker VPP. Our application plugin registers for a specific udp port and gets the packets from udp_local node. In Single Worker VPP: We see that our app-plugin-node is receiving frames of bigger vector size ( most of the times 256 ), which is resulting is good performance figures. In Multi Worker VPP: ( 2 workers ) == Though we have two workers here, we are receiving udp packets from one worker only ( based on our rx-placement ). With this model, we thought our performance would be atleast equal to the single worker mode, but we are seeing degraded performance. In this mode, we see that, our app-plugin-node is receiving frames with vector size as 1 ( most of the times ), due to which, we are seeing degraded performance figures, we believe. We checked the underlying udp-local node, and observed that, it is always receiving frames of vector sizes 256 and 99.99% of these frames belong to our plugin related udp port. Hence, they all are sent to our app-plugin-node. But these packets are received by our node with vector size as 1, in each frame, which is causing an issue. We are using exact same test tool to pump the traffic in both single and multi thread VPP models. Is there anything that we need to do to avoid vector sizes as 1 in this case ? Also, any reason why this behavior is seen only when we have multi workers, even though the other worker is not really tampering with any of the packets that we are dealing here. Any inputs on this would really help us. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14812): https://lists.fd.io/g/vpp-dev/message/14812 Mute This Topic: https://lists.fd.io/mt/66962512/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Multi-Worker VPP Performance Issue
Hi, We are facing a strange problem when we moved from single worker to multi worker VPP. Our application plugin registers for a specific udp port and gets the packets from udp_local node. In Single Worker VPP: We see that our app-plugin-node is receiving frames of bigger vector size ( most of the times 256 ), which is resulting is good performance figures. In Multi Worker VPP: ( 2 workers ) == Though we have two workers here, we are receiving udp packets from one worker only ( based on our rx-placement ). With this model, we thought our performance would be atleast equal to the single worker mode, but we are seeing degraded performance. In this mode, we see that, our app-plugin-node is receiving frames with vector size as 1 ( most of the times ), due to which, we are seeing degraded performance figures, we believe. We checked the underlying udp-local node, and observed that, it is always receiving frames of vector sizes 256 and 99.99% of these frames belong to our plugin related udp port. Hence, they all are sent to our app-plugin-node. But these packets are received by our node with vector size as 1, in each frame, which is causing an issue. We are using exact same test tool to pump the traffic in both single and multi thread VPP models. Is there anything that we need to do to avoid vector sizes as 1 in this case ? Also, any reason why this behavior is seen only when we have multi workers, even though the other worker is not really tampering with any of the packets that we are dealing here. Any inputs on this would really help us. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14688): https://lists.fd.io/g/vpp-dev/message/14688 Mute This Topic: https://lists.fd.io/mt/61942008/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Performance and Vector Size Issue - with multi worker VPP
[Edited Message Follows] Hi, We are facing a strange problem when we moved from single worker to multi worker VPP. Our application plugin registers for a specific udp port and gets the packets from udp_local node. In Single Worker VPP: We see that our app-plugin-node is receiving frames of bigger vector size ( most of the times 256 ), which is resulting is good performance figures. In Multi Worker VPP: ( 2 workers ) == Though we have two workers here, we are receiving udp packets from one worker only ( based on our rx-placement ). With this model, we thought our performance would be atleast equal to the single worker mode, but we are seeing degraded performance. In this mode, we see that, our app-plugin-node is receiving frames with vector size as 1 ( most of the times ), due to which, we are seeing degraded performance figures, we believe. We checked the underlying udp-local node, and observed that, it is always receiving frames of vector sizes 256 and 99.99% of these frames belong to our plugin related udp port. Hence, they all are sent to our app-plugin-node. But these packets are received by our node with vector size as 1, in each frame, which is causing an issue. We are using exact same test tool to pump the traffic in both single and multi thread VPP models. Is there anything that we need to do to avoid vector sizes as 1 in this case ? Also, any reason why this behavior is seen only when we have multi workers, even though the other worker is not really tampering with any of the packets that we are dealing here. Any inputs on this would really help us. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14669): https://lists.fd.io/g/vpp-dev/message/14669 Mute This Topic: https://lists.fd.io/mt/61598908/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Performance and Vector Size Issue - with multi worker VPP
Hi, We are facing a strange problem when we moved from single worker to multi worker VPP. Our application plugin registers for a specific udp port and gets the packets from udp_local node. In Single Worker VPP: We see that our app-plugin-node is receiving frames of bigger vector size ( most of the times 256 ), which is resulting is good performance figures. In Multi Worker VPP: ( 2 workers ) == Though we have two workers here, we are receiving udp packets from one worker only ( based on our rx-placement ). With this model, we thought our performance would be atleast equal to the single worker mode, but we are seeing degraded performance. In this mode, we see that out app-plugin-node is receiving frames with vector size as 1 ( most of the times ), due to which, we are seeing degraded performance figures, we believe. We checked the underlying udp-local node, and observed that, it is always receiving frames of vector sizes 256 and 99.99% of these frames belong to our plugin. Hence, they all are sent to our app-plugin-node. But these packets are received by our node with vector size as 1, in each frame. Is there anything that we need to do to avoid vector sizes as 1 ? Also, any reason why this behavior is seen only when we have multi workers, even though the other worker is not really tampering with any of the packets that we are dealing here. Any inputs on this would really help us. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14669): https://lists.fd.io/g/vpp-dev/message/14669 Mute This Topic: https://lists.fd.io/mt/61598908/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Crash in vlib_worker_thread_barrier_sync_int
Hi ,
We are seeing crash in vlib_worker_thread_barrier_sync_int() function as soon
as we send a CLI command to VPP.
We are sending CLI command to VPP via a script, which may not be waiting enough
for VPP initialization to settle.
I see that the crash is happening in the following piece of code.
while (*vlib_worker_threads->workers_at_barrier != count)
{
if ((now = vlib_time_now (vm)) > deadline)
{
fformat (stderr, "%s: worker thread deadlock\n", __FUNCTION__);
os_panic (); Here
}
>From what I see in the code, not all worker threads got a chance to increment
>workers_at_barrier and hence not able to hit the condition within the
>deadline ( 1 sec) and hence the crash. Is my understanding correct ?
If so, are we sending the CLI command too early to VPP ( before all workers
initialized ), and hence this crash ?
Any inputs on this would really help us.
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#14605): https://lists.fd.io/g/vpp-dev/message/14605
Mute This Topic: https://lists.fd.io/mt/59188069/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] fd to poll in VPP-API-CLIENT with no RX thread
Thank you Dave. Will import this patch and try. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14597): https://lists.fd.io/g/vpp-dev/message/14597 Mute This Topic: https://lists.fd.io/mt/55586364/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] fd to poll in VPP-API-CLIENT with no RX thread
Thanks a lot Dave for offering the help. The change would be of great help for us. Please let us know, once you have a patch, so that we can selectively take that patch and import. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14591): https://lists.fd.io/g/vpp-dev/message/14591 Mute This Topic: https://lists.fd.io/mt/55586364/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] fd to poll in VPP-API-CLIENT with no RX thread
Thanks Ole for the response. We will weigh between performance of "unix domain sockets" vs "lock contention with rx thread" approached and go accordingly. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14585): https://lists.fd.io/g/vpp-dev/message/14585 Mute This Topic: https://lists.fd.io/mt/55586364/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] fd to poll in VPP-API-CLIENT with no RX thread
Thanks Dave for the quick reply. Main problem we are seeing with having RX thread is: 1) Our application sends VPP-API-request in main thread and stores the Request and context information in a list 2) If the response comes in the RX thread, we need to access the context information list in this thread which may force us to have locks across these data structures to clear contention between main and RX thread. We are worried abt performance degradation in control plane due to these locks when we have thousands of FIB entries ( in the order of 50K ). -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14581): https://lists.fd.io/g/vpp-dev/message/14581 Mute This Topic: https://lists.fd.io/mt/55586364/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] fd to poll in VPP-API-CLIENT with no RX thread
Hi, We are trying to come up with VPP-API-CLIENT module that has no special RX thread. As we dont have no RX thread, we are planning to read messages from shared memory from our main dispatcher. To do this, we want to poll on an FD to check if any message arrived in the SHM queue. I was looking for some eventfd over this shared memory which indicates a message arrival. However, I dont see an eventfd getting flagged by VPP. This will make the client purely asynchronous without RX thread. I see some eventfd code in src/vnet/session/* code. But this does not seems to be used by general VPP API server code. Any inputs on how to achieve this pls ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14577): https://lists.fd.io/g/vpp-dev/message/14577 Mute This Topic: https://lists.fd.io/mt/55586364/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Thread handoff node - Is it mandatory to have
Thanks Damjon for the suggestion. As suggested, I have tried processing the buffers and send to another thread and another node-in same thread in the same function using two apis you mentioned. Though it is surely achievable, the code looks lot cluttery and non-uniform now. Hence, I am thinking to have a different graph node which takes care of thread handoff. This will make my all input nodes not worry about how to handoff, but they will just set the next node to handoff node. Though I will have one extra node in this approach, it seems cleaner to maintain/less prone to make mistakes in the long run. Hope this is fine. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14374): https://lists.fd.io/g/vpp-dev/message/14374 Mute This Topic: https://lists.fd.io/mt/37501109/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] accessing object pools from multiple threads
Thanks Andrew for the quick reply. Can you please point me to the code where the barrier lock is getting applied for acls. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14051): https://lists.fd.io/g/vpp-dev/message/14051 Mute This Topic: https://lists.fd.io/mt/34285184/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] accessing object pools from multiple threads
Hi , I have a basic question on how object pools are accessed by multiple threads in VPP. Do we have any locks underlying that safe gaurd the object pools ? For example: In acl_main_t, we have a pool of acl lists as below. *acl_list_t *acls; /* Pool of ACLs */* This list will be populated by vpp main_thread for add/del. Also, this list will be accessed by worker thread during packet processing. So, is this list guarded by any lock at lower layers for synchronizing the access between main thread and worker thread ? Thanks & Regards, Satish -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14049): https://lists.fd.io/g/vpp-dev/message/14049 Mute This Topic: https://lists.fd.io/mt/34285184/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] non-zero counter values in unrelated graph nodes
Hi , We are facing a strange issue, which we are not able to debug even after spending a good amount of time. We are seeing "show node counters" displaying very high number of values all of a sudden for few unrelated nodes like "null-node" and "vmxnet3-input". The values are also keep on changing ( going down and up ) on multiple runs of "show node counters". This kind of indicates that counters are not getting incrementing by those graph nodes as such, but the issue is due to the memory pertaining to "counters". Are there any known issues in this area by any chance ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13958): https://lists.fd.io/g/vpp-dev/message/13958 Mute This Topic: https://lists.fd.io/mt/34105342/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP API client with no rx pthread
Hi , We are trying to develop a VPP API client which needs synchronous reply handling. Hence, we were thinking of NOT having a separate pthread for receiving the response from VPP. We are planning to use no_rx_pthread version of connect api. Is there any example code to receive and handle the response synchronously. I see all the examples are using separate pthread for receiving. Any input on this will be of great help. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13952): https://lists.fd.io/g/vpp-dev/message/13952 Mute This Topic: https://lists.fd.io/mt/34101834/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] memif impacting other workers
Hi, We are seeing one strange issue when we use memif. We have vpp with 2 workers. This VPP is running as memif-server. worker-0 ==> rx-placement with memif worker-1 ==> rx-placement with an IP interface ( dpdk based ) As soon as we have the memif client connected to VPP, we see degradation in performance on the worker-1, which is not at all related to memif message processing. mpstat command is showing the dpdk-cores going to way less than 100% ( to around 60% ), as soon as we have the memif channels get established. idle cpu is increasing. Any inputs on why we are seeing this issue ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13862): https://lists.fd.io/g/vpp-dev/message/13862 Mute This Topic: https://lists.fd.io/mt/33060971/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] table id for sub interfaces
Thanks john for the quick inputs. That helps. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13857): https://lists.fd.io/g/vpp-dev/message/13857 Mute This Topic: https://lists.fd.io/mt/33050491/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] table id for sub interfaces
Hi, Is there anyway I can associate a VPP sub-interface to a particular ip-fib table id ? I could able to associate a physical interface to a table id, but not a sub interface. I did not find a CLI option to do this. Any inputs pls. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13851): https://lists.fd.io/g/vpp-dev/message/13851 Mute This Topic: https://lists.fd.io/mt/33050491/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] lockless architecture
Hi Dave, I totally agree with you that my email was an over-simplification of a bigger issue for sure. What we are trying to see is: can a single thread handle both control plane and data plane activities for a particular application object ( ex: a flow object) , so that, a need of lock can be avoided. ( Making sure that control plane messages and data plane packets landing on the same thread is a different issue.) -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13725): https://lists.fd.io/g/vpp-dev/message/13725 Mute This Topic: https://lists.fd.io/mt/32853245/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] lockless architecture
Hi , Need some inputs on the lock less architecture with multi-threaded VPP system. >From the experiences we had so far, taking a lock ( whatsoever small time the >lock is taken for ) degrades the performance. Hence, trying to see if we have any feasibility of lockless architecture in VPP. We have gone through this in deep but seeing few issues on implementing this. 1) The vnet library and other core plugins like acls/classifiers does not seem to be thread-safe. The data structures in these does not seem be be thread specific, and hence applications from different threads may not able to modify them without acquiring a lock. This poses an issue. 2) I think, this is the reason, the control plane operations via vpp-api are handled by single main thread. However, this model will have an issue of main thread needing to take a write-lock when it is trying to modify any data via control plane. Worker threads during this time have to wait for this lock causing performance degradation. So, if both controlplane and data plane operations are performed by same thread (let's say by a specific worker), then we have an issue of underlying vnet layer not being thread-safe. If go with the approach of main thread and worker thread handling control and data plane activities respectively, then we need to take lock which may degrade performance. Any viable approach that can avoid both of these issues ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13723): https://lists.fd.io/g/vpp-dev/message/13723 Mute This Topic: https://lists.fd.io/mt/32853245/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Is VPP vnet policer thread safe
Hi , Would like to know if vnet policer is thread safe or not. To be more specific: can multiple WORKER threads (not main thread) add vnet policers at the same time without any issues ? In contrast to the "main" thread adding the policers via vpp-api, the "worker" threads add the policers due to some constraints in our architecture. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13679): https://lists.fd.io/g/vpp-dev/message/13679 Mute This Topic: https://lists.fd.io/mt/32782772/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] tracing packets of a specific 5-tuple
Thanks Dave for the inputs. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13673): https://lists.fd.io/g/vpp-dev/message/13673 Mute This Topic: https://lists.fd.io/mt/32744859/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] tracing packets of a specific 5-tuple
Hi, We have a custom graph node that receives packets from udp graph node and we would like to trace packets pertaining to a specific 5-tuple flow from our custom graph node point onwards. Does following steps really possible to achieve for this functionality ? 1. Make the custom-graph node to enable tracing by using VLIB_BUFFER_TRACE_TRAJECTORY_INIT 2. add CLI option to give the 5-tuple data 3. Based on the give 5-tuple data, store this in the custom graph node 4. compare the packet with the 5-tuple data, and mark the buffer to be traced. 5. now onwards , all the other graph nodes will trace it. Also, what % of impact on the data throughput with VPP tracing ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13671): https://lists.fd.io/g/vpp-dev/message/13671 Mute This Topic: https://lists.fd.io/mt/32744859/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] MemIf interface selection for TX traffic
Hi ,
Have a question on the Memif TX interface selection.
I have VPP with 2 worker threads connecting to an external application.
Have two memif interfaces, so that each thread can make use of one Memif
interface dedicately.
show memif
interface memif0/0
remote-name "remoteApp"
remote-interface "remoteIf"
socket-id 0 id 0 mode ip
flags admin-up connected
listener-fd 39 conn-fd 38
num-s2m-rings 1 num-m2s-rings 1 buffer-size 0 num-regions 2
region 0 size 65792 fd 43
region 1 size 264241152 fd 44
master-to-slave ring 0:
region 0 offset 32896 ring-size 2048 int-fd 52
head 5303 tail 3255 flags 0x interrupts 1434
slave-to-master ring 0:
region 0 offset 0 ring-size 2048 int-fd 48
head 6480 tail 6480 flags 0x0001 interrupts 0
interface memif0/1
remote-name "remoteApp"
remote-interface "remoteIf"
socket-id 0 id 1 mode ip
flags admin-up connected
listener-fd 39 conn-fd 40
num-s2m-rings 1 num-m2s-rings 1 buffer-size 0 num-regions 2
region 0 size 65792 fd 46
region 1 size 264241152 fd 50
master-to-slave ring 0:
region 0 offset 32896 ring-size 2048 int-fd 57
head 2048 tail 0 flags 0x interrupts 0
slave-to-master ring 0:
region 0 offset 0 ring-size 2048 int-fd 54
head 0 tail 0 flags 0x0001 interrupts 0
Added IP addresses on these two interfaces as below.
memif0/0 ==> 1.1.1.2
memif0/1 ==> 2.2.2.2
Correspondingly, on the remote APP, I created IP endpoints like 1.1.1.1 and
2.2.2.1
When I try to ping 1.1.1.1 from VPP, I expect that it will go memif0/0, but it
is going memif0/1
So, my question here is, what is the logic here for memif device selection for
TX traffic.
I see the code as below:
VNET_DEVICE_CLASS_TX_FN (memif_device_class) (vlib_main_t * vm,
vlib_node_runtime_t * node,
vlib_frame_t * frame)
{
memif_main_t *nm = _main;
vnet_interface_output_runtime_t *rund = (void *) node->runtime_data;
memif_if_t *mif = pool_elt_at_index (nm->interfaces, rund->dev_instance);
> memif interface selection
I didnt quite understand the above logic and not sure if this will be helpful
to tie up a thread/IP interface to a particular memif interface or not.
Is there a way, where the TX traffic can be tied to a particular memif
Interface ?
--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#13406): https://lists.fd.io/g/vpp-dev/message/13406
Mute This Topic: https://lists.fd.io/mt/32270034/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] PerfMon Plugin Issue
Hi Dave, Thanks for the reply. I am still seeing the same issue. Here is what I am doing. 1) Created 2 VPP instances 2) Created Memif interfaces on each of them and assigned IP addresses to them 3) Able to ping the IPs from one another successfully. 4) On VPP-1, started ping of interface on VPP-2 instance as below DBGvpp# ping 1.1.1.2 repeat 10 interval 0.0001 5) While the ping is running on VPP-1 CLI, ran following commands on VPP-2 CLI DBGvpp# set pmc cpu-cycles Start collection for 1 events, wait 1.00 seconds DEADMAN: collection still running... Data collection complete... 6) Ran show pmc to check the data DBGvpp# show pmc Data collection in progress... This is always showing it as "Data collection in progress". I am using official version of fdio.1904 in this activity. Am I missing anything obvious ? -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13294): https://lists.fd.io/g/vpp-dev/message/13294 Mute This Topic: https://lists.fd.io/mt/32064990/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] PerfMon Plugin Issue
Hi , I am trying to use the perfmon plugin for cpu-cycles event. I am doing a continuous ping over a memif channel. But, I am seeing that the command always return saying "Data Collection in progress". DBGvpp# set pmc cpu-cycles Start collection for 1 events, wait 1.00 seconds DEADMAN: collection still running... Data collection complete... DBGvpp# show pmc Data collection in progress... DBGvpp# Is there anything else I need to enable to use this perfmon plugin (or) any known issue with recent commits. -- Thanks & Regards, Murthy -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13286): https://lists.fd.io/g/vpp-dev/message/13286 Mute This Topic: https://lists.fd.io/mt/32064990/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
