[vpp-dev] Crash in VPP 21.06 #vnet
Hi , I am observing some infrequent crash in VPP. I am using VCL for receiving UDP packets in my application. We compiled VPP with DPDP and using Mellanox NIC to receive UDP packets ( the MTU is set to 9000). Attached is the startup.conf file. #0 0x7efd06b8837f in raise () from /lib64/libc.so.6 #1 0x7efd06b72db5 in abort () from /lib64/libc.so.6 #2 0x55ffa1ed1134 in os_exit (code=code@entry=1) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vpp/vnet/main.c:431 #3 0x7efd07de215a in unix_signal_handler (signum=11, si=, uc=) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vlib/unix/main.c:187 #4 #5 0x7efd07b4cbc6 in _mm_storeu_si128 (__B=..., __P=) at /usr/lib/gcc/x86_64-redhat-linux/8/include/emmintrin.h:721 #6 clib_mov16 (src=0x7efcb99f8c00 "`", dst=) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vppinfra/memcpy_avx2.h:65 #7 clib_memcpy_fast_avx2 (n=45, src=0x7efcb99f8c00, dst=) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vppinfra/memcpy_avx2.h:166 #8 clib_memcpy_fast (n=45, src=0x7efcb99f8c00, dst=) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vppinfra/string.h:97 #9 svm_fifo_copy_to_chunk_ma_skx (f=0x7efccb716040, c=0x7ef44e1e2cb0, tail_idx=, src=0x7efcb99f8c00 "`", len=45, last=0x7ef44e000c08) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/svm/svm_fifo.c:53 #10 0x7efd07b49b73 in svm_fifo_copy_to_chunk (last=, len=, src=, tail_idx=, c=, f=0x7efccb716040) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/svm/svm_fifo.c:96 #11 svm_fifo_enqueue_segments (f=0x7efccb716040, segs=segs@entry=0x7efcb99f7fb0, n_segs=n_segs@entry=2, allow_partial=allow_partial@entry=0 '\000') at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/svm/svm_fifo.c:1006 #12 0x7efd0883cd25 in session_enqueue_dgram_connection (s=s@entry=0x7efccb6d3580, hdr=0x7efcb99f8c00, b=0x1005415480, proto=proto@entry=1 '\001', queue_event=queue_event@entry=1 '\001') at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vnet/session/session.c:637 #13 0x7efd0869e089 in udp_connection_enqueue (uc0=0x7efccb671e40, s0=0x7efccb6d3580, hdr0=hdr0@entry=0x7efcb99f8c00, thread_index=thread_index@entry=4, b=b@entry=0x1005415480, queue_event=queue_event@entry=1 '\001', error0=0x7efcb99f80d4) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vnet/udp/udp_input.c:159 #14 0x7efd0869e4d7 in udp46_input_inline (is_ip4=0 '\000', frame=, node=, vm=) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vnet/udp/udp_input.c:311 #15 udp6_input (vm=, node=, frame=) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vnet/udp/udp_input.c:368 #16 0x7efd07d8e172 in dispatch_pending_node (vm=0x7efcca76a840, pending_frame_index=, last_time_stamp=) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vlib/main.c:1024 #17 0x7efd07d8fc6f in vlib_worker_loop (vm=vm@entry=0x7efcca76a840) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vlib/main.c:1649 #18 0x7efd07dc8738 in vlib_worker_thread_fn (arg=) at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vlib/threads.c:1560 #19 0x7efd072efee8 in clib_calljmp () at /usr/src/debug/vpp-21.06.0-4~g2a4131173_dirty.x86_64/src/vppinfra/longjmp.S:123 #20 0x7efcba7fbd20 in ?? () #21 0x7efcc4ff41c5 in eal_thread_loop.cold () from /usr/lib/vpp_plugins/dpdk_plugin.so startup.conf Description: Binary data -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21514): https://lists.fd.io/g/vpp-dev/message/21514 Mute This Topic: https://lists.fd.io/mt/91623364/21656 Mute #vnet:https://lists.fd.io/g/vpp-dev/mutehashtag/vnet Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP 2005 is crashing on stopping the VCL applications #vpp-hoststack
Thanks! Dave
On Tue, Aug 18, 2020 at 9:59 AM Dave Wallace wrote:
> Florin cherry-picked the change and I just merged it.
>
> Thanks,
> -daw-
>
> On 8/17/2020 5:19 PM, Dave Barach via lists.fd.io wrote:
>
> You can press the “cherrypick” button as easily as Florin... Hint...
>
>
>
> *From:* vpp-dev@lists.fd.io *On
> Behalf Of *Raj Kumar
> *Sent:* Monday, August 17, 2020 5:09 PM
> *To:* Ayush Gautam
> *Cc:* Florin Coras ;
> vpp-dev
> *Subject:* Re: [vpp-dev] VPP 2005 is crashing on stopping the VCL
> applications #vpp-hoststack
>
>
>
> Hi Florin,
>
> Can we please have the fix[1] on "stable/2005" branch.
>
>
>
> [1] https://gerrit.fd.io/r/c/vpp/+/28173
>
>
>
> Thanks,
>
> -Raj
>
>
>
> On Wed, Aug 5, 2020 at 7:30 PM Raj Kumar via lists.fd.io gmail@lists.fd.io> wrote:
>
> Hi Florin,
>
> Yes , this[1] fixed the issue.
>
>
>
> thanks,
>
> -Raj
>
>
>
> On Wed, Aug 5, 2020 at 1:57 AM Florin Coras
> wrote:
>
> Hi Raj,
>
>
>
> Does this [1] fix the issue?
>
>
>
> Regards,
>
> Florin
>
>
>
> [1] https://gerrit.fd.io/r/c/vpp/+/28173
>
>
>
> On Aug 4, 2020, at 8:24 AM, Raj Kumar wrote:
>
>
>
> Hi Florin,
>
> I tried vppcom_epoll_wait() on 2 different servers by using a simple
> application ( with only 1 worker thread) . But, still vppcom_epoll_wait()
> is not being timed out if I do not use use-mq-eventfd .
>
> Here are the server details -
>
> server 1 - Red hat 7.5 , vpp release - 20.01
>
> server 2 - Centos 8.1 , vpp release - 20.05
>
>
>
> thanks,
>
> -Raj
>
>
>
>
>
> On Tue, Aug 4, 2020 at 10:24 AM Florin Coras
> wrote:
>
> Hi Raj,
>
>
>
> Interesting. For some reason, the message queue’s underlying
> pthread_cond_timedwait does not work in your setup. Not sure what to make
> of that, unless maybe you’re trying to epoll wait from multiple threads
> that share the same message queue. That is not supported since each thread
> must have its own message queue, i.e., all threads that call epoll should
> be registered as workers. Alternatively, some form of locking or vls,
> instead of vcl, should be used.
>
>
>
> The downside to switching the message queue to eventfd notifications,
> instead of mutext/condvar, is that waits are inefficient, i.e., they act
> pretty much like spinlocks. Do keep that in mind.
>
>
>
> Regards,
>
> Florin
>
>
>
> On Aug 4, 2020, at 6:37 AM, Raj Kumar wrote:
>
>
>
> Hi Florin,
>
> After adding use-mq-eventfd in VCL configuration, it is working as
> expected.
>
> Thanks! for your help.
>
>
>
> vcl {
> rx-fifo-size 400
> tx-fifo-size 400
> app-scope-local
> app-scope-global
> use-mq-eventfd
> api-socket-name /tmp/vpp-api.sock
> }
>
>
>
> thanks,
>
> -Raj
>
>
>
> On Tue, Aug 4, 2020 at 12:08 AM Florin Coras
> wrote:
>
> Hi Raj,
>
>
>
> Glad to hear that issue is solved. What vcl config are you running? Did
> you configure use-mq-eventd?
>
>
>
> Regards,
>
> Florin
>
>
>
> On Aug 3, 2020, at 8:33 PM, Raj Kumar wrote:
>
>
>
> Hi Florin,
>
> This issue is resolved now. In my application, on receiving the kill
> signal main thread was sending phread_cancel() to the child thread
> because of that child thread was not exiting gracefully.
>
> I have one question; it seems that vppcom_epoll_wait(epfd, rcvEvents,
> MAX_RETURN_EVENTS, 6.0) is not returning after timed out if the timeout
> value is a non zero value. It timed out only if the timeout value is 0.
>
> The issue that I am facing is that if there is no traffic at all (
> receiver is just listening on the connections ) then the worker thread is
> not exiting as it is blocked by vppcom_epoll_wait().
>
>
>
> Thanks,
>
> -Raj
>
>
>
>
>
>
>
> On Wed, Jul 29, 2020 at 11:23 PM Florin Coras
> wrote:
>
> Hi Raj,
>
>
>
> In that case it should work. Just from the trace lower it’s hard to figure
> out what exactly happened. Also, keep in mind that vcl is not thread safe,
> so make sure you’re not trying to share sessions or allow two workers to
> interact with the message queue(s) at the same time.
>
>
>
> Regards,
>
> Florin
>
>
>
> On Jul 29, 2020, at 8:17 PM, Raj Kumar wrote:
>
>
>
> Hi Florin,
>
> I am using kill to stop the application. But , the application has a
> kill signal handler and after receiving the signal it is exiting gracefully.
>
> About vppcom_app_exit, I think this
Re: [vpp-dev] VPP 2005 is crashing on stopping the VCL applications #vpp-hoststack
Hi Florin,
Can we please have the fix[1] on "stable/2005" branch.
[1] https://gerrit.fd.io/r/c/vpp/+/28173
Thanks,
-Raj
On Wed, Aug 5, 2020 at 7:30 PM Raj Kumar via lists.fd.io wrote:
> Hi Florin,
> Yes , this[1] fixed the issue.
>
> thanks,
> -Raj
>
> On Wed, Aug 5, 2020 at 1:57 AM Florin Coras
> wrote:
>
>> Hi Raj,
>>
>> Does this [1] fix the issue?
>>
>> Regards,
>> Florin
>>
>> [1] https://gerrit.fd.io/r/c/vpp/+/28173
>>
>> On Aug 4, 2020, at 8:24 AM, Raj Kumar wrote:
>>
>> Hi Florin,
>> I tried vppcom_epoll_wait() on 2 different servers by using a simple
>> application ( with only 1 worker thread) . But, still vppcom_epoll_wait()
>> is not being timed out if I do not use use-mq-eventfd .
>> Here are the server details -
>> server 1 - Red hat 7.5 , vpp release - 20.01
>> server 2 - Centos 8.1 , vpp release - 20.05
>>
>> thanks,
>> -Raj
>>
>>
>> On Tue, Aug 4, 2020 at 10:24 AM Florin Coras
>> wrote:
>>
>>> Hi Raj,
>>>
>>> Interesting. For some reason, the message queue’s underlying
>>> pthread_cond_timedwait does not work in your setup. Not sure what to make
>>> of that, unless maybe you’re trying to epoll wait from multiple threads
>>> that share the same message queue. That is not supported since each thread
>>> must have its own message queue, i.e., all threads that call epoll should
>>> be registered as workers. Alternatively, some form of locking or vls,
>>> instead of vcl, should be used.
>>>
>>> The downside to switching the message queue to eventfd notifications,
>>> instead of mutext/condvar, is that waits are inefficient, i.e., they act
>>> pretty much like spinlocks. Do keep that in mind.
>>>
>>> Regards,
>>> Florin
>>>
>>> On Aug 4, 2020, at 6:37 AM, Raj Kumar wrote:
>>>
>>> Hi Florin,
>>> After adding use-mq-eventfd in VCL configuration, it is working as
>>> expected.
>>> Thanks! for your help.
>>>
>>> vcl {
>>> rx-fifo-size 400
>>> tx-fifo-size 400
>>> app-scope-local
>>> app-scope-global
>>> use-mq-eventfd
>>> api-socket-name /tmp/vpp-api.sock
>>> }
>>>
>>> thanks,
>>> -Raj
>>>
>>> On Tue, Aug 4, 2020 at 12:08 AM Florin Coras
>>> wrote:
>>>
>>>> Hi Raj,
>>>>
>>>> Glad to hear that issue is solved. What vcl config are you running? Did
>>>> you configure use-mq-eventd?
>>>>
>>>> Regards,
>>>> Florin
>>>>
>>>> On Aug 3, 2020, at 8:33 PM, Raj Kumar wrote:
>>>>
>>>> Hi Florin,
>>>> This issue is resolved now. In my application, on receiving the kill
>>>> signal main thread was sending phread_cancel() to the child thread
>>>> because of that child thread was not exiting gracefully.
>>>> I have one question; it seems that vppcom_epoll_wait(epfd, rcvEvents,
>>>> MAX_RETURN_EVENTS, 6.0) is not returning after timed out if the timeout
>>>> value is a non zero value. It timed out only if the timeout value is 0.
>>>> The issue that I am facing is that if there is no traffic at all (
>>>> receiver is just listening on the connections ) then the worker thread is
>>>> not exiting as it is blocked by vppcom_epoll_wait().
>>>>
>>>> Thanks,
>>>> -Raj
>>>>
>>>>
>>>>
>>>> On Wed, Jul 29, 2020 at 11:23 PM Florin Coras
>>>> wrote:
>>>>
>>>>> Hi Raj,
>>>>>
>>>>> In that case it should work. Just from the trace lower it’s hard to
>>>>> figure out what exactly happened. Also, keep in mind that vcl is not
>>>>> thread
>>>>> safe, so make sure you’re not trying to share sessions or allow two
>>>>> workers
>>>>> to interact with the message queue(s) at the same time.
>>>>>
>>>>> Regards,
>>>>> Florin
>>>>>
>>>>> On Jul 29, 2020, at 8:17 PM, Raj Kumar wrote:
>>>>>
>>>>> Hi Florin,
>>>>> I am using kill to stop the application. But , the application
>>>>> has a kill signal handler and after receiving the signal it is exiting
>>>>> gracefully.
>>>>> About vppcom_app_exit, I think this function is registered with
&g
Re: [vpp-dev] VPP 2005 is crashing on stopping the VCL applications #vpp-hoststack
Hi Florin,
Yes , this[1] fixed the issue.
thanks,
-Raj
On Wed, Aug 5, 2020 at 1:57 AM Florin Coras wrote:
> Hi Raj,
>
> Does this [1] fix the issue?
>
> Regards,
> Florin
>
> [1] https://gerrit.fd.io/r/c/vpp/+/28173
>
> On Aug 4, 2020, at 8:24 AM, Raj Kumar wrote:
>
> Hi Florin,
> I tried vppcom_epoll_wait() on 2 different servers by using a simple
> application ( with only 1 worker thread) . But, still vppcom_epoll_wait()
> is not being timed out if I do not use use-mq-eventfd .
> Here are the server details -
> server 1 - Red hat 7.5 , vpp release - 20.01
> server 2 - Centos 8.1 , vpp release - 20.05
>
> thanks,
> -Raj
>
>
> On Tue, Aug 4, 2020 at 10:24 AM Florin Coras
> wrote:
>
>> Hi Raj,
>>
>> Interesting. For some reason, the message queue’s underlying
>> pthread_cond_timedwait does not work in your setup. Not sure what to make
>> of that, unless maybe you’re trying to epoll wait from multiple threads
>> that share the same message queue. That is not supported since each thread
>> must have its own message queue, i.e., all threads that call epoll should
>> be registered as workers. Alternatively, some form of locking or vls,
>> instead of vcl, should be used.
>>
>> The downside to switching the message queue to eventfd notifications,
>> instead of mutext/condvar, is that waits are inefficient, i.e., they act
>> pretty much like spinlocks. Do keep that in mind.
>>
>> Regards,
>> Florin
>>
>> On Aug 4, 2020, at 6:37 AM, Raj Kumar wrote:
>>
>> Hi Florin,
>> After adding use-mq-eventfd in VCL configuration, it is working as
>> expected.
>> Thanks! for your help.
>>
>> vcl {
>> rx-fifo-size 4000000
>> tx-fifo-size 400
>> app-scope-local
>> app-scope-global
>> use-mq-eventfd
>> api-socket-name /tmp/vpp-api.sock
>> }
>>
>> thanks,
>> -Raj
>>
>> On Tue, Aug 4, 2020 at 12:08 AM Florin Coras
>> wrote:
>>
>>> Hi Raj,
>>>
>>> Glad to hear that issue is solved. What vcl config are you running? Did
>>> you configure use-mq-eventd?
>>>
>>> Regards,
>>> Florin
>>>
>>> On Aug 3, 2020, at 8:33 PM, Raj Kumar wrote:
>>>
>>> Hi Florin,
>>> This issue is resolved now. In my application, on receiving the kill
>>> signal main thread was sending phread_cancel() to the child thread
>>> because of that child thread was not exiting gracefully.
>>> I have one question; it seems that vppcom_epoll_wait(epfd, rcvEvents,
>>> MAX_RETURN_EVENTS, 6.0) is not returning after timed out if the timeout
>>> value is a non zero value. It timed out only if the timeout value is 0.
>>> The issue that I am facing is that if there is no traffic at all (
>>> receiver is just listening on the connections ) then the worker thread is
>>> not exiting as it is blocked by vppcom_epoll_wait().
>>>
>>> Thanks,
>>> -Raj
>>>
>>>
>>>
>>> On Wed, Jul 29, 2020 at 11:23 PM Florin Coras
>>> wrote:
>>>
>>>> Hi Raj,
>>>>
>>>> In that case it should work. Just from the trace lower it’s hard to
>>>> figure out what exactly happened. Also, keep in mind that vcl is not thread
>>>> safe, so make sure you’re not trying to share sessions or allow two workers
>>>> to interact with the message queue(s) at the same time.
>>>>
>>>> Regards,
>>>> Florin
>>>>
>>>> On Jul 29, 2020, at 8:17 PM, Raj Kumar wrote:
>>>>
>>>> Hi Florin,
>>>> I am using kill to stop the application. But , the application
>>>> has a kill signal handler and after receiving the signal it is exiting
>>>> gracefully.
>>>> About vppcom_app_exit, I think this function is registered with
>>>> atexit() inside vppcom_app_create() so it should call when the application
>>>> exits.
>>>> Even, I also tried this vppcom_app_exit() explicitly before exiting the
>>>> application but still I am seeing the same issue.
>>>>
>>>> My application is a multithreaded application. Can you please suggest
>>>> some cleanup functions ( vppcom functions) that I should call before
>>>> exiting a thread and the main application for a proper cleanup.
>>>> I also tried vppcom_app_destroy() before exiting the main application
>>>> but still I am seeing the same issue.
>>>>
>>>> thanks,
>
Re: [vpp-dev] VPP 2005 is crashing on stopping the VCL applications #vpp-hoststack
Hi Florin,
I tried vppcom_epoll_wait() on 2 different servers by using a simple
application ( with only 1 worker thread) . But, still vppcom_epoll_wait()
is not being timed out if I do not use use-mq-eventfd .
Here are the server details -
server 1 - Red hat 7.5 , vpp release - 20.01
server 2 - Centos 8.1 , vpp release - 20.05
thanks,
-Raj
On Tue, Aug 4, 2020 at 10:24 AM Florin Coras wrote:
> Hi Raj,
>
> Interesting. For some reason, the message queue’s underlying
> pthread_cond_timedwait does not work in your setup. Not sure what to make
> of that, unless maybe you’re trying to epoll wait from multiple threads
> that share the same message queue. That is not supported since each thread
> must have its own message queue, i.e., all threads that call epoll should
> be registered as workers. Alternatively, some form of locking or vls,
> instead of vcl, should be used.
>
> The downside to switching the message queue to eventfd notifications,
> instead of mutext/condvar, is that waits are inefficient, i.e., they act
> pretty much like spinlocks. Do keep that in mind.
>
> Regards,
> Florin
>
> On Aug 4, 2020, at 6:37 AM, Raj Kumar wrote:
>
> Hi Florin,
> After adding use-mq-eventfd in VCL configuration, it is working as
> expected.
> Thanks! for your help.
>
> vcl {
> rx-fifo-size 400
> tx-fifo-size 400
> app-scope-local
> app-scope-global
> use-mq-eventfd
> api-socket-name /tmp/vpp-api.sock
> }
>
> thanks,
> -Raj
>
> On Tue, Aug 4, 2020 at 12:08 AM Florin Coras
> wrote:
>
>> Hi Raj,
>>
>> Glad to hear that issue is solved. What vcl config are you running? Did
>> you configure use-mq-eventd?
>>
>> Regards,
>> Florin
>>
>> On Aug 3, 2020, at 8:33 PM, Raj Kumar wrote:
>>
>> Hi Florin,
>> This issue is resolved now. In my application, on receiving the kill
>> signal main thread was sending phread_cancel() to the child thread
>> because of that child thread was not exiting gracefully.
>> I have one question; it seems that vppcom_epoll_wait(epfd, rcvEvents,
>> MAX_RETURN_EVENTS, 6.0) is not returning after timed out if the timeout
>> value is a non zero value. It timed out only if the timeout value is 0.
>> The issue that I am facing is that if there is no traffic at all (
>> receiver is just listening on the connections ) then the worker thread is
>> not exiting as it is blocked by vppcom_epoll_wait().
>>
>> Thanks,
>> -Raj
>>
>>
>>
>> On Wed, Jul 29, 2020 at 11:23 PM Florin Coras
>> wrote:
>>
>>> Hi Raj,
>>>
>>> In that case it should work. Just from the trace lower it’s hard to
>>> figure out what exactly happened. Also, keep in mind that vcl is not thread
>>> safe, so make sure you’re not trying to share sessions or allow two workers
>>> to interact with the message queue(s) at the same time.
>>>
>>> Regards,
>>> Florin
>>>
>>> On Jul 29, 2020, at 8:17 PM, Raj Kumar wrote:
>>>
>>> Hi Florin,
>>> I am using kill to stop the application. But , the application has
>>> a kill signal handler and after receiving the signal it is exiting
>>> gracefully.
>>> About vppcom_app_exit, I think this function is registered with atexit()
>>> inside vppcom_app_create() so it should call when the application exits.
>>> Even, I also tried this vppcom_app_exit() explicitly before exiting the
>>> application but still I am seeing the same issue.
>>>
>>> My application is a multithreaded application. Can you please suggest
>>> some cleanup functions ( vppcom functions) that I should call before
>>> exiting a thread and the main application for a proper cleanup.
>>> I also tried vppcom_app_destroy() before exiting the main application
>>> but still I am seeing the same issue.
>>>
>>> thanks,
>>> -Raj
>>>
>>> On Wed, Jul 29, 2020 at 5:34 PM Florin Coras
>>> wrote:
>>>
>>>> Hi Raj,
>>>>
>>>> Does stopping include a call to vppcom_app_exit or killing the
>>>> applications? If the latter, the apps might be killed with some
>>>> mutexes/spinlocks held. For now, we only support the former.
>>>>
>>>> Regards,
>>>> Florin
>>>>
>>>> > On Jul 29, 2020, at 1:49 PM, Raj Kumar
>>>> wrote:
>>>> >
>>>> > Hi,
>>>> > In my UDP application , I am using VPP host stack to receive packets
>>>> and memIf to transmit packets. There are
Re: [vpp-dev] VPP 2005 is crashing on stopping the VCL applications #vpp-hoststack
Hi Florin,
After adding use-mq-eventfd in VCL configuration, it is working as
expected.
Thanks! for your help.
vcl {
rx-fifo-size 400
tx-fifo-size 400
app-scope-local
app-scope-global
use-mq-eventfd
api-socket-name /tmp/vpp-api.sock
}
thanks,
-Raj
On Tue, Aug 4, 2020 at 12:08 AM Florin Coras wrote:
> Hi Raj,
>
> Glad to hear that issue is solved. What vcl config are you running? Did
> you configure use-mq-eventd?
>
> Regards,
> Florin
>
> On Aug 3, 2020, at 8:33 PM, Raj Kumar wrote:
>
> Hi Florin,
> This issue is resolved now. In my application, on receiving the kill
> signal main thread was sending phread_cancel() to the child thread
> because of that child thread was not exiting gracefully.
> I have one question; it seems that vppcom_epoll_wait(epfd, rcvEvents,
> MAX_RETURN_EVENTS, 6.0) is not returning after timed out if the timeout
> value is a non zero value. It timed out only if the timeout value is 0.
> The issue that I am facing is that if there is no traffic at all (
> receiver is just listening on the connections ) then the worker thread is
> not exiting as it is blocked by vppcom_epoll_wait().
>
> Thanks,
> -Raj
>
>
>
> On Wed, Jul 29, 2020 at 11:23 PM Florin Coras
> wrote:
>
>> Hi Raj,
>>
>> In that case it should work. Just from the trace lower it’s hard to
>> figure out what exactly happened. Also, keep in mind that vcl is not thread
>> safe, so make sure you’re not trying to share sessions or allow two workers
>> to interact with the message queue(s) at the same time.
>>
>> Regards,
>> Florin
>>
>> On Jul 29, 2020, at 8:17 PM, Raj Kumar wrote:
>>
>> Hi Florin,
>> I am using kill to stop the application. But , the application has
>> a kill signal handler and after receiving the signal it is exiting
>> gracefully.
>> About vppcom_app_exit, I think this function is registered with atexit()
>> inside vppcom_app_create() so it should call when the application exits.
>> Even, I also tried this vppcom_app_exit() explicitly before exiting the
>> application but still I am seeing the same issue.
>>
>> My application is a multithreaded application. Can you please suggest
>> some cleanup functions ( vppcom functions) that I should call before
>> exiting a thread and the main application for a proper cleanup.
>> I also tried vppcom_app_destroy() before exiting the main application but
>> still I am seeing the same issue.
>>
>> thanks,
>> -Raj
>>
>> On Wed, Jul 29, 2020 at 5:34 PM Florin Coras
>> wrote:
>>
>>> Hi Raj,
>>>
>>> Does stopping include a call to vppcom_app_exit or killing the
>>> applications? If the latter, the apps might be killed with some
>>> mutexes/spinlocks held. For now, we only support the former.
>>>
>>> Regards,
>>> Florin
>>>
>>> > On Jul 29, 2020, at 1:49 PM, Raj Kumar wrote:
>>> >
>>> > Hi,
>>> > In my UDP application , I am using VPP host stack to receive packets
>>> and memIf to transmit packets. There are a total 6 application connected to
>>> VPP.
>>> > if I stop the application(s) then VPP is crashing. In vpp
>>> configuration , 4 worker threads are configured. If there is no worker
>>> thread configured then I do not see this crash.
>>> > Here is the VPP task trace -
>>> > (gdb) bt
>>> > #0 0x751818df in raise () from /lib64/libc.so.6
>>> > #1 0x7516bcf5 in abort () from /lib64/libc.so.6
>>> > #2 0xc123 in os_panic () at
>>> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vpp/vnet/main.c:366
>>> > #3 0x76b466bb in vlib_worker_thread_barrier_sync_int
>>> (vm=0x76d78200 , func_name=)
>>> > at
>>> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlib/threads.c:1529
>>> > #4 0x77bc5ef0 in vl_msg_api_handler_with_vm_node
>>> > (am=am@entry=0x77dd2ea0
>>> ,
>>> > vlib_rp=vlib_rp@entry=0x7fee7c001000, the_msg=0x7fee7c02bbd8,
>>> vm=vm@entry=0x76d78200 ,
>>> > node=node@entry=0x7fffb6295000, is_private=is_private@entry=1
>>> '\001')
>>> > at
>>> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibapi/api_shared.c:596
>>> > #5 0x77bb000f in void_mem_api_handle_msg_i (is_private=1
>>> '\001', node=0x7fffb6295000, vm=0x76d78200 ,
>>> > vlib_rp=0x7fee7c001000, am=0x77dd2ea0 )
>>> > at
>>> /usr/src/de
Re: [vpp-dev] VPP 2005 is crashing on stopping the VCL applications #vpp-hoststack
Hi Florin, This issue is resolved now. In my application, on receiving the kill signal main thread was sending phread_cancel() to the child thread because of that child thread was not exiting gracefully. I have one question; it seems that vppcom_epoll_wait(epfd, rcvEvents, MAX_RETURN_EVENTS, 6.0) is not returning after timed out if the timeout value is a non zero value. It timed out only if the timeout value is 0. The issue that I am facing is that if there is no traffic at all ( receiver is just listening on the connections ) then the worker thread is not exiting as it is blocked by vppcom_epoll_wait(). Thanks, -Raj On Wed, Jul 29, 2020 at 11:23 PM Florin Coras wrote: > Hi Raj, > > In that case it should work. Just from the trace lower it’s hard to figure > out what exactly happened. Also, keep in mind that vcl is not thread safe, > so make sure you’re not trying to share sessions or allow two workers to > interact with the message queue(s) at the same time. > > Regards, > Florin > > On Jul 29, 2020, at 8:17 PM, Raj Kumar wrote: > > Hi Florin, > I am using kill to stop the application. But , the application has a > kill signal handler and after receiving the signal it is exiting gracefully. > About vppcom_app_exit, I think this function is registered with atexit() > inside vppcom_app_create() so it should call when the application exits. > Even, I also tried this vppcom_app_exit() explicitly before exiting the > application but still I am seeing the same issue. > > My application is a multithreaded application. Can you please suggest some > cleanup functions ( vppcom functions) that I should call before exiting a > thread and the main application for a proper cleanup. > I also tried vppcom_app_destroy() before exiting the main application but > still I am seeing the same issue. > > thanks, > -Raj > > On Wed, Jul 29, 2020 at 5:34 PM Florin Coras > wrote: > >> Hi Raj, >> >> Does stopping include a call to vppcom_app_exit or killing the >> applications? If the latter, the apps might be killed with some >> mutexes/spinlocks held. For now, we only support the former. >> >> Regards, >> Florin >> >> > On Jul 29, 2020, at 1:49 PM, Raj Kumar wrote: >> > >> > Hi, >> > In my UDP application , I am using VPP host stack to receive packets >> and memIf to transmit packets. There are a total 6 application connected to >> VPP. >> > if I stop the application(s) then VPP is crashing. In vpp >> configuration , 4 worker threads are configured. If there is no worker >> thread configured then I do not see this crash. >> > Here is the VPP task trace - >> > (gdb) bt >> > #0 0x751818df in raise () from /lib64/libc.so.6 >> > #1 0x7516bcf5 in abort () from /lib64/libc.so.6 >> > #2 0xc123 in os_panic () at >> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vpp/vnet/main.c:366 >> > #3 0x76b466bb in vlib_worker_thread_barrier_sync_int >> (vm=0x76d78200 , func_name=) >> > at >> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlib/threads.c:1529 >> > #4 0x77bc5ef0 in vl_msg_api_handler_with_vm_node >> > (am=am@entry=0x77dd2ea0 >> , >> > vlib_rp=vlib_rp@entry=0x7fee7c001000, the_msg=0x7fee7c02bbd8, >> vm=vm@entry=0x76d78200 , >> > node=node@entry=0x7fffb6295000, is_private=is_private@entry=1 >> '\001') >> > at >> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibapi/api_shared.c:596 >> > #5 0x77bb000f in void_mem_api_handle_msg_i (is_private=1 >> '\001', node=0x7fffb6295000, vm=0x76d78200 , >> > vlib_rp=0x7fee7c001000, am=0x77dd2ea0 ) >> > at >> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/memory_api.c:698 >> > #6 vl_mem_api_handle_msg_private (vm=vm@entry=0x76d78200 >> , node=node@entry=0x7fffb6295000, reg_index=> out>) >> > at >> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/memory_api.c:762 >> > #7 0x77bbe346 in vl_api_clnt_process (vm=, >> node=0x7fffb6295000, f=) >> > at >> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/vlib_api.c:370 >> > #8 0x76b161d6 in vlib_process_bootstrap (_a=) >> > at >> /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlib/main.c:1502 >> > #9 0x7602ac0c in clib_calljmp () from >> /lib64/libvppinfra.so.20.05 >> > #10 0x7fffb5e93dd0 in ?? () >> > #11 0x76b19821 in dispatch_process (vm=0x76d78200 >> , p=0x7fffb629
Re: [vpp-dev] VPP 2005 is crashing on stopping the VCL applications #vpp-hoststack
Hi Florin, I am using kill to stop the application. But , the application has a kill signal handler and after receiving the signal it is exiting gracefully. About vppcom_app_exit, I think this function is registered with atexit() inside vppcom_app_create() so it should call when the application exits. Even, I also tried this vppcom_app_exit() explicitly before exiting the application but still I am seeing the same issue. My application is a multithreaded application. Can you please suggest some cleanup functions ( vppcom functions) that I should call before exiting a thread and the main application for a proper cleanup. I also tried vppcom_app_destroy() before exiting the main application but still I am seeing the same issue. thanks, -Raj On Wed, Jul 29, 2020 at 5:34 PM Florin Coras wrote: > Hi Raj, > > Does stopping include a call to vppcom_app_exit or killing the > applications? If the latter, the apps might be killed with some > mutexes/spinlocks held. For now, we only support the former. > > Regards, > Florin > > > On Jul 29, 2020, at 1:49 PM, Raj Kumar wrote: > > > > Hi, > > In my UDP application , I am using VPP host stack to receive packets and > memIf to transmit packets. There are a total 6 application connected to > VPP. > > if I stop the application(s) then VPP is crashing. In vpp configuration > , 4 worker threads are configured. If there is no worker thread configured > then I do not see this crash. > > Here is the VPP task trace - > > (gdb) bt > > #0 0x751818df in raise () from /lib64/libc.so.6 > > #1 0x7516bcf5 in abort () from /lib64/libc.so.6 > > #2 0xc123 in os_panic () at > /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vpp/vnet/main.c:366 > > #3 0x76b466bb in vlib_worker_thread_barrier_sync_int > (vm=0x76d78200 , func_name=) > > at > /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlib/threads.c:1529 > > #4 0x77bc5ef0 in vl_msg_api_handler_with_vm_node > > (am=am@entry=0x77dd2ea0 > , > > vlib_rp=vlib_rp@entry=0x7fee7c001000, the_msg=0x7fee7c02bbd8, > vm=vm@entry=0x76d78200 , > > node=node@entry=0x7fffb6295000, is_private=is_private@entry=1 > '\001') > > at > /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibapi/api_shared.c:596 > > #5 0x77bb000f in void_mem_api_handle_msg_i (is_private=1 > '\001', node=0x7fffb6295000, vm=0x76d78200 , > > vlib_rp=0x7fee7c001000, am=0x77dd2ea0 ) > > at > /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/memory_api.c:698 > > #6 vl_mem_api_handle_msg_private (vm=vm@entry=0x76d78200 > , node=node@entry=0x7fffb6295000, reg_index= out>) > > at > /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/memory_api.c:762 > > #7 0x77bbe346 in vl_api_clnt_process (vm=, > node=0x7fffb6295000, f=) > > at > /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/vlib_api.c:370 > > #8 0x76b161d6 in vlib_process_bootstrap (_a=) > > at > /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlib/main.c:1502 > > #9 0x7602ac0c in clib_calljmp () from > /lib64/libvppinfra.so.20.05 > > #10 0x7fffb5e93dd0 in ?? () > > #11 0x76b19821 in dispatch_process (vm=0x76d78200 > , p=0x7fffb6295000, last_time_stamp=15931923011231136, > > f=0x0) at > /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vppinfra/types.h:133 > > #12 0x7f0f66009024 in ?? () > > > > > > Thanks, > > -Raj > > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17113): https://lists.fd.io/g/vpp-dev/message/17113 Mute This Topic: https://lists.fd.io/mt/75873900/21656 Mute #vpp-hoststack: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/vpp-hoststack Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP 2005 is crashing on stopping the VCL applications #vpp-hoststack
Hi, In my UDP application , I am using VPP host stack to receive packets and memIf to transmit packets. There are a total 6 application connected to VPP. if I stop the application(s) then VPP is crashing. In vpp configuration , 4 worker threads are configured. If there is no worker thread configured then I do not see this crash. Here is the VPP task trace - (gdb) bt #0 0x751818df in raise () from /lib64/libc.so.6 #1 0x7516bcf5 in abort () from /lib64/libc.so.6 #2 0xc123 in os_panic () at /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vpp/vnet/main.c:366 #3 0x76b466bb in vlib_worker_thread_barrier_sync_int (vm=0x76d78200 , func_name=) at /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlib/threads.c:1529 #4 0x77bc5ef0 in vl_msg_api_handler_with_vm_node (am=am@entry=0x77dd2ea0 , vlib_rp=vlib_rp@entry=0x7fee7c001000, the_msg=0x7fee7c02bbd8, vm=vm@entry=0x76d78200 , node=node@entry=0x7fffb6295000, is_private=is_private@entry=1 '\001') at /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibapi/api_shared.c:596 #5 0x77bb000f in void_mem_api_handle_msg_i (is_private=1 '\001', node=0x7fffb6295000, vm=0x76d78200 , vlib_rp=0x7fee7c001000, am=0x77dd2ea0 ) at /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/memory_api.c:698 #6 vl_mem_api_handle_msg_private (vm=vm@entry=0x76d78200 , node=node@entry=0x7fffb6295000, reg_index=) at /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/memory_api.c:762 #7 0x77bbe346 in vl_api_clnt_process (vm=, node=0x7fffb6295000, f=) at /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlibmemory/vlib_api.c:370 #8 0x76b161d6 in vlib_process_bootstrap (_a=) at /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vlib/main.c:1502 #9 0x7602ac0c in clib_calljmp () from /lib64/libvppinfra.so.20.05 #10 0x7fffb5e93dd0 in ?? () #11 0x76b19821 in dispatch_process (vm=0x76d78200 , p=0x7fffb6295000, last_time_stamp=15931923011231136, f=0x0) at /usr/src/debug/vpp-20.05-9~g0bf9c294c_dirty.x86_64/src/vppinfra/types.h:133 #12 0x7f0f66009024 in ?? () Thanks, -Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17110): https://lists.fd.io/g/vpp-dev/message/17110 Mute This Topic: https://lists.fd.io/mt/75873900/21656 Mute #vpp-hoststack: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/vpp-hoststack Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] How to active tx udp checksum offload #dpdk #mellanox
Hi,
I am using vpp stable/2005 code. I want to enable UDP checksum offload for tx .
I changed the vpp startup.cfg file -
## Disables UDP / TCP TX checksum offload. Typically needed for use
## faster vector PMDs (together with no-multi-seg)
# no-tx-checksum-offload
## Enable UDP / TCP TX checksum offload
## This is the reversed option of 'no-tx-checksum-offload'
*enable-tcp-udp-checksum*
But , still it is not activated . However all these tx offloads are available.
With VPP 19.05 , I was able to activate it by adding following piece of code in
./plugins/dpdk/device/init.c
case VNET_DPDK_PMD_CXGBE:
case VNET_DPDK_PMD_MLX4:
case VNET_DPDK_PMD_MLX5:
case VNET_DPDK_PMD_QEDE:
case VNET_DPDK_PMD_BNXT:
xd->port_type = port_type_from_speed_capa (_info);
*if (dm->conf->no_tx_checksum_offload == 0)*
*{*
*xd->port_conf.txmode.offloads |= DEV_TX_OFFLOAD_TCP_CKSUM;*
*xd->port_conf.txmode.offloads |= DEV_TX_OFFLOAD_UDP_CKSUM;*
*xd->flags |=*
*DPDK_DEVICE_FLAG_TX_OFFLOAD |*
*DPDK_DEVICE_FLAG_INTEL_PHDR_CKSUM;*
*}*
break;
But, the above code does not work with vpp 20.05. I think , there is a newer
version of DPDK in this release.
Please let me know if I am doing something wrong.
HundredGigabitEthernet12/0/0 2 up HundredGigabitEthernet12/0/0
Link speed: 100 Gbps
Ethernet address b8:83:03:9e:68:f0
Mellanox ConnectX-4 Family
carrier up full duplex mtu 9206
flags: admin-up pmd maybe-multiseg subif rx-ip4-cksum
rx: queues 2 (max 1024), desc 1024 (min 0 max 65535 align 1)
tx: queues 4 (max 1024), desc 1024 (min 0 max 65535 align 1)
pci: device 15b3:1013 subsystem 1590:00c8 address :12:00.00 numa 0
switch info: name :12:00.0 domain id 0 port id 65535
max rx packet len: 65536
promiscuous: unicast off all-multicast on
vlan offload: strip off filter off qinq off
rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum vlan-filter
jumbo-frame scatter timestamp keep-crc rss-hash
rx offload active: ipv4-cksum udp-cksum tcp-cksum jumbo-frame scatter
*tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum tcp-tso*
*outer-ipv4-cksum vxlan-tnl-tso gre-tnl-tso geneve-tnl-tso*
*multi-segs udp-tnl-tso ip-tnl-tso*
*tx offload active: multi-segs*
rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4 ipv6-tcp-ex
ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other
ipv6-ex ipv6 l4-dst-only l4-src-only l3-dst-only l3-src-only
rss active: ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4 ipv6-tcp-ex
ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other
ipv6-ex ipv6
tx burst mode: No MPW + MULTI + TSO + INLINE + METADATA
rx burst mode: Scalar
tx frames ok 14311830
tx bytes ok 128602546562
rx frames ok 1877
rx bytes ok 228452
thanks,
-Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#16863): https://lists.fd.io/g/vpp-dev/message/16863
Mute This Topic: https://lists.fd.io/mt/75246142/21656
Mute #dpdk: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/dpdk
Mute #mellanox: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/mellanox
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Observing multiple VRRP Routers acting as master while testing Master/Back-up functionality using vrrp plugin
Hello Amit, state Master flags: preempt yes accept *no* unicast no Your clue lies here. Check https://vpp.flirble.org/master/d7/d40/vrrp_8c_source.html#l00182 and https://vpp.flirble.org/master/dc/dfb/vrrp__cli_8c.html#a2fd76fa6d5cd9ddfef75af8f0d12e016 HTH. Muthu On Sat, Jun 13, 2020 at 2:01 PM Amit Mehra wrote: > Hi, > > I am trying to test Master/Backup functionality on 2 VPP nodes, each > running it's own VRRP plugin and I am observing 2 VRRP routers acting as > Masters at the same time. Please find below the configuration that i am > using for my testing:- > > On VVP Node 1:- > set interface mtu 1500 GigabitEthernet13/0/0 > set interface ip address GigabitEthernet13/0/0 10.20.37.118/24 > set interface state GigabitEthernet13/0/0 up > vrrp vr add GigabitEthernet13/0/0 vr_id 1 priority 255 10.20.37.118 > vrrp proto start GigabitEthernet13/0/0 vr_id 1 > > This becomes a Master, as it is the address owner and has the priority as > 255. > > On VPP Node2:- > set interface mtu 1500 GigabitEthernet13/0/0 > set interface ip address GigabitEthernet13/0/0 10.20.37.109/24 > set interface state GigabitEthernet13/0/0 up > vrrp vr add GigabitEthernet13/0/0 vr_id 1 priority 200 accept_mode > 10.20.37.118 > vrrp proto start GigabitEthernet13/0/0 vr_id 1 > > This becomes a back-up and has priority 200. > > Note:- 10.20.37.118 is the IP that will be pinged from external > machine(which would be having same subnet as 10.20.37.xx) and that is why > have configured this IP on VR (VVP instance 2) with "accept_mode" ON. > > So, VR on VPP-1 comes up as Master and VR on VPP-2 comes up as Back-up. > Now, if i kill VPP-1, then VR on VPP-2 becomes a Master and IP 10.20.37.118 > also gets added to the VPP interface. Please find the output of show int > addr below > > vpp# show int addr > GigabitEthernet13/0/0 (up): > L3 10.20.37.109/24 > L3 10.20.37.118/24 > GigabitEthernet1b/0/0 (dn): > local0 (dn): > > vpp# show vrrp vr > [0] sw_if_index 1 VR ID 1 IPv4 >state Master flags: preempt yes accept yes unicast no >priority: configured 200 adjusted 200 >timers: adv interval 100 master adv 100 skew 21 master down 321 >virtual MAC 00:00:5e:00:01:01 >addresses 10.20.37.118 >peer addresses >tracked interfaces > > Now, if i bring VPP-1 up, the VR on VVP-1 also becomes a Master and VR on > VPP-2 also remains in Master State.At this moment, both the VRs i.e on > VPP-1 and VPP-2 are acting as Masters. Please find the output of show vrrp > vr from both the VPP instances:- > > From VPP-1 > vpp# show vrrp vr > [0] sw_if_index 1 VR ID 1 IPv4 >state Master flags: preempt yes accept no unicast no >priority: configured 255 adjusted 255 >timers: adv interval 100 master adv 0 skew 0 master down 0 >virtual MAC 00:00:5e:00:01:01 >addresses 10.20.37.118 >peer addresses >tracked interfaces > > vpp# show int addr > GigabitEthernet13/0/0 (up): > L3 10.20.37.118/24 > GigabitEthernet1b/0/0 (dn): > local0 (dn): > > On VPP-2 > vpp# show vrrp vr > [0] sw_if_index 1 VR ID 1 IPv4 >state Master flags: preempt yes accept yes unicast no >priority: configured 200 adjusted 200 >timers: adv interval 100 master adv 100 skew 21 master down 321 >virtual MAC 00:00:5e:00:01:01 >addresses 10.20.37.118 >peer addresses >tracked interfaces > > vpp# show int addr > GigabitEthernet13/0/0 (up): > L3 10.20.37.109/24 > L3 10.20.37.118/24 > GigabitEthernet1b/0/0 (dn): > local0 (dn): > > Is this a known issue or am i missing something in my configuration? How > to overcome this issue of multiple VRRP routers acting as Master at the > same time? > > Regards > Amit > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16719): https://lists.fd.io/g/vpp-dev/message/16719 Mute This Topic: https://lists.fd.io/mt/74854562/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
Hi Florin, VPP from the "stable/2005" branch is working fine. Thanks! for your help. I have one question; for the UDP tx if the destination address is not in the same subnet then it works after adding a route for the next hop. In my case , VPP interface ip is 2001:5b0::701:b883:31f:29e:68f0 and I am sending UDP traffic to address 2001:5b0::700:ba83:3ff:fe9e:6848. It works fine after adding the following route in VPP fib ip route add 2001:5b0::700:ba83:3ff:fe9e:6848/128 via 2001:5b0::701::254 HundredGigabitEthernet12/0/0.701 My requirement is to use SRv6 for packet routing . I tried by adding following sid entry in vpp sr policy add bsid 2001:5b0::700:ba83:3ff:fe9e:6848 next 2001:5b0::701::254 insert But, with this configuration , UDP tx application is not able to connect. udpTxThread started!!! ... tx port = 9988vppcom_session_create() ..16777216 vppcom_session_connect:1741: vcl<47606:1>: session handle 16777216 (STATE_CLOSED): connecting to peer IPv6 2001:5b0::700:ba83:3ff:fe9e:6848 port 9988 proto UDP vcl_session_connected_handler:450: vcl<47606:1>: ERROR: session index 0: connect failed! no resolving interface vppcom_session_connect:1756: vcl<47606:1>: session 0 [0x0]: connect failed! vppcom_session_connect() failed ... -111 >From the traces , it looks like that VCL session is not looking into the SID entries. Please let me know if I am doing something wrong. thanks, -Raj On Thu, Jun 4, 2020 at 4:49 PM Florin Coras wrote: > Hi Raj, > > You have it here [4]. I’ll merge it once it verifies. > > Regards, > Florin > > [4] https://gerrit.fd.io/r/c/vpp/+/27432 > > On Jun 4, 2020, at 1:41 PM, Raj Kumar wrote: > > Hi Florin, > > To pick up all the following patches I downloaded VPP code from Master > branch. > > [1] https://gerrit.fd.io/r/c/vpp/+/27111 > [2] https://gerrit.fd.io/r/c/vpp/+/27106 > [3] https://gerrit.fd.io/r/c/vpp/+/27235 > > But , with this build I observed that UDP listener is always migrating the > received connection on the first worker thread . With stable/2005 ( + > patches) code, VPP was migrating the connections on different worker > threads. I am using UDP socket with CONNECTED attribute. > > > I found that on stable/2005 you already merged [2] and [3]. Please let me > know if you are planning to merge [1] on the stable/2005 branch. For now, I > want to stay with stable/2005. > > thanks, > -Raj > > On Sun, May 31, 2020 at 11:32 PM Florin Coras > wrote: > >> Hi Raj, >> >> Inline. >> >> On May 31, 2020, at 8:10 PM, Raj Kumar wrote: >> >> Hi Florin, >> The UDPC connections are working fine . I was making a basic mistake. For >> the second application, I missed to export the LD_LIBRARY_PATH to the >> build directory because of that application was linking to the old library >> ( form /usr/lib64). >> Now, I tried both UDP tx and rx connection ( UDP connected) . Both are >> working fine. >> >> >> FC: Great! >> >> >> I have a question; all the connections originating from VPP host stack >> (UDP tx) are always going on the worker thread 1. Is there any way to >> assign these connections to the different worker threads ( similar to the >> UDP rx) ? I can not rely on the receiver ( to initiate the connection) as >> that is a third party application. >> >> >> FC: At this time no. >> >> >> Earlier with the previous VPP release , I tried by assigning UDP tx >> connections to the different worker threads in a round robin manner . I am >> wondering if we can try some thing similar in the new release. >> >> >> It might just work. Let me know if you try it out. >> >> >> Also, please let me know in which VPP release, the following patches >> would be available. >> >> >> FC: The first two are part of 20.05, the third will be ported to the >> first 20.05 point release. >> >> Regards, >> Florin >> >> >> [1] https://gerrit.fd.io/r/c/vpp/+/27111 >> [2] https://gerrit.fd.io/r/c/vpp/+/27106 >> [3] https://gerrit.fd.io/r/c/vpp/+/27235 >> >> >> Here are the VPP stats - >> >> vpp# sh app server >> Connection App Wrk >> [0:0][U] 2001:5b0::701:b883:31f:29e:udp6_rx[shm] 1 >> [0:1][U] 2001:5b0::701:b883:31f:29e:udp6_rx[shm] 1 >> vpp# >> vpp# sh app client >> Connection App >> [1:0][U] 2001:5b0::701:b883:31f:29e:udp6_tx[shm] >> [1:1][U] 2001:5b0::701:b883:31f:29e:udp6_tx[shm] >> vpp# >> vpp# sh session verbose
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
Hi Florin, To pick up all the following patches I downloaded VPP code from Master branch. [1] https://gerrit.fd.io/r/c/vpp/+/27111 [2] https://gerrit.fd.io/r/c/vpp/+/27106 [3] https://gerrit.fd.io/r/c/vpp/+/27235 But , with this build I observed that UDP listener is always migrating the received connection on the first worker thread . With stable/2005 ( + patches) code, VPP was migrating the connections on different worker threads. I am using UDP socket with CONNECTED attribute. I found that on stable/2005 you already merged [2] and [3]. Please let me know if you are planning to merge [1] on the stable/2005 branch. For now, I want to stay with stable/2005. thanks, -Raj On Sun, May 31, 2020 at 11:32 PM Florin Coras wrote: > Hi Raj, > > Inline. > > On May 31, 2020, at 8:10 PM, Raj Kumar wrote: > > Hi Florin, > The UDPC connections are working fine . I was making a basic mistake. For > the second application, I missed to export the LD_LIBRARY_PATH to the > build directory because of that application was linking to the old library > ( form /usr/lib64). > Now, I tried both UDP tx and rx connection ( UDP connected) . Both are > working fine. > > > FC: Great! > > > I have a question; all the connections originating from VPP host stack > (UDP tx) are always going on the worker thread 1. Is there any way to > assign these connections to the different worker threads ( similar to the > UDP rx) ? I can not rely on the receiver ( to initiate the connection) as > that is a third party application. > > > FC: At this time no. > > > Earlier with the previous VPP release , I tried by assigning UDP tx > connections to the different worker threads in a round robin manner . I am > wondering if we can try some thing similar in the new release. > > > It might just work. Let me know if you try it out. > > > Also, please let me know in which VPP release, the following patches would > be available. > > > FC: The first two are part of 20.05, the third will be ported to the first > 20.05 point release. > > Regards, > Florin > > > [1] https://gerrit.fd.io/r/c/vpp/+/27111 > [2] https://gerrit.fd.io/r/c/vpp/+/27106 > [3] https://gerrit.fd.io/r/c/vpp/+/27235 > > > Here are the VPP stats - > > vpp# sh app server > Connection App Wrk > [0:0][U] 2001:5b0::701:b883:31f:29e:udp6_rx[shm] 1 > [0:1][U] 2001:5b0::701:b883:31f:29e:udp6_rx[shm] 1 > vpp# > vpp# sh app client > Connection App > [1:0][U] 2001:5b0::701:b883:31f:29e:udp6_tx[shm] > [1:1][U] 2001:5b0::701:b883:31f:29e:udp6_tx[shm] > vpp# > vpp# sh session verbose > ConnectionState Rx-f > Tx-f > [0:0][U] 2001:5b0::701:b883:31f:29e:9880:12345LISTEN 0 > 0 > [0:1][U] 2001:5b0::701:b883:31f:29e:9881:56789LISTEN 0 > 0 > Thread 0: active sessions 2 > > ConnectionState Rx-f > Tx-f > [1:0][U] 2001:5b0::701:b883:31f:29e:9880:58199OPENED 0 > 3999756 > [1:1][U] 2001:5b0::701:b883:31f:29e:9880:10442OPENED 0 > 3999756 > Thread 1: active sessions 2 > > ConnectionState Rx-f > Tx-f > [2:0][U] 2001:5b0::701:b883:31f:29e:9881:56789OPENED 0 > 0 > Thread 2: active sessions 1 > Thread 3: no sessions > > ConnectionState Rx-f > Tx-f > [4:0][U] 2001:5b0::701:b883:31f:29e:9880:12345OPENED 0 > 0 > Thread 4: active sessions 1 > > Thanks, > -Raj > > > > On Sun, May 31, 2020 at 7:35 PM Florin Coras > wrote: > >> Hi Raj, >> >> Inline. >> >> On May 31, 2020, at 4:07 PM, Raj Kumar wrote: >> >> Hi Florin, >> I was trying this test with debug binaries, but as soon as I enable the >> interface , vpp is crashing. >> >> >> FC: It looks like somehow corrupted buffers make their way into the error >> drop node. What traffic are you running through vpp and is this master or >> are you running some custom code? >> >> >> On the original problem ( multiple listener); if I open multiple sockets >> from the same multi threaded application then it works fine. But, if I >> start another application then only I see the VPP crash( which I mentioned >> in my previous email). >> >> >> FC: Is the second app trying to listen on the same ip:port pair? That is >> not supported and the second listen request should’ve been rejected. Do you >> have a bt? >> >> Regar
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
Hi Florin, I was trying this test with debug binaries, but as soon as I enable the interface , vpp is crashing. On the original problem ( multiple listener); if I open multiple sockets from the same multi threaded application then it works fine. But, if I start another application then only I see the VPP crash( which I mentioned in my previous email). Here is the stack trace with debug binaries which is crashing on startup. Please let me know what wrong I am doing here ? #0 0x74b748df in raise () from /lib64/libc.so.6 #1 0x74b5ecf5 in abort () from /lib64/libc.so.6 #2 0x00407a28 in os_panic () at /opt/vpp/src/vpp/vnet/main.c:366 #3 0x75a279af in debugger () at /opt/vpp/src/vppinfra/error.c:84 #4 0x75a27d92 in _clib_error (how_to_die=2, function_name=0x7663d540 <__FUNCTION__.35997> "vlib_buffer_validate_alloc_free", line_number=367, fmt=0x7663d0ba "%s %U buffer 0x%x") at /opt/vpp/src/vppinfra/error.c:143 #5 0x765752a3 in vlib_buffer_validate_alloc_free (vm=0x76866340 , buffers=0x7fffb586d630, n_buffers=1, expected_state=VLIB_BUFFER_KNOWN_ALLOCATED) at /opt/vpp/src/vlib/buffer.c:366 #6 0x765675f0 in vlib_buffer_pool_put (vm=0x76866340 , buffer_pool_index=0 '\000', buffers=0x7fffb586d630, n_buffers=1) at /opt/vpp/src/vlib/buffer_funcs.h:754 #7 0x76567dde in vlib_buffer_free_inline (vm=0x76866340 , buffers=0x7fffb67e9b14, n_buffers=0, maybe_next=1) at /opt/vpp/src/vlib/buffer_funcs.h:924 #8 0x76567e2e in vlib_buffer_free (vm=0x76866340 , buffers=0x7fffb67e9b10, n_buffers=1) at /opt/vpp/src/vlib/buffer_funcs.h:943 #9 0x76568cb0 in process_drop_punt (vm=0x76866340 , node=0x7fffb5475e00, frame=0x7fffb67e9b00, disposition=ERROR_DISPOSITION_DROP) at /opt/vpp/src/vlib/drop.c:231 #10 0x76568db0 in error_drop_node_fn_hsw (vm=0x76866340 , node=0x7fffb5475e00, frame=0x7fffb67e9b00) at /opt/vpp/src/vlib/drop.c:247 #11 0x765c3447 in dispatch_node (vm=0x76866340 , node=0x7fffb5475e00, type=VLIB_NODE_TYPE_INTERNAL, dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fffb67e9b00, last_time_stamp=15480427682338516) at /opt/vpp/src/vlib/main.c:1235 #12 0x765c3c02 in dispatch_pending_node (vm=0x76866340 , pending_frame_index=2, last_time_stamp=15480427682338516) at /opt/vpp/src/vlib/main.c:1403 #13 0x765c58c9 in vlib_main_or_worker_loop (vm=0x76866340 , is_main=1) at /opt/vpp/src/vlib/main.c:1862 #14 0x765c6320 in vlib_main_loop (vm=0x76866340 ) at /opt/vpp/src/vlib/main.c:1990 #15 0x765c70e0 in vlib_main (vm=0x76866340 , input=0x7fffb586efb0) at /opt/vpp/src/vlib/main.c:2236 #16 0x7662f311 in thread0 (arg=140737329390400) at /opt/vpp/src/vlib/unix/main.c:658 #17 0x75a465cc in clib_calljmp () at /opt/vpp/src/vppinfra/longjmp.S:123 #18 0x7fffd0a0 in ?? () #19 0x7662f8a7 in vlib_unix_main (argc=50, argv=0x705f00) at /opt/vpp/src/vlib/unix/main.c:730 #20 0x00407387 in main (argc=50, argv=0x705f00) at /opt/vpp/src/vpp/vnet/main.c:291 thanks, -Raj On Mon, May 25, 2020 at 5:02 PM Florin Coras wrote: > Hi Raj, > > Okay, so at least with that we have support for bounded listeners (note > that [2] was merged but to set the connected option you now have to > use vppcom_session_attr). > > As for the trace, something seems off. Why exactly does it crash? It looks > as if session_get_transport_proto (ls) crashes because of ls being null, > but prior to that ls is dereferenced and it does’t crash. Could you try > with debug binaries? > > Regards, > Florin > > On May 25, 2020, at 1:43 PM, Raj Kumar wrote: > > Hi Florin, > This works fine with single udp listener. I can see connections going on > different cores. But, if I run more than one listener then VPP is crashing. > Here are the VPP stack traces - > > (gdb) bt > #0 0x in ?? () > #1 0x77761239 in session_listen (ls=, > sep=sep@entry=0x7fffb557fd50) > at /opt/vpp/src/vnet/session/session_types.h:247 > #2 0x77788b3f in app_listener_alloc_and_init > (app=app@entry=0x7fffb76f7d98, > sep=sep@entry=0x7fffb557fd50, > listener=listener@entry=0x7fffb557fd28) at > /opt/vpp/src/vnet/session/application.c:196 > #3 0x77788ed8 in vnet_listen (a=a@entry=0x7fffb557fd50) at > /opt/vpp/src/vnet/session/application.c:1005 > #4 0x77779e08 in session_mq_listen_handler (data=0x1300787e9) at > /opt/vpp/src/vnet/session/session_node.c:65 > #5 session_mq_listen_handler (data=data@entry=0x1300787e9) at > /opt/vpp/src/vnet/session/session_node.c:36 > #6 0x77bbcdb9 in vl_api_rpc_call_t_handler (mp=0x1300787d0) at > /opt/vpp/src/vlibmemory/vlib_api.c:520 > #7 0x77bc5ead in vl_msg_api_handler_with_vm_node > (am=am@entry=0x7ff
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hello, I have just added an use case over at https://wiki.fd.io/view/VPP/IPSec_and_IKEv2#IPSec_between_VPP_peers.2C_tunneling_IPv4_over_IPv6 It is pretty bare bones for now, but I hope to continue to improve it. Feel free to point out mistakes if there are any. I also will try to write a longer version explaining more things (more like capturing what neale explained to me) with traces in the VPP user docs. Thanks Neale, Filip and everyone. On Fri, May 15, 2020 at 3:11 PM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > *From: * on behalf of Muthu Raj < > muthuraj.muth...@gmail.com> > *Date: *Friday 15 May 2020 at 09:20 > *To: *"Neale Ranns (nranns)" > *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com>, "vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > Sorry about the trace. > > > > Not your fault at all I was commenting that the trace VPP produced was > not clear in indicating the miss. > > > > The match in the SPD is against SA 20’s tunnel addresses not the policy’s > local/remote range. > > > > Thanks for clarifying this. I was confused here. > > > > I created the SA and policy with this in mind and got it to work > successfully. > > Thanks a lot for your help. > > > > Glad to hear it. > > > > I will spend some time this coming week and try to get a small write up > onto https://wiki.fd.io/ > > It may be of help to someone. > > > > I’m sure it will be. Thanks you! > > > > /neale > > > > Muthu > > > > On Thu, May 14, 2020 at 8:52 PM Neale Ranns (nranns) > wrote: > > > > Hi Muthu, > > > > The tracing is not great, but what you see indicates a miss in the SPD. > > The match in the SPD is against SA 20’s tunnel addresses not the policy’s > local/remote range. > > > > /neale > > > > > > *From: *Muthu Raj > *Date: *Thursday 14 May 2020 at 15:51 > *To: *"muthuraj.muth...@gmail.com" > *Cc: *"Neale Ranns (nranns)" , "Filip Tehlar -X > (ftehlar - PANTHEON TECH SRO at Cisco)" , " > vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > So I've since tried out setting SPD on the interface with the IPv6 > address, and even though I am not able to ping the interface, I see that it > does receive and process packets (which I had erroneously assumed it did > not when it became unpingable). > > > > > > I added a new SPD and added a policy like so > > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range - remote-ip-range - > > > > > This is what the trace looks like: > > > > Packet 10 > > 01:02:05:902414: dpdk-input > lan0 rx queue 0 > buffer 0x13daa8: current data 0, length 96, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x9 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 96 > buf_len 2176, data_len 96, ol_flags 0x181, data_off 128, phys_addr > 0xe2f6aa80 > packet_type 0x211 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > 01:02:05:868297: ethernet-input > frame: flags 0x3, hw-if-index 2, sw-if-index 2 > IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67 > 01:02:05:868299: ip6-input > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868299: ipsec6-input-feature > IPSEC_ESP: sa_id 0 spd 2 policy 0 spi 1000 (0x03e8) seq 13693 > 01:02:05:868300: ip6-lookup > fib 0 dpo-idx 8 flow hash: 0x > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868301: ip6-local > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868301: ip6-punt > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868302: error-punt > rx:wan0.67 > 01:02:05:868302: punt > ip6-input: valid ip6 packets > > > > > > IPSEC_ESP: sa_id 0 spd 2 policy 0 spi 1000 (0x03e8) seq 13693 > > > > Here, the spd 2 actually does not have any policy in the 0 index. > > here is what show ipsec spd 2 looks like: > > > > ip6-inbound-protect:
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
Hi Florin, This works fine with single udp listener. I can see connections going on different cores. But, if I run more than one listener then VPP is crashing. Here are the VPP stack traces - (gdb) bt #0 0x in ?? () #1 0x77761239 in session_listen (ls=, sep=sep@entry=0x7fffb557fd50) at /opt/vpp/src/vnet/session/session_types.h:247 #2 0x77788b3f in app_listener_alloc_and_init (app=app@entry=0x7fffb76f7d98, sep=sep@entry=0x7fffb557fd50, listener=listener@entry=0x7fffb557fd28) at /opt/vpp/src/vnet/session/application.c:196 #3 0x77788ed8 in vnet_listen (a=a@entry=0x7fffb557fd50) at /opt/vpp/src/vnet/session/application.c:1005 #4 0x77779e08 in session_mq_listen_handler (data=0x1300787e9) at /opt/vpp/src/vnet/session/session_node.c:65 #5 session_mq_listen_handler (data=data@entry=0x1300787e9) at /opt/vpp/src/vnet/session/session_node.c:36 #6 0x77bbcdb9 in vl_api_rpc_call_t_handler (mp=0x1300787d0) at /opt/vpp/src/vlibmemory/vlib_api.c:520 #7 0x77bc5ead in vl_msg_api_handler_with_vm_node (am=am@entry=0x77dd2ea0 , vlib_rp=, the_msg=0x1300787d0, vm=vm@entry=0x76d7c200 , node=node@entry=0x7fffb553f000, is_private=is_private@entry=0 '\000') at /opt/vpp/src/vlibapi/api_shared.c:609 #8 0x77bafee6 in vl_mem_api_handle_rpc (vm=vm@entry=0x76d7c200 , node=node@entry=0x7fffb553f000) at /opt/vpp/src/vlibmemory/memory_api.c:748 #9 0x77bbd5b3 in vl_api_clnt_process (vm=, node=0x7fffb553f000, f=) at /opt/vpp/src/vlibmemory/vlib_api.c:326 #10 0x76b1b116 in vlib_process_bootstrap (_a=) at /opt/vpp/src/vlib/main.c:1502 #11 0x7602fbfc in clib_calljmp () from /opt/vpp/build-root/build-vpp-native/vpp/lib/libvppinfra.so.20.05 #12 0x7fffb5fa2dd0 in ?? () #13 0x76b1e751 in vlib_process_startup (f=0x0, p=0x7fffb553f000, vm=0x76d7c200 ) at /opt/vpp/src/vppinfra/types.h:133 #14 dispatch_process (vm=0x76d7c200 , p=0x7fffb553f000, last_time_stamp=14118080390223872, f=0x0) at /opt/vpp/src/vlib/main.c:1569 #15 0x004b84e0 in ?? () #16 0x in ?? () thanks, -Raj On Mon, May 25, 2020 at 2:17 PM Florin Coras wrote: > Hi Raj, > > Ow, now you’ve hit the untested part of [2]. Could you try this [3]? > > Regards, > Florin > > [3] https://gerrit.fd.io/r/c/vpp/+/27235 > > On May 25, 2020, at 10:44 AM, Raj Kumar wrote: > > Hi Florin, > > I tried the patches[1] & [2] , but still VCL application is crashing. > However, session is created in VPP. > > vpp# sh session verbose 2 > [0:0][U] 2001:5b0::700:b883:31f:29e:9880:9978-LISTEN > index 0 flags: CONNECTED, OWNS_PORT, LISTEN > Thread 0: active sessions 1 > Thread 1: no sessions > Thread 2: no sessions > Thread 3: no sessions > Thread 4: no sessions > vpp# sh app server > Connection App Wrk > [0:0][U] 2001:5b0::700:b883:31f:29e:udp6_rx[shm] 1 > vpp# > > Here are the VCL application traces. Attached is the updated vppcom.c file. > > [root@J3SGISNCCRO01 vcl_test]# VCL_DEBUG=2 gdb > udp6_server_vcl_threaded_udpc >GNU gdb (GDB) Red Hat Enterprise Linux 8.2-6.el8 > Copyright (C) 2018 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later < > http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > Type "show copying" and "show warranty" for details. > This GDB was configured as "x86_64-redhat-linux-gnu". > Type "show configuration" for configuration details. > For bug reporting instructions, please see: > <http://www.gnu.org/software/gdb/bugs/>. > Find the GDB manual and other documentation resources online at: > <http://www.gnu.org/software/gdb/documentation/>. > > For help, type "help". > Type "apropos word" to search for commands related to "word"... > Reading symbols from udp6_server_vcl_threaded_udpc...(no debugging symbols > found)...done. > (gdb) r 2001:5b0::700:b883:31f:29e:9880 9978 > Starting program: /home/super/vcl_test/udp6_server_vcl_threaded_udpc > 2001:5b0::700:b883:31f:29e:9880 9978 > Missing separate debuginfos, use: yum debuginfo-install > glibc-2.28-72.el8_1.1.x86_64 > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib64/libthread_db.so.1". > > server addr - 2001:5b0::700:b883:31f:29e:9880 > server port 9978, > total port = 1 > VCL<64516>: configured VCL debug level (2) from VCL_DEBUG! > VCL<64516>: allocated VCL heap = 0x7fffe6988010, size 268435456 > (0x1000) > VCL<64516>: configured rx_fifo_s
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
Hi Florin, I am facing a weird problem. After making the VPP code changes, I recompiled/re-installed VPP by using the following commands- make rebuild-release make pkg-rpm rpm -ivh /opt/vpp/build-root/*.rpm But, it looks that VPP is still using the old code I also stopped the VPP service before compiling and installing the new code. Also, recompiled the application using the new vppcom library. But, the line# in the following trace indicates that VPP is using the old code vppcom_session_create:*1279*: vcl<28267:1>: created session 1 May be because of this issue , VPP is still crashing with UDPC. Please let me know if there is any other way to compile VPP with the local code changes. thanks, -Raj . On Tue, May 19, 2020 at 12:31 AM Florin Coras wrote: > Hi Raj, > > By the looks of it, something’s not right because in the logs VCL still > reports it’s binding using UDPC. You probably cherry-picked [1] but it > needs [2] as well. More inline. > > [1] https://gerrit.fd.io/r/c/vpp/+/27111 > [2] https://gerrit.fd.io/r/c/vpp/+/27106 > > On May 18, 2020, at 8:42 PM, Raj Kumar wrote: > > > Hi Florin, > I tried the path [1] , but still VPP is crashing when application is > using listen with UDPC. > > [1] https://gerrit.fd.io/r/c/vpp/+/27111 > > > > On a different topic , I have some questions. Could you please provide > your inputs - > > 1) I am using Mellanox NIC. Any idea how can I enable Tx checksum offload > ( for udp). Also, how to change the Tx burst mode and Rx burst mode to the > Vector . > > HundredGigabitEthernet12/0/1 3 up HundredGigabitEthernet12/0/1 > Link speed: 100 Gbps > Ethernet address b8:83:03:9e:98:81 > * Mellanox ConnectX-4 Family* > carrier up full duplex mtu 9206 > flags: admin-up pmd maybe-multiseg rx-ip4-cksum > rx: queues 4 (max 1024), desc 1024 (min 0 max 65535 align 1) > tx: queues 5 (max 1024), desc 1024 (min 0 max 65535 align 1) > pci: device 15b3:1013 subsystem 1590:00c8 address :12:00.01 numa 0 > switch info: name :12:00.1 domain id 1 port id 65535 > max rx packet len: 65536 > promiscuous: unicast off all-multicast on > vlan offload: strip off filter off qinq off > rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum > vlan-filter >jumbo-frame scatter timestamp keep-crc rss-hash > rx offload active: ipv4-cksum udp-cksum tcp-cksum jumbo-frame scatter > tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum tcp-tso >outer-ipv4-cksum vxlan-tnl-tso gre-tnl-tso > geneve-tnl-tso >multi-segs udp-tnl-tso ip-tnl-tso >* tx offload active: multi-segs* > rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4 > ipv6-tcp-ex >ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other >ipv6-ex ipv6 l4-dst-only l4-src-only l3-dst-only > l3-src-only > rss active:ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4 > ipv6-tcp-ex >ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other >ipv6-ex ipv6 > > * tx burst mode: No MPW + MULTI + TSO + INLINE + METADATArx burst > mode: Scalar* > > > FC: Not sure why (might not be supported) but the offloads are not enabled > in dpdk_lib_init for VNET_DPDK_PMD_MLX* nics. You could try replicating > what’s done for the Intel cards and see if that works. Alternatively, you > might want to try the rdma driver, although I don’t know if it supports > csum offloading (cc Ben and Damjan). > > > 2) My application needs to send routing header (SRv6) and Destination > option extension header. On RedHat 8.1 , I was using socket option to add > routing and destination option extension header. > With VPP , I can use SRv6 policy to let VPP add the routing header. But, I > am not sure if there is any option in VPP or HostStack to add the > destination option header. > > > FC: We don’t currently support this. > > Regards, > Florin > > > > Coming back to the original problem, here are the traces- > > VCL<39673>: configured VCL debug level (2) from VCL_DEBUG! > VCL<39673>: using default heapsize 268435456 (0x1000) > VCL<39673>: allocated VCL heap = 0x7f6b40221010, size 268435456 > (0x1000) > VCL<39673>: using default configuration. > vppcom_connect_to_vpp:487: vcl<39673:0>: app (udp6_rx) connecting to VPP > api (/vpe-api)... > vppcom_connect_to_vpp:502: vcl<39673:0>: app (udp6_rx) is connected to VPP! > vppcom_app_create:1200: vcl<39673:0>: sending session enable > vppcom_app_create:1208: vcl<39673:0>: sending app attach > vppcom_app_create:1217: vcl<3967
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
_rp=, the_msg=0x13007ebe8, vm=vm@entry=0x76d7c200 , node=node@entry=0x7fffb571a000, is_private=is_private@entry=0 '\000') at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlibapi/api_shared.c:609 #8 0x77baff06 in vl_mem_api_handle_rpc (vm=vm@entry=0x76d7c200 , node=node@entry=0x7fffb571a000) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlibmemory/memory_api.c:748 #9 0x77bbd5d3 in vl_api_clnt_process (vm=, node=0x7fffb571a000, f=) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlibmemory/vlib_api.c:326 #10 0x76b1b136 in vlib_process_bootstrap (_a=) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:1502 #11 0x7602fc0c in clib_calljmp () from /lib64/libvppinfra.so.20.05 #12 0x7fffb5e34dd0 in ?? () #13 0x76b1e771 in vlib_process_startup (f=0x0, p=0x7fffb571a000, vm=0x76d7c200 ) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vppinfra/types.h:133 #14 dispatch_process (vm=0x76d7c200 , p=0x7fffb571a000, last_time_stamp=12611933408198086, f=0x0) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:1569 thanks, -Raj On Sat, May 16, 2020 at 8:18 PM Florin Coras wrote: > Hi Raj, > > Inline. > > On May 16, 2020, at 2:30 PM, Raj Kumar wrote: > > Hi Florin, > > I am using VPP 20.05 rc0 . Should I upgrade it ? > > > FC: Not necessarily, as long as it’s relatively recent, i.e., it includes > all of the recent udp updates. > > > Thanks! for providing the patch, i will try it on Monday. Actually, I am > testing in a controlled environment where I can not change the VPP > libraries. I will try it on my server. > > > FC: Sounds good. Let me know how it goes! > > > On the UDP connection; yes, the error EINPROGRESS was there because I am > using non-blocking connection. Now, I am ignoring this error. > Sometime , VPP crashes when I kills my application ( not gracefully) even > when there is a single connection . > > > FC: That might have to do with the app dying such that 1) it does not > detach from vpp (e.g., sigkill and atexit function is not executed) 2) it > dies with the message queue mutex held and 3) vpp tries to enqueue more > events before detecting that it crashed (~30s). > > > The good part is that now I am able to move connections on different cores > by connecting it on receiving the first packet and then re-binding the > socket to listen. > Basically, this approach works but I have not tested it thoroughly. > However , I am still in favor of using the UDPC connection. > > > FC: If you have enough logic in your app to emulate a handshake, i.e., > always have the client send a few bytes and wait for a reply from the > server before opening a new connection, then this approach is probably more > flexible from core placement perspective. > > The patch tries to emulate the old udpc with udp (udpc in vpp was > confusing for consumers). You might get away with listening from multiple > vcl workers on the same udp ip:port pair and have vpp load balance accepts > between them, but I’ve never tested that. You can do this only with udp > listeners that have been initialized as connected (so only with the patch). > > > Btw, in trace logs I see some ssvm_delete related error when re-binding > the connection. > > > FC: I think it’s fine. Going over the interactions step by step to see if > understand what you’re doing (and hopefully help you understand what vpp > does underneath). > > > vpp# sh session verbose > ConnectionState Rx-f > Tx-f > [0:0][U] 2001:5b0::700:b883:31f:29e:9880:6677-LISTEN 0 > 0 > Thread 0: active sessions 1 > > ConnectionState Rx-f > Tx-f > [1:0][U] 2001:5b0::700:b883:31f:29e:9880:6677-OPENED 0 > 0 > Thread 1: active sessions 1 > > ConnectionState Rx-f > Tx-f > [2:0][U] 2001:5b0::700:b883:31f:29e:9880:6677-OPENED 0 > 0 > Thread 2: active sessions 1 > Thread 3: no sessions > Thread 4: no sessions > > VCL<24434>: configured VCL debug level (2) from VCL_DEBUG! > VCL<24434>: using default heapsize 268435456 (0x1000) > VCL<24434>: allocated VCL heap = 0x7f7f18d1b010, size 268435456 > (0x1000) > VCL<24434>: using default configuration. > vppcom_connect_to_vpp:487: vcl<24434:0>: app (udp6_rx) connecting to VPP > api (/vpe-api)... > vppcom_connect_to_vpp:502: vcl<24434:0>: app (udp6_rx) is connected to VPP! > vppcom_app_create:1200: vcl<24434:0>: sending session enable > vppcom_app_create:1208: vcl<24434:0>: sending app attach &g
[vpp-dev] Troubleshooting IPSec in VPP
Hello, I am trying out IPSec on VPP, and used the wiki[1] to create an IPSec tunnel between an AWS instance(remote) and my home. The tunnel was established successfully, and when pinging an IP on the remote side, the icmp req flows over the tunnel, is seen by the remote box, and responded back as well. I also see that the packets indeed end up reaching my home VPP instance - however, they do not reach the last hop. When I run show int, the ipip0 interface does not show the rx counter at all, and when running `show errors` I do not see the counter for the `ipsec4-tun-input` node either. Neither do I see the `esp4-encrypt-tun' counter. My preliminary guess is that it has something to do with the fact that on AWS we cannot see the public IP inside the instance and so that cannot be assigned to the interface itself, so probably the ESP packets are generated with source as the private IP address corresponding to the public IP. With strongswan, we specify an explicit sourceip parameter, like in the snippet below left=1.2.3.4 leftid=1.2.3.4 leftsubnet=172.16.0.0/16 right=4.5.6.7 #AWS public IP rightsourceip=10.6.82.34 #AWS private IP for that public IP, seen inside the instance. rightsubnet=10.6.0.0/16 I am attaching the ikev2 sa as seen from both sides. How would I fix this issue? Any help is appreciated very much. Thanks in advance. This is from the home side. I've changed the IPs on home and remote side. The private IP addresses have been left as it is. vpp# show ikev2 sa iip 1.2.3.4 ispi d8607eea97ac12a9 rip 4.5.6.7 rspi d6726c2768b2420 encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048 nonce i:eb01e6ef107ba7018679bd239e25d4557f2465323caf0d3213b453ca59af3deb r:1d9cb8f11cd69d4b2f73b182028d8aa8854a49bb3c99797f3994575c2994154c SK_d1eee29fff1ff234f1452006a79a7e27787e83331b29954300a70a9d6061f2fde SK_a i:7f86a547c2d9cb2a4035e4926ca6e23c745c6c8c r:04a71f139f2076058ceafb9be73eb359e43bc308 SK_e i:281c47cd100f69a3425031667150d3054124ff887d77a4a1f43fd7dece7486fc r:3f72f8e973ee62962dc9dffd64d80af9e83993acbcd3690adf85044a23310409 SK_p i:79c096024c45499bd43b5d716c56e5152252c433b112195201dd5c4c23a1f1c7 r:fb7e3b35d57b2987bf61f04858a4afaeee10045c6001594f9f2e505b94d950d8 identifier (i) fqdn vpp.aws identifier (r) fqdn vpp.home child sa 0: encr:aes-cbc-256 integ:sha1-96 esn:yes spi(i) 147e7a05 spi(r) de36dcbc SK_e i:31e22be618e3fe60faf935759e75fdc699f743486dd18f07de8b78747d10d229 r:30b10195fdb1cd5b7384a2db92d5a51fd9fab7f6fc7db775957e3dc862d72532 SK_a i:f98f3539966a66afec330c7cdf85fbe2794e01d3 r:9504182eb614d90aa8fe742122ec9d98c1b6e224 traffic selectors (i): 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 - 65535 traffic selectors (r): 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535 iip 1.2.3.4 ispi d8607eea97ac12a9 rip 4.5.6.7 rspi d6726c2768b2420 Here is the AWS side vpp# show ikev2 sa iip 1.2.3.4 ispi a72ae3cef809725c rip rspi b8b7b8ef09266a6d encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048 nonce i:d3e4299761fd93edd3df16456cb0ca9f717f67e57155fa7cb4cd0b9a1d371019 r:e9a33a33b901366438e262d225a418e9489839415562d3e3673107e0d81d830f SK_d7e4f795db87a02c5b4d5ea738945521473f5e449b783f3ac4b954be7716b7909 SK_a i:13639a11b6e96e65dd38d095a87fc1b5ceefdc6b r:97c96809563dfe39c3d2762c1ff1bf0a8fbc3576 SK_e i:114661a058686bd4362d8515ce83a7d7de098af11b08084c407ad51843316135 r:d812542cfa988e6c302fc52d848fb2d7b7321d6c3e77ee04134338a21c0ccba8 SK_p i:a65ea61c70b3cb749dedc205b7715b4c278a4bc630c6508d89a55a00cd00a2cd r:9e23352bac4d21f6f0d2ec8de82e556db3ddaba0ade0c4d664a020da3986d17b identifier (i) fqdn vpp.home identifier (r) fqdn vpp.aws child sa 0: encr:aes-cbc-256 integ:sha1-96 esn:yes spi(i) 31c649f8 spi(r) 967b11c4 SK_e i:6a1b5898746bc922af1beba021768cd6417a0e8a4c555e5544781fee302cf633 r:2035a8be8fae47c284cef445381cef487bcd670bddc31558109c0303bc0f5399 SK_a i:da119e539529803a3d2a883c01a825211c782bd2 r:2330bc2dd9eb3741e3df649bcc3f7e5320fba512 traffic selectors (i): 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 - 65535 traffic selectors (r): 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535 iip 1.2.3.4 ispi a72ae3cef809725c rip rspi b8b7b8ef09266a6d iip 1.2.3.4 ispi d8607eea97ac12a9 rip rspi d6726c2768b2420 encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048 nonce i:eb01e6ef107ba7018679bd239e25d4557f2465323caf0d3213b453ca59af3deb r:1d9cb8f11cd69d4b2f73b182028d8aa8854a49bb3c99797f3994575c2994154c SK_d1eee29fff1ff234f1452006a79a7e27787e83331b29954300a70a9d6061f2fde SK_a i:7f86a547c2d9cb2a4035e4926ca6e23c745c6c8c r:04a71f139f2076058ceafb9be73eb359e43bc308 SK_e
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
_session_listen:1458: vcl<24434:1>: session 16777219: sending vpp
listen request...
*ssvm_delete_shm:205: unlink segment '24177-3': No such file or directory
(errno 2)*
vcl_segment_detach:467: vcl<24434:1>: detached segment 3 handle 0
vcl_session_app_del_segment_handler:863: vcl<24434:1>: Unmapped segment: 0
vcl_session_connected_handler:505: vcl<24434:1>: session 2 [0x1]
connected! rx_fifo 0x224051a80, refcnt 1, tx_fifo 0x224051980, refcnt 1
vcl_session_app_add_segment_handler:855: vcl<24434:1>: mapped new segment
'24177-4' size 134217728
vcl_session_bound_handler:607: vcl<24434:1>: session 3 [0x0]: listen
succeeded!
vppcom_epoll_ctl:2658: vcl<24434:1>: EPOLL_CTL_ADD: vep_sh 16777216, sh
16777219, events 0x1, data 0x!
thanks,
-Raj
On Sat, May 16, 2020 at 2:23 PM Florin Coras wrote:
> Hi Raj,
>
> Assuming you are trying to open more than one connected udp session, does
> this [1] solve the problem (note it's untested)?
>
> To reproduce legacy behavior, this allows you to listen on
> VPPCOM_PROTO_UDPC but that is now converted by vcl into a udp listen that
> propagates with a “connected” flag to vpp. That should result in a udp
> listener that behaves like an “old” udpc listener.
>
> Regards,
> Florin
>
> [1] https://gerrit.fd.io/r/c/vpp/+/27111
>
> On May 16, 2020, at 10:56 AM, Florin Coras via lists.fd.io <
> fcoras.lists=gmail@lists.fd.io> wrote:
>
> Hi Raj,
>
> Are you using master latest/20.05 rc1 or something older? The fact that
> you’re getting a -115 (EINPROGRESS) suggests you might’ve marked the
> connection as “non-blocking” although you created it as blocking. If that’s
> so, the return value is not an error.
>
> Also, how if vpp crashing? Are you by chance trying to open a lot of udp
> connections back to back?
>
> Regards,
> Florin
>
> On May 16, 2020, at 10:23 AM, Raj Kumar wrote:
>
> Hi Florin,
> I tried to connect on receiving the first UDP packet . But, it did not
> work. I am getting error -115 in the application and VPP is crashing.
>
> This is something I tried in the code (udp receiver) -
> sockfd = vppcom_session_create(VPPCOM_PROTO_UDP, 0);
> rv_vpp = vppcom_session_bind (sockfd, );
> if (FD_ISSET(session_idx, ))
> {
> n = vppcom_session_recvfrom(sockfd, (char *)buffer, MAXLINE, 0,
> );
> if(first_pkt)
> rv_vpp = vppcom_session_connect (sockfd, );
> //Here getting rv_vpp as -115
> }
> Please let me know if I am doing something wrong.
>
> Here are the traces -
>
> VCL<16083>: configured VCL debug level (2) from VCL_DEBUG!
> VCL<16083>: using default heapsize 268435456 (0x1000)
> VCL<16083>: allocated VCL heap = 0x7fd255ed2010, size 268435456
> (0x1000)
> VCL<16083>: using default configuration.
> vppcom_connect_to_vpp:487: vcl<16083:0>: app (udp6_rx) connecting to VPP
> api (/vpe-api)...
> vppcom_connect_to_vpp:502: vcl<16083:0>: app (udp6_rx) is connected to VPP!
> vppcom_app_create:1200: vcl<16083:0>: sending session enable
> vppcom_app_create:1208: vcl<16083:0>: sending app attach
> vppcom_app_create:1217: vcl<16083:0>: app_name 'udp6_rx', my_client_index
> 0 (0x0)
>
> vppcom_connect_to_vpp:487: vcl<16083:1>: app (udp6_rx-wrk-1) connecting to
> VPP api (/vpe-api)...
> vppcom_connect_to_vpp:502: vcl<16083:1>: app (udp6_rx-wrk-1) is connected
> to VPP!
> vl_api_app_worker_add_del_reply_t_handler:235: vcl<94:-1>: worker 1
> vpp-worker 1 added
> vcl_worker_register_with_vpp:262: vcl<16083:1>: added worker 1
> vppcom_session_create:1279: vcl<16083:1>: created session 0
> vppcom_session_bind:1426: vcl<16083:1>: session 0 handle 16777216: binding
> to local IPv6 address 2001:5b0::700:b883:31f:29e:9880 port 6677, proto
> UDP
> vppcom_session_listen:1458: vcl<16083:1>: session 16777216: sending vpp
> listen request...
> vcl_session_bound_handler:607: vcl<16083:1>: session 0 [0x0]: listen
> succeeded!
> vppcom_session_connect:1742: vcl<16083:1>: session handle 16777216
> (STATE_CLOSED): connecting to peer IPv6 2001:5b0::700:b883:31f:29e:9886
> port 51190 proto UDP
> udpRxThread started!!! ... rx port = 6677vppcom_session_connect()
> failed ... -115
> vcl_session_cleanup:1300: vcl<16083:1>: session 0 [0x0] closing
> vcl_worker_cleanup_cb:190: vcl<94:-1>: cleaned up worker 1
> vl_client_disconnect:309: peer unresponsive, give up
>
> thanks,
> -Raj
>
>
> On Fri, May 15, 2020 at 8:10 PM Florin Coras
> wrote:
>
>> Hi Raj,
>>
>> There are no explicit vcl apis that allow a udp listener to be switched
>> to connected mode. We might decide to do this at one point
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
Hi Florin,
I tried to connect on receiving the first UDP packet . But, it did not
work. I am getting error -115 in the application and VPP is crashing.
This is something I tried in the code (udp receiver) -
sockfd = vppcom_session_create(VPPCOM_PROTO_UDP, 0);
rv_vpp = vppcom_session_bind (sockfd, );
if (FD_ISSET(session_idx, ))
{
n = vppcom_session_recvfrom(sockfd, (char *)buffer, MAXLINE, 0,
);
if(first_pkt)
rv_vpp = vppcom_session_connect (sockfd, );
//Here getting rv_vpp as -115
}
Please let me know if I am doing something wrong.
Here are the traces -
VCL<16083>: configured VCL debug level (2) from VCL_DEBUG!
VCL<16083>: using default heapsize 268435456 (0x1000)
VCL<16083>: allocated VCL heap = 0x7fd255ed2010, size 268435456 (0x1000)
VCL<16083>: using default configuration.
vppcom_connect_to_vpp:487: vcl<16083:0>: app (udp6_rx) connecting to VPP
api (/vpe-api)...
vppcom_connect_to_vpp:502: vcl<16083:0>: app (udp6_rx) is connected to VPP!
vppcom_app_create:1200: vcl<16083:0>: sending session enable
vppcom_app_create:1208: vcl<16083:0>: sending app attach
vppcom_app_create:1217: vcl<16083:0>: app_name 'udp6_rx', my_client_index 0
(0x0)
vppcom_connect_to_vpp:487: vcl<16083:1>: app (udp6_rx-wrk-1) connecting to
VPP api (/vpe-api)...
vppcom_connect_to_vpp:502: vcl<16083:1>: app (udp6_rx-wrk-1) is connected
to VPP!
vl_api_app_worker_add_del_reply_t_handler:235: vcl<94:-1>: worker 1
vpp-worker 1 added
vcl_worker_register_with_vpp:262: vcl<16083:1>: added worker 1
vppcom_session_create:1279: vcl<16083:1>: created session 0
vppcom_session_bind:1426: vcl<16083:1>: session 0 handle 16777216: binding
to local IPv6 address 2001:5b0::700:b883:31f:29e:9880 port 6677, proto
UDP
vppcom_session_listen:1458: vcl<16083:1>: session 16777216: sending vpp
listen request...
vcl_session_bound_handler:607: vcl<16083:1>: session 0 [0x0]: listen
succeeded!
vppcom_session_connect:1742: vcl<16083:1>: session handle 16777216
(STATE_CLOSED): connecting to peer IPv6 2001:5b0::700:b883:31f:29e:9886
port 51190 proto UDP
udpRxThread started!!! ... rx port = 6677vppcom_session_connect() failed
... -115
vcl_session_cleanup:1300: vcl<16083:1>: session 0 [0x0] closing
vcl_worker_cleanup_cb:190: vcl<94:-1>: cleaned up worker 1
vl_client_disconnect:309: peer unresponsive, give up
thanks,
-Raj
On Fri, May 15, 2020 at 8:10 PM Florin Coras wrote:
> Hi Raj,
>
> There are no explicit vcl apis that allow a udp listener to be switched to
> connected mode. We might decide to do this at one point through a new bind
> api (non-posix like) since we do support this for builtin applications.
>
> However, you now have the option of connecting a bound session. That is,
> on the first received packet on a udp listener, you can grab the peer’s
> address and connect it. Iperf3 in udp mode, which is part of our make test
> infra, does exactly that. Subsequently, it re-binds the port to accept more
> connections. Would that work for you?
>
> Regards,
> Florin
>
> On May 15, 2020, at 4:06 PM, Raj Kumar wrote:
>
> Thanks! Florin,
>
> OK, I understood that I need to change my application to use UDP socket
> and then use vppcom_session_connect().
> This is fine for the UDP client ( sender) .
>
> But ,in UDP Server ( receiver) , I am not sure how to use the
> vppcom_session_connect(). .
> I am using vppcom_session_listen() to listen on the connections and then
> calling vppcom_session_accept() to accept a new connection.
>
> With UDP*C,* I was able to utilize the RSS ( receiver side scaling)
> feature to move the received connections on the different cores /threads.
>
> Just want to confirm if I can achieve the same with UDP.
>
> I will change my application and will update you about the result.
>
> Thanks,
> -Raj
>
>
> On Fri, May 15, 2020 at 5:17 PM Florin Coras
> wrote:
>
>> Hi Raj,
>>
>> We removed udpc transport in vpp. I’ll push a patch that removes it from
>> vcl as well.
>>
>> Calling connect on a udp connection will give you connected semantics
>> now. Let me know if that solves the issue for you.
>>
>> Regards,
>> Florin
>>
>>
>> On May 15, 2020, at 12:15 PM, Raj Kumar wrote:
>>
>> Hi,
>> I am getting segmentation fault in VPP when using VCL VPPCOM_PROTO_UDP*C*
>> socket. This issue is observed with both UDP sender and UDP receiver
>> application.
>>
>> However, both UDP sender and receiver works fine with VPPCOM_PROTO_UDP.
>>
>> Here is the stack trace -
>>
>> (gdb) bt
>> #0 0x in ?? ()
>> #1 0x7775da59 in session_open_vc (app_wrk_index=1,
>> rmt=0x7fffb5e34cc0, opaque=0)
>
Re: [vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
Thanks! Florin, OK, I understood that I need to change my application to use UDP socket and then use vppcom_session_connect(). This is fine for the UDP client ( sender) . But ,in UDP Server ( receiver) , I am not sure how to use the vppcom_session_connect(). . I am using vppcom_session_listen() to listen on the connections and then calling vppcom_session_accept() to accept a new connection. With UDP*C,* I was able to utilize the RSS ( receiver side scaling) feature to move the received connections on the different cores /threads. Just want to confirm if I can achieve the same with UDP. I will change my application and will update you about the result. Thanks, -Raj On Fri, May 15, 2020 at 5:17 PM Florin Coras wrote: > Hi Raj, > > We removed udpc transport in vpp. I’ll push a patch that removes it from > vcl as well. > > Calling connect on a udp connection will give you connected semantics now. > Let me know if that solves the issue for you. > > Regards, > Florin > > > On May 15, 2020, at 12:15 PM, Raj Kumar wrote: > > Hi, > I am getting segmentation fault in VPP when using VCL VPPCOM_PROTO_UDP*C* > socket. This issue is observed with both UDP sender and UDP receiver > application. > > However, both UDP sender and receiver works fine with VPPCOM_PROTO_UDP. > > Here is the stack trace - > > (gdb) bt > #0 0x in ?? () > #1 0x7775da59 in session_open_vc (app_wrk_index=1, > rmt=0x7fffb5e34cc0, opaque=0) > at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vnet/session/session.c:1217 > #2 0x77779257 in session_mq_connect_handler (data=0x7fffb676e7a8) > at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vnet/session/session_node.c:138 > #3 0x77780f48 in session_event_dispatch_ctrl (elt=0x7fffb643f51c, > wrk=0x7fffb650a640) > at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vnet/session/session.h:262 > #4 session_queue_node_fn (vm=, node=, > frame=) > at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vnet/session/session_node.c:1409 > #5 0x76b214c1 in dispatch_node (last_time_stamp=, > frame=0x0, dispatch_state=VLIB_NODE_STATE_POLLING, > type=VLIB_NODE_TYPE_INPUT, node=0x7fffb5a9a980, vm=0x76d7c200 > ) > at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:1235 > #6 vlib_main_or_worker_loop (is_main=1, vm=0x76d7c200 > ) > at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:1815 > #7 vlib_main_loop (vm=0x76d7c200 ) at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:1990 > #8 vlib_main (vm=, vm@entry=0x76d7c200 > , input=input@entry=0x7fffb5e34fa0) > at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:2236 > #9 0x76b61756 in thread0 (arg=140737334723072) at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/unix/main.c:658 > #10 0x7602fc0c in clib_calljmp () from /lib64/libvppinfra.so.20.05 > #11 0x7fffd1e0 in ?? () > #12 0x76b627ed in vlib_unix_main (argc=, > argv=) > at > /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/unix/main.c:730 > > Earlier , I tested this functionality with VPP 20.01 release with the > following patches and it worked perfectly. > https://gerrit.fd.io/r/c/vpp/+/24332 > https://gerrit.fd.io/r/c/vpp/+/24334 > https://gerrit.fd.io/r/c/vpp/+/24462 > > Thanks, > -Raj > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16415): https://lists.fd.io/g/vpp-dev/message/16415 Mute This Topic: https://lists.fd.io/mt/74234856/21656 Mute #vpp-hoststack: https://lists.fd.io/mk?hashtag=vpp-hoststack=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Segmentation fault in VPP 20.05 release when using VCL VPPCOM_PROTO_UDPC #vpp-hoststack
Hi, I am getting segmentation fault in VPP when using VCL VPPCOM_PROTO_UDP *C* socket. This issue is observed with both UDP sender and UDP receiver application. However, both UDP sender and receiver works fine with VPPCOM_PROTO_UDP. Here is the stack trace - (gdb) bt #0 0x in ?? () #1 0x7775da59 in session_open_vc (app_wrk_index=1, rmt=0x7fffb5e34cc0, opaque=0) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vnet/session/session.c:1217 #2 0x77779257 in session_mq_connect_handler (data=0x7fffb676e7a8) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vnet/session/session_node.c:138 #3 0x77780f48 in session_event_dispatch_ctrl (elt=0x7fffb643f51c, wrk=0x7fffb650a640) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vnet/session/session.h:262 #4 session_queue_node_fn (vm=, node=, frame=) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vnet/session/session_node.c:1409 #5 0x76b214c1 in dispatch_node (last_time_stamp=, frame=0x0, dispatch_state=VLIB_NODE_STATE_POLLING, type=VLIB_NODE_TYPE_INPUT, node=0x7fffb5a9a980, vm=0x76d7c200 ) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:1235 #6 vlib_main_or_worker_loop (is_main=1, vm=0x76d7c200 ) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:1815 #7 vlib_main_loop (vm=0x76d7c200 ) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:1990 #8 vlib_main (vm=, vm@entry=0x76d7c200 , input=input@entry=0x7fffb5e34fa0) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/main.c:2236 #9 0x76b61756 in thread0 (arg=140737334723072) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/unix/main.c:658 #10 0x7602fc0c in clib_calljmp () from /lib64/libvppinfra.so.20.05 #11 0x7fffd1e0 in ?? () #12 0x76b627ed in vlib_unix_main (argc=, argv=) at /usr/src/debug/vpp-20.05-rc0~748_g83d129837.x86_64/src/vlib/unix/main.c:730 Earlier , I tested this functionality with VPP 20.01 release with the following patches and it worked perfectly. https://gerrit.fd.io/r/c/vpp/+/24332 https://gerrit.fd.io/r/c/vpp/+/24334 https://gerrit.fd.io/r/c/vpp/+/24462 Thanks, -Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16410): https://lists.fd.io/g/vpp-dev/message/16410 Mute This Topic: https://lists.fd.io/mt/74234856/21656 Mute #vpp-hoststack: https://lists.fd.io/mk?hashtag=vpp-hoststack=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Neale, Sorry about the trace. The match in the SPD is against SA 20’s tunnel addresses not the policy’s > local/remote range. Thanks for clarifying this. I was confused here. I created the SA and policy with this in mind and got it to work successfully. Thanks a lot for your help. I will spend some time this coming week and try to get a small write up onto https://wiki.fd.io/ It may be of help to someone. Muthu On Thu, May 14, 2020 at 8:52 PM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > The tracing is not great, but what you see indicates a miss in the SPD. > > The match in the SPD is against SA 20’s tunnel addresses not the policy’s > local/remote range. > > > > /neale > > > > > > *From: *Muthu Raj > *Date: *Thursday 14 May 2020 at 15:51 > *To: *"muthuraj.muth...@gmail.com" > *Cc: *"Neale Ranns (nranns)" , "Filip Tehlar -X > (ftehlar - PANTHEON TECH SRO at Cisco)" , " > vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > So I've since tried out setting SPD on the interface with the IPv6 > address, and even though I am not able to ping the interface, I see that it > does receive and process packets (which I had erroneously assumed it did > not when it became unpingable). > > > > > > I added a new SPD and added a policy like so > > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range - remote-ip-range - > > > > > This is what the trace looks like: > > > > Packet 10 > > 01:02:05:902414: dpdk-input > lan0 rx queue 0 > buffer 0x13daa8: current data 0, length 96, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x9 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 96 > buf_len 2176, data_len 96, ol_flags 0x181, data_off 128, phys_addr > 0xe2f6aa80 > packet_type 0x211 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > 01:02:05:868297: ethernet-input > frame: flags 0x3, hw-if-index 2, sw-if-index 2 > IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67 > 01:02:05:868299: ip6-input > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868299: ipsec6-input-feature > IPSEC_ESP: sa_id 0 spd 2 policy 0 spi 1000 (0x03e8) seq 13693 > 01:02:05:868300: ip6-lookup > fib 0 dpo-idx 8 flow hash: 0x > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868301: ip6-local > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868301: ip6-punt > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868302: error-punt > rx:wan0.67 > 01:02:05:868302: punt > ip6-input: valid ip6 packets > > > > > > IPSEC_ESP: sa_id 0 spd 2 policy 0 spi 1000 (0x03e8) seq 13693 > > > > Here, the spd 2 actually does not have any policy in the 0 index. > > here is what show ipsec spd 2 looks like: > > > > ip6-inbound-protect: >[4] priority 100 action protect type ip6-inbound-protect protocol any > sa 20 > local addr range 2001::2 - 2001::2 port range 0 - 65535 > remote addr range 2001::1 - 2001::1 port range 0 - 65535 > packets 0 bytes 0 > > > > Is there a mistake in the way SPD has been added? > > Or is something else the issue? > > > > Here is trace as seen by the sender: > > > > Packet 1 > > > 20:48:33:279228: dpdk-input > eth0 rx queue 0 > buffer 0x8ee28: current data 0, length 98, buffer-pool 0, ref-count 1, > totlen-nifb 0, trace handle 0x0 > ext-hdr-valid > l4-cksum-computed l4-cksum-correct > PKT MBUF: port 1, nb_segs 1, pkt_len 98 > buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr > 0xb3fb8a80 > packet_type 0x10 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0xed0f5862 fdir.hi 0x0 fdir.lo 0xed0f5862 > Packet Types > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: 12:6a:27:79:97:3c -> 12:71:86:3a:04:7b > ICMP: 10.6.33.33 -> 172.30.0.2 > tos 0x00, ttl 64, length 84, checksum 0xf51f dscp CS0 ecn NON_ECN > fragment id 0x6e42, flags DONT_FRAGMENT > ICMP echo_request checksum 0x15b1 > 20:48:33:279234: ethernet-input > frame: flags
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
checksum 0xf51f dscp CS0 ecn NON_ECN fragment id 0x6e42, flags DONT_FRAGMENT ICMP echo_request checksum 0x15b1 20:48:33:279242: ip4-rewrite tx_sw_if_index 3 dpo-idx 6 : ipv4 via 11.11.11.10 loop0: mtu:9000 next:4 246e969ce5 dfdead0800 flow hash: 0x : 246e969ce5dfdead080045546e4240003f01f61f0a062121ac1e 0020: 0002080015b178da0011bd5ec60f05001011 20:48:33:279244: ipsec4-output-feature spd 1 policy 2 20:48:33:279246: esp4-encrypt esp: sa-index 0 spi 1000 (0x03e8) seq 13028 sa-seq-hi 0 crypto aes-cbc-128 inte grity sha1-96 20:48:33:279255: ip6-load-balance fib 3 dpo-idx 3 flow hash: 0x IPSEC_ESP: 2001::1 -> 2001::2 tos 0x00, flow label 0x0, hop limit 254, payload length 132 20:48:33:279258: ip6-rewrite tx_sw_if_index 1 adj-idx 3 : ipv6 via fe80::1043:12ff:fec4:2197 wan0: mtu:9000 next :3 124312c4219712273c81f8f386dd flow hash: 0x : 124312c4219712273c81f8f386dd608432fd26001f18254c9301e507 0020: 7624cf3f7184260410c1001003e832e42112 0040: 6acf95626ff015f514ce23230e5cab93ba24df0ade533f87b96bc02856a7f0e7 0060: df4922f832163a63346bb518f1308c757bbda288b750bd2cae26732a 20:48:33:279258: wan0-output wan0 IP6: 12:27:3c:81:f8:f3 -> 12:43:12:c4:21:97 IPSEC_ESP: 2001::1 -> 2001::2 tos 0x00, flow label 0x0, hop limit 253, payload length 132 20:48:33:279260: wan0-tx wan0 tx queue 0 buffer 0x8ee28: current data -64, length 186, buffer-pool 0, ref-count 1, trace han dle 0x0 ext-hdr-valid l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 l3-hdr-offset 14 PKT MBUF: port 1, nb_segs 1, pkt_len 186 buf_len 2176, data_len 186, ol_flags 0x0, data_off 64, phys_addr 0xb3fb8a80 packet_type 0x10 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0xed0f5862 fdir.hi 0x0 fdir.lo 0xed0f5862 Packet Types RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers IP6: 12:27:3c:81:f8:f3 -> 12:43:12:c4:21:97 IPSEC_ESP: 2001::1 -> 2001::2 tos 0x00, flow label 0x0, hop limit 253, payload length 132 Let me know if you need more info. Thanks, Muthu On Wed, May 13, 2020 at 9:15 PM Muthu Raj via lists.fd.io wrote: > Hi Neale, > > Thanks a lot for looking into this and providing a fix, much appreciated. > > I have applied the fix, and that solved the issue on the sender side. > The packet successfully reached the other side. However, the issue is on > the other side now. > > The packet trace looks like this: (I've put dummy local and remote IPv6 > IPs) > > Packet 12 > > 02:06:48:110231: dpdk-input > wan0 rx queue 0 > buffer 0x11e885: current data 0, length 74, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0xb >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 1, nb_segs 1, pkt_len 74 > buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr > 0xe27a21c0 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 1, nb_segs 1, pkt_len 190 > buf_len 2176, data_len 190, ol_flags 0x181, data_off 128, phys_addr > 0xe279a300 > packet_type 0x41 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 67 > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV6 (0x0040) IPv6 packet without extension headers > IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67 > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 02:06:48:074127: ethernet-input > frame: flags 0x3, hw-if-index 2, sw-if-index 2 > IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67 > 02:06:48:074128: ip6-input > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 02:06:48:074128: ip6-lookup > fib 0 dpo-idx 8 flow hash: 0x > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 02:06:48:074129: ip6-local > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 02:06:48:074130: ip6-punt > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
interface, that gets all the packets destined for wan0.67 (may be through an l2 bridge?) and SPD should be there. Am I in the right path? If yes, could you direct me to documentation regarding oneway L2 bridging to a virtual / loopback interface? Thanks so much for looking into this, once again. Thanks, Muthu On Wed, May 13, 2020 at 2:54 PM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > I tried your 4over6 config, it doesn’t work… here’s the fix: > > https://gerrit.fd.io/r/c/vpp/+/27019 > > > > /neale > > > > *From: *Muthu Raj > *Date: *Monday 11 May 2020 at 21:36 > *To: *"Neale Ranns (nranns)" > *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com>, "vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > Thank you so much for patiently explaining. I initially added next-hop > like this - > > ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218 > > > But it still got dropped with ip4-arp: ARP requests sent. > > > > Then I read your message again, and realised that the ARP response really > doesn't matter. So I set a static ARP entry like so - > > set ip neighbor lan0.218 172.30.0.6 c8:5b:76:d9:7f:9c > > > > That thankfully, changed the nodes visited. Unfortunately, it still fails > with an error like below - ipsec6-no-such-tunnel. > > I understand that it must be some mistake in setting up the tunnel, but > once again, I am not sure how to proceed. > > > > The packet trace is at the end. > > > > Here is the way I set-up the tunnel: > > > > ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec spd add 1 > set interface ipsec spd lan0.218 1 > ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 > ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 > ipsec policy add spd 1 priority 10 outbound action protect sa 10 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218 > > set ip neighbor lan0.218 172.30.0.6 24:6e:96:9c:e5:df > > > > *Packet trace:* > > > 00:01:35:388393: dpdk-input > lan0 rx queue 0 > buffer 0x1389c3: current data 0, length 102, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x10 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 102 > buf_len 2176, data_len 102, ol_flags 0x181, data_off 128, phys_addr > 0xe2e27140 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN > fragment id 0x91cf, flags DONT_FRAGMENT > ICMP echo_request checksum 0x15bb > 00:01:35:388395: ethernet-input > frame: flags 0x3, hw-if-index 1, sw-if-index 1 > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > 00:01:35:388396: ip4-input > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN > fragment id 0x91cf, flags DONT_FRAGMENT > ICMP echo_request checksum 0x15bb > 00:01:35:388397: ip4-lookup > fib 0 dpo-idx 6 flow hash: 0x > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN > fragment id 0x91cf, flags DONT_FRAGMENT > ICMP echo_request checksum 0x15bb > 00:01:35:388397: ip4-rewrite > tx_sw_if_index 4 dpo-idx 6 : ipv4 via 172.30.0.6 lan0.218: mtu:9000 > 246e969ce5dfb4969118ebbc81da0800 flow ha > sh: 0x > : > 246e969ce5dfb4969118ebbc81da0800455491cf40003f01e891ac1e > 0020: 00020a060b22080015bb4fa504f9a1a5b95e00
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Neale, Thank you so much for patiently explaining. I initially added next-hop like this - ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218 But it still got dropped with ip4-arp: ARP requests sent. Then I read your message again, and realised that the ARP response really doesn't matter. So I set a static ARP entry like so - set ip neighbor lan0.218 172.30.0.6 c8:5b:76:d9:7f:9c That thankfully, changed the nodes visited. Unfortunately, it still fails with an error like below - ipsec6-no-such-tunnel. I understand that it must be some mistake in setting up the tunnel, but once again, I am not sure how to proceed. The packet trace is at the end. Here is the way I set-up the tunnel: ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 integ-key integ-alg sha1-96 tunnel-src tunnel-dst ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 integ-key integ-alg sha1-96 tunnel-src tunnel-dst ipsec spd add 1 set interface ipsec spd lan0.218 1 ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 ipsec policy add spd 1 priority 10 outbound action protect sa 10 local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - 10.6.255.255 ipsec policy add spd 1 priority 10 inbound action protect sa 20 local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - 10.6.255.255 ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218 set ip neighbor lan0.218 172.30.0.6 24:6e:96:9c:e5:df *Packet trace:* 00:01:35:388393: dpdk-input lan0 rx queue 0 buffer 0x1389c3: current data 0, length 102, buffer-pool 1, ref-count 1, totlen-nifb 0, trace handle 0x10 ext-hdr-valid l4-cksum-computed l4-cksum-correct PKT MBUF: port 0, nb_segs 1, pkt_len 102 buf_len 2176, data_len 102, ol_flags 0x181, data_off 128, phys_addr 0xe2e27140 packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0 Packet Offload Flags PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 Packet Types RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN fragment id 0x91cf, flags DONT_FRAGMENT ICMP echo_request checksum 0x15bb 00:01:35:388395: ethernet-input frame: flags 0x3, hw-if-index 1, sw-if-index 1 IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 00:01:35:388396: ip4-input ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN fragment id 0x91cf, flags DONT_FRAGMENT ICMP echo_request checksum 0x15bb 00:01:35:388397: ip4-lookup fib 0 dpo-idx 6 flow hash: 0x ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN fragment id 0x91cf, flags DONT_FRAGMENT ICMP echo_request checksum 0x15bb 00:01:35:388397: ip4-rewrite tx_sw_if_index 4 dpo-idx 6 : ipv4 via 172.30.0.6 lan0.218: mtu:9000 246e969ce5dfb4969118ebbc81da0800 flow ha sh: 0x : 246e969ce5dfb4969118ebbc81da0800455491cf40003f01e891ac1e 0020: 00020a060b22080015bb4fa504f9a1a5b95e67cf0c00 00:01:35:388397: ipsec4-output-feature spd 1 policy 2 00:01:35:388398: esp4-encrypt esp: sa-index 0 spi 1000 (0x03e8) seq 11 sa-seq-hi 0 crypto aes-cbc-128 integrity sha1-96 00:01:35:388401: punt-dispatch reason: [2] ipsec6-no-such-tunnel 00:01:35:388403: drop punt-dispatch: No registrations Thanks again, Muthu On Tue, May 12, 2020 at 12:24 AM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > Your lan0.218 has address in the 172.16.0.5/16, set the next hop to the > peer on that link in that subnet. If you don’t have one, in this case, you > can use any address… the problem is that IPSec SPD runs as an output > feature, but we don’t run features for packets that hit the glean (i.e. I > need to ARP) adjacency. So the route needs to resolve via a neighbour > adjacency. It doesn’t have to be a complete adj, hence you don’t need an > ARP response, so any address will do. > > > > /neale > > > > *From: * on behalf of Muthu Raj < > muthuraj.muth...@gmail.com> > *Date: *Monday 11 May 2020 at 19:22 > *To: *"Neale Ranns (nranns)" > *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com>, "vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > Thanks
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Neale, Thanks for responding so quickly. It might sound a little bit pedestrian, but what would be the next hop in this case? Would it be the IPv6 IP (over which the tunnel is established) ? I have added it through the lan0.218 interface with the assumption that the sa will kick in and send over the tunnel. On Mon, May 11, 2020, 10:26 PM Neale Ranns (nranns) wrote: > > > > > *From: *Muthu Raj > *Date: *Monday 11 May 2020 at 18:42 > *To: *"Neale Ranns (nranns)" > *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com>, "vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > The 10.6.0.0/16 is actually on the remote side, and should flow over the > IPv6 IPsec tunnel. > > I was hoping by adding route via the lan0.218 interface, the spd will kick > in and send over the tunnel. I guess I'm missing something very glaring, so > would appreciate pointers. > > > > You’re missing next-hop for the route. > > > > /neale > > > > The source of the traced packet is actually another node, that sends to > the lan0.218 IP for the remote subnet destination. > > > > > > Thanks, > > Muthu > > > > On Mon, May 11, 2020, 9:41 PM Neale Ranns (nranns) > wrote: > > > > Hi Muthu, > > > > Your lan0 interface is not point-2-point, so your route needs a nexthop; > >ip route add 10.6.0.0/16 via 172.30.x.y lan0.218 > > > > x and y can be anything, apart from x=0 and y=5. > > > > /neale > > > > *From: * on behalf of Muthu Raj < > muthuraj.muth...@gmail.com> > *Date: *Monday 11 May 2020 at 17:39 > *To: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com> > *Cc: *"vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Filip, > > > > I have traced the packet. > > > > Here is how I setup the tunnel > > > > || <===> || Subnet=10.6.0.0/16> > > > > ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec spd add 1 > set interface ipsec spd lan0.218 1 > ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 > ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 > ipsec policy add spd 1 priority 10 outbound action protect sa 10 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > > ip route add 10.6.0.0/16 via lan0.218 > > > > vpp# show int address > lan0 (up): > lan0.218 (up): > L3 172.30.0.5/16 > local0 (up): > wan0 (up): > wan0.67 (up): > L3 > > > > Corresponding config on remote side. > > Trace: > > > > 00:32:40:825694: dpdk-input > lan0 rx queue 0 > buffer 0x13e1d1: current data 0, length 74, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x11 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 74 > buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr > 0xe05874c0 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712830: ethernet-input > frame: flags 0x3, hw-if-index 1, sw-if-index 1 > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > 00:32:40:712831: ip4-input > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2,
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Neale, The 10.6.0.0/16 is actually on the remote side, and should flow over the IPv6 IPsec tunnel. I was hoping by adding route via the lan0.218 interface, the spd will kick in and send over the tunnel. I guess I'm missing something very glaring, so would appreciate pointers. The source of the traced packet is actually another node, that sends to the lan0.218 IP for the remote subnet destination. Thanks, Muthu On Mon, May 11, 2020, 9:41 PM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > Your lan0 interface is not point-2-point, so your route needs a nexthop; > >ip route add 10.6.0.0/16 via 172.30.x.y lan0.218 > > > > x and y can be anything, apart from x=0 and y=5. > > > > /neale > > > > *From: * on behalf of Muthu Raj < > muthuraj.muth...@gmail.com> > *Date: *Monday 11 May 2020 at 17:39 > *To: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com> > *Cc: *"vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Filip, > > > > I have traced the packet. > > > > Here is how I setup the tunnel > > > > || <===> || Subnet=10.6.0.0/16> > > > > ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec spd add 1 > set interface ipsec spd lan0.218 1 > ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 > ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 > ipsec policy add spd 1 priority 10 outbound action protect sa 10 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > > ip route add 10.6.0.0/16 via lan0.218 > > > > vpp# show int address > lan0 (up): > lan0.218 (up): > L3 172.30.0.5/16 > local0 (up): > wan0 (up): > wan0.67 (up): > L3 > > > > Corresponding config on remote side. > > Trace: > > > > 00:32:40:825694: dpdk-input > lan0 rx queue 0 > buffer 0x13e1d1: current data 0, length 74, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x11 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 74 > buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr > 0xe05874c0 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712830: ethernet-input > frame: flags 0x3, hw-if-index 1, sw-if-index 1 > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > 00:32:40:712831: ip4-input > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712831: ip4-lookup > fib 0 dpo-idx 3 flow hash: 0x > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712832: ip4-glean > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712833: ip4-drop > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712833: error-drop > rx:lan0.218 > 00:32:40:712834: drop > ip4-glean: ARP requests sent > > > > I believe for some reason
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Filip, I have traced the packet. Here is how I setup the tunnel || <===> || ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 integ-key integ-alg sha1-96 tunnel-src tunnel-dst ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 integ-key integ-alg sha1-96 tunnel-src tunnel-dst ipsec spd add 1 set interface ipsec spd lan0.218 1 ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 ipsec policy add spd 1 priority 10 outbound action protect sa 10 local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - 10.6.255.255 ipsec policy add spd 1 priority 10 inbound action protect sa 20 local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - 10.6.255.255 ip route add 10.6.0.0/16 via lan0.218 vpp# show int address lan0 (up): lan0.218 (up): L3 172.30.0.5/16 local0 (up): wan0 (up): wan0.67 (up): L3 Corresponding config on remote side. Trace: 00:32:40:825694: dpdk-input lan0 rx queue 0 buffer 0x13e1d1: current data 0, length 74, buffer-pool 1, ref-count 1, totlen-nifb 0, trace handle 0x11 ext-hdr-valid l4-cksum-computed l4-cksum-correct PKT MBUF: port 0, nb_segs 1, pkt_len 74 buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr 0xe05874c0 packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0 Packet Offload Flags PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 Packet Types RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712830: ethernet-input frame: flags 0x3, hw-if-index 1, sw-if-index 1 IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 00:32:40:712831: ip4-input ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712831: ip4-lookup fib 0 dpo-idx 3 flow hash: 0x ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712832: ip4-glean ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712833: ip4-drop ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712833: error-drop rx:lan0.218 00:32:40:712834: drop ip4-glean: ARP requests sent I believe for some reason the policy match is not happening. And I am not able to find examples of IPv4 over IPv6 tunnel, so I might be making a mistake in setting up the tunnel itself. Let me know if you need more info, and thank you so much for looking into this. Much appreciated. Thanks, Muthu On Mon, May 11, 2020 at 3:41 PM Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) wrote: > Hi Muthu, > > you can enable packet tracing [1] and see what nodes are visited. > > Filip > > [1] > https://wiki.fd.io/view/VPP/How_To_Use_The_Packet_Generator_and_Packet_Tracer#Step_3._Examine_the_setup_script > VPP/How To Use The Packet Generator and Packet Tracer - fd.io > <https://wiki.fd.io/view/VPP/How_To_Use_The_Packet_Generator_and_Packet_Tracer#Step_3._Examine_the_setup_script> > Introduction. The VPP platform includes packet generation and packet > tracing facilities. The following example shows steps that you might > typically use to run a debug version of the vpp executable file, generate > packets, and to analyze results. > wiki.fd.io > > -- > *From:* Muthu Raj > *Sent:* Thursday, May 7, 2020 7:04 PM > *To:* Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) < > fteh...@cisco.com> > *Cc:* vpp-dev@lists.fd.io > *Subject:* Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > Hello Filip, > > I just tried adding a loopback interface and setting the SPD on that > interface. > Here is how I tried it: > > create loopback interface > set int state loop0 up > set interface ip table loop0 0 > set int ip address loop0 11.11.11.11/32 > ipse
Re: [vpp-dev] Multiple UDP receiver applications on same port #vpp-hoststack
Hi Florin, Thanks for the clarification. Thanks, -Raj On Wed, Feb 26, 2020 at 5:14 PM Florin Coras wrote: > Hi Raj, > > Now that’s interesting. VPP detects when an application dies (binary api > mechanism) and forces through the session layer a de-attach which in turn > leads to an unbind. > > In case of udp, not tcp, we have a shim layer that redirects udp packets > to whomever registered for a certain port. In your case, udp-input was > registered twice but because currently we don’t have a reference count, the > unbind removes the registration for both binds. > > Will add it to my todo list if nobody beats me to it. > > Regards, > Florin > > On Feb 26, 2020, at 1:28 PM, Raj Kumar wrote: > > Hi, > When 2 or more UDP rx applications ( using VCL) receiving on the same port > ( bind on the same port but different ip address) then on stopping either > one of the application , all other application stopped receiving the > traffic. As soon as , I restart the application all other applications also > start receiving the traffic. > > vpp# sh ip6 int > > vppnet1.2001 is admin up > > Local unicast address(es): > > fd0d:edc4::2001::213/64 > > fd0d:edc4::2001::223/64 > > Link-local address(es): > > fe80::ba83:3ff:fe79:af8c > When both applications are running : - > vpp# sh session verbose 1 > ConnectionState Rx-f > Tx-f > [#0][U] fd0d:edc4::2001::213:9915->:::0 - 0 > 0 > [#0][U] fd0d:edc4::2001::223:9915->:::0 - 0 > 0 > Thread 0: active sessions 2 > Thread 1: no sessions > Thread 2: no sessions > Thread 3: no sessions > Thread 4: no sessions > > ConnectionState Rx-f > Tx-f > [#5][U] fd0d:edc4::2001::213:9915->fd0d:edc4:f- 15226 > 0 > Thread 5: active sessions 1 > > ConnectionState Rx-f > Tx-f > [#6][U] fd0d:edc4::2001::223:9915->fd0d:edc4:f- 0 > 0 > Thread 6: active sessions 1 > > > On stopping first application : - > > vpp# sh session verbose 1 > > ConnectionState Rx-f > Tx-f > > [#0][U] fd0d:edc4::2001::223:9915->:::0 - 0 > 0 > > Thread 0: active sessions 1 > > Thread 1: no sessions > > Thread 2: no sessions > > Thread 3: no sessions > > Thread 4: no sessions > > Thread 5: no sessions > > > ConnectionState Rx-f > Tx-f > > [#6][U] fd0d:edc4::2001::223:9915->fd0d:edc4:f- 0 > 0 > > > Thread 6: active sessions 1 > One active session is there but in 'sh err' "no listener punt" error > increments and application is not receiving the data. > > 310150540 ip6-udp-lookup no listener punt > > packet trace :- > > --- Start of thread 6 vpp_wk_5 --- > Packet 1 > > 01:12:04:676114: dpdk-input > vppnet1 rx queue 2 > buffer 0x10b18f: current data 0, length 7634, buffer-pool 0, ref-count > 1, totlen-nifb 0, trace handle 0x600 >ext-hdr-valid > PKT MBUF: port 0, nb_segs 1, pkt_len 7634 > buf_len 9344, data_len 7634, ol_flags 0x182, data_off 128, phys_addr > 0x744c6440 > packet_type 0x2e1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x678c0829 fdir.hi 0x0 fdir.lo 0x678c0829 > Packet Offload Flags > PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV6_EXT_UNKNOWN (0x00e0) IPv6 packet with or without > extension headers > RTE_PTYPE_L4_UDP (0x0200) UDP packet > IP6: b8:83:03:79:9f:e8 -> b8:83:03:79:af:8c 802.1q vlan 2001 > UDP: fd0d:edc4::2001::104 -> fd0d:edc4::2001::223 > tos 0x00, flow label 0x0, hop limit 64, payload length 7576 > UDP: 23456 -> 9915 > length 7576, checksum 0x225d > 01:12:04:676154: ethernet-input > frame: flags 0x3, hw-if-index 2, sw-if-index 2 > IP6: b8:83:03:79:9f:e8 -> b8:83:03:79:af:8c 802.1q vlan 2001 > 01:12:04:676156: ip6-input > UDP: fd0d:edc4::2001::104 -> fd0d:edc4::2001::223 > tos 0x00, flow label 0x0, hop limit 64, payload length 7576 > UDP: 23456 -> 9915 > length 7576, checksum 0x225d > 01:12:04:676164: ip6-lookup > fib 0 dpo-idx 20 flow hash: 0x
[vpp-dev] Multiple UDP receiver applications on same port #vpp-hoststack
Hi, When 2 or more UDP rx applications ( using VCL) receiving on the same port ( bind on the same port but different ip address) then on stopping either one of the application , all other application stopped receiving the traffic. As soon as , I restart the application all other applications also start receiving the traffic. vpp# sh ip6 int vppnet1.2001 is admin up Local unicast address(es): fd0d:edc4::2001::213/64 fd0d:edc4::2001::223/64 Link-local address(es): fe80::ba83:3ff:fe79:af8c When both applications are running : - vpp# sh session verbose 1 Connection State Rx-f Tx-f [#0][U] fd0d:edc4::2001::213:9915->:::0 - 0 0 [#0][U] fd0d:edc4::2001::223:9915->:::0 - 0 0 Thread 0: active sessions 2 Thread 1: no sessions Thread 2: no sessions Thread 3: no sessions Thread 4: no sessions Connection State Rx-f Tx-f [#5][U] fd0d:edc4::2001::213:9915->fd0d:edc4:f- 15226 0 Thread 5: active sessions 1 Connection State Rx-f Tx-f [#6][U] fd0d:edc4::2001::223:9915->fd0d:edc4:f- 0 0 Thread 6: active sessions 1 On stopping first application : - vpp# sh session verbose 1 Connection State Rx-f Tx-f [#0][U] fd0d:edc4::2001::223:9915->:::0 - 0 0 Thread 0: active sessions 1 Thread 1: no sessions Thread 2: no sessions Thread 3: no sessions Thread 4: no sessions Thread 5: no sessions Connection State Rx-f Tx-f [#6][U] fd0d:edc4::2001::223:9915->fd0d:edc4:f- 0 0 Thread 6: active sessions 1 One active session is there but in 'sh err' "no listener punt" error increments and application is not receiving the data. 310150540 ip6-udp-lookup no listener punt packet trace :- --- Start of thread 6 vpp_wk_5 --- Packet 1 01:12:04:676114: dpdk-input vppnet1 rx queue 2 buffer 0x10b18f: current data 0, length 7634, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x600 ext-hdr-valid PKT MBUF: port 0, nb_segs 1, pkt_len 7634 buf_len 9344, data_len 7634, ol_flags 0x182, data_off 128, phys_addr 0x744c6440 packet_type 0x2e1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x678c0829 fdir.hi 0x0 fdir.lo 0x678c0829 Packet Offload Flags PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid Packet Types RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet RTE_PTYPE_L3_IPV6_EXT_UNKNOWN (0x00e0) IPv6 packet with or without extension headers RTE_PTYPE_L4_UDP (0x0200) UDP packet IP6: b8:83:03:79:9f:e8 -> b8:83:03:79:af:8c 802.1q vlan 2001 UDP: fd0d:edc4::2001::104 -> fd0d:edc4::2001::223 tos 0x00, flow label 0x0, hop limit 64, payload length 7576 UDP: 23456 -> 9915 length 7576, checksum 0x225d 01:12:04:676154: ethernet-input frame: flags 0x3, hw-if-index 2, sw-if-index 2 IP6: b8:83:03:79:9f:e8 -> b8:83:03:79:af:8c 802.1q vlan 2001 01:12:04:676156: ip6-input UDP: fd0d:edc4::2001::104 -> fd0d:edc4::2001::223 tos 0x00, flow label 0x0, hop limit 64, payload length 7576 UDP: 23456 -> 9915 length 7576, checksum 0x225d 01:12:04:676164: ip6-lookup fib 0 dpo-idx 20 flow hash: 0x UDP: fd0d:edc4::2001::104 -> fd0d:edc4::2001::223 tos 0x00, flow label 0x0, hop limit 64, payload length 7576 UDP: 23456 -> 9915 length 7576, checksum 0x225d 01:12:04:676166: ip6-local UDP: fd0d:edc4::2001::104 -> fd0d:edc4::2001::223 tos 0x00, flow label 0x0, hop limit 64, payload length 7576 UDP: 23456 -> 9915 length 7576, checksum 0x225d 01:12:04:676169: ip6-udp-lookup UDP: src-port 23456 dst-port 9915 01:12:04:676169: ip6-punt UDP: fd0d:edc4::2001::104 -> fd0d:edc4::2001::223 tos 0x00, flow label 0x0, hop limit 64, payload length 7576 UDP: 23456 -> 9915 length 7576, checksum 0x225d 01:12:04:676169: error-punt rx:vppnet1.2001 01:12:04:676170: punt ip6-udp-lookup: no listener punt thanks, -Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15570): https://lists.fd.io/g/vpp-dev/message/15570 Mute This Topic: https://lists.fd.io/mt/71574152/21656 Mute #vpp-hoststack: https://lists.fd.io/mk?hashtag=vpp-hoststack=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] #vpp-hoststack - Issue with UDP receiver application using VCL library
fff:2001::203:7729->fd0d:edc4:ESTABLISHED0 0
[1:2][T] fd0d:edc4::2001::203:39216->fd0d:edc4ESTABLISHED0
399
Thread 1: active sessions 3
ConnectionState Rx-f
Tx-f
[2:0][T] fd0d:edc4::2001::203:51962->fd0d:edc4ESTABLISHED0 0
[2:1][T] fd0d:edc4::2001::203:56849->fd0d:edc4ESTABLISHED0
399
[2:2][T] fd0d:edc4::2001::203:6689->fd0d:edc4:ESTABLISHED0 0
[2:3][T] fd0d:edc4::2001::203:7729->fd0d:edc4:ESTABLISHED0 0
Thread 2: active sessions 4
ConnectionState Rx-f
Tx-f
[3:0][T] fd0d:edc4::2001::203:29141->fd0d:edc4ESTABLISHED0 0
[3:1][T] fd0d:edc4::2001::203:6689->fd0d:edc4:ESTABLISHED0 0
Thread 3: active sessions 2
ConnectionState Rx-f
Tx-f
[4:0][T] fd0d:edc4::2001::203:6669->fd0d:edc4:ESTABLISHED0 0
[4:1][T] fd0d:edc4::2001::203:57550->fd0d:edc4ESTABLISHED0
399
[4:2][T] fd0d:edc4::2001::203:56939->fd0d:edc4ESTABLISHED0 0
Thread 4: active sessions 3
thanks,
-Raj
On Tue, Jan 21, 2020 at 9:43 PM Florin Coras wrote:
> Hi Raj,
>
> Inline.
>
> On Jan 21, 2020, at 3:41 PM, Raj Kumar wrote:
>
> Hi Florin,
> There is no drop on the interfaces. It is 100G card.
> In UDP tx application, I am using 1460 bytes of buffer to send on
> select(). I am getting 5 Gbps throughput ,but if I start one more
> application then total throughput goes down to 4 Gbps as both the sessions
> are on the same thread.
> I increased the tx buffer to 8192 bytes and then I can get 11 Gbps
> throughput but again if I start one more application the throughput goes
> down to 10 Gbps.
>
>
> FC: I assume you’re using vppcom_session_write to write to the session.
> How large is “len” typically? See lower on why that matters.
>
>
>
> I found one issue in the code ( You must be aware of that) , the UDP send
> MSS is hard-coded to 1460 ( /vpp/src/vnet/udp/udp.c file). So, the large
> packets are getting fragmented.
>
> udp_send_mss (transport_connection_t * t)
> {
> /* TODO figure out MTU of output interface */
> return 1460;
> }
>
>
> FC: That’s a typical mss and actually what tcp uses as well. Given the
> nics, they should be fine sending a decent number of mpps without the need
> to do jumbo ip datagrams.
>
> if I change the MSS to 8192 then I am getting 17 Mbps throughput. But , if
> i start one more application then throughput is going down to 13 Mbps.
>
>
> It looks like the 17 Mbps is per core limit and since all the sessions are
> pined to the same thread we can not get more throughput. Here, per core
> throughput look good to me. Please let me know there is any way to use
> multiple threads for UDP tx applications.
>
>
> In your previous email you mentioned that we can use connected udp socket
> in the UDP receiver. Can we do something similar for UDP tx ?
>
>
> FC: I think it may work fine if vpp has main + 1 worker. I have a draft
> patch here [1] that seems to work with multiple workers but it’s not
> heavily tested.
>
> Out of curiosity, I ran a vcl_test_client/server test with 1 worker and
> with XL710s, I’m seeing this:
>
> CLIENT RESULTS: Streamed 65536017791 bytes
> in 14.392678 seconds (36.427420 Gbps half-duplex)!
>
> Should be noted that because of how datagrams are handled in the session
> layer, throughput is sensitive to write sizes. I ran the client like:
> ~/vcl_client -p udpc 6.0.1.2 1234 -U -N 100 -T 65536
>
> Or in english, unidirectional test, tx buffer of 64kB and 1M writes of
> that buffer. My vcl config was such that tx fifos were 4MB and rx fifos
> 2MB. The sender had few tx packet drops (1657) and the receiver few rx
> packet drops (801). If you plan to use it, make sure arp entries are first
> resolved (e.g., use ping) otherwise the first packet is lost.
>
> Throughput drops to ~15Gbps with 8kB writes. You should probably also test
> with bigger writes with udp.
>
> [1] https://gerrit.fd.io/r/c/vpp/+/24462
>
>
> From the hardware stats , it seems that UDP tx checksum offload is not
> enabled/active which could impact the performance. I think, udp tx
> checksum should be enabled by default if it is not disabled using
> parameter "no-tx-checksum-offload".
>
>
> FC: Performance might be affected by the limited number of offloads
> available. Here’s what I see on my XL710s:
>
> rx offload active: ipv4-cksum jumbo-frame scatter
> tx offload active: udp-cksum tcp-cksum multi-segs
>
>
> Ethernet address b8:83:03:79:af:8c
> Mellanox ConnectX-4 Family
> carrier up
Re: [vpp-dev] #vpp-hoststack - Issue with UDP receiver application using VCL library
Correction : -
Please read 17 Mbps as 17 Gbps and 13Mbps as 13Gbps in my previous mail.
thanks,
-Raj
On Tue, Jan 21, 2020 at 6:41 PM Raj Kumar wrote:
> Hi Florin,
> There is no drop on the interfaces. It is 100G card.
> In UDP tx application, I am using 1460 bytes of buffer to send on
> select(). I am getting 5 Gbps throughput ,but if I start one more
> application then total throughput goes down to 4 Gbps as both the sessions
> are on the same thread.
> I increased the tx buffer to 8192 bytes and then I can get 11 Gbps
> throughput but again if I start one more application the throughput goes
> down to 10 Gbps.
>
> I found one issue in the code ( You must be aware of that) , the UDP send
> MSS is hard-coded to 1460 ( /vpp/src/vnet/udp/udp.c file). So, the large
> packets are getting fragmented.
> udp_send_mss (transport_connection_t * t)
> {
> /* TODO figure out MTU of output interface */
> return 1460;
> }
> if I change the MSS to 8192 then I am getting 17 Mbps throughput. But , if
> i start one more application then throughput is going down to 13 Mbps.
>
> It looks like the 17 Mbps is per core limit and since all the sessions are
> pined to the same thread we can not get more throughput. Here, per core
> throughput look good to me. Please let me know there is any way to use
> multiple threads for UDP tx applications.
>
> In your previous email you mentioned that we can use connected udp socket
> in the UDP receiver. Can we do something similar for UDP tx ?
>
> From the hardware stats , it seems that UDP tx checksum offload is not
> enabled/active which could impact the performance. I think, udp tx
> checksum should be enabled by default if it is not disabled using
> parameter "no-tx-checksum-offload".
>
> Ethernet address b8:83:03:79:af:8c
> Mellanox ConnectX-4 Family
> carrier up full duplex mtu 9206
> flags: admin-up pmd maybe-multiseg subif rx-ip4-cksum
> rx: queues 5 (max 65535), desc 1024 (min 0 max 65535 align 1)
> tx: queues 6 (max 65535), desc 1024 (min 0 max 65535 align 1)
> pci: device 15b3:1017 subsystem 1590:0246 address :12:00.00 numa 0
> max rx packet len: 65536
> promiscuous: unicast off all-multicast on
> vlan offload: strip off filter off qinq off
> rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum
> vlan-filter
>jumbo-frame scatter timestamp keep-crc
> rx offload active: ipv4-cksum jumbo-frame scatter
> tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum tcp-tso
>outer-ipv4-cksum vxlan-tnl-tso gre-tnl-tso
> multi-segs
>udp-tnl-tso ip-tnl-tso
> tx offload active: multi-segs
> rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4
> ipv6-tcp-ex
>ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other
>ipv6-ex ipv6
> rss active:ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4
> ipv6-tcp-ex
> ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other
> ipv6-ex ipv6
> tx burst function: (nil)
> rx burst function: mlx5_rx_burst
>
> thanks,
> -Raj
>
> On Mon, Jan 20, 2020 at 7:55 PM Florin Coras
> wrote:
>
>> Hi Raj,
>>
>> Good to see progress. Check with “show int” the tx counters on the sender
>> and rx counters on the receiver as the interfaces might be dropping
>> traffic. One sender should be able to do more than 5Gbps.
>>
>> How big are the writes to the tx fifo? Make sure the tx buffer is some
>> tens of kB.
>>
>> As for the issue with the number of workers, you’ll have to switch to
>> udpc (connected udp), to ensure you have a separate connection for each
>> ‘flow’, and to use accept in combination with epoll to accept the sessions
>> udpc creates.
>>
>> Note that udpc currently does not work correctly with vcl and multiple
>> vpp workers if vcl is the sender (not the receiver) and traffic is
>> bidirectional. The sessions are all created on the first thread and once
>> return traffic is received, they’re migrated to the thread selected by RSS
>> hashing. VCL is not notified when that happens and it runs out of sync. You
>> might not be affected by this, as you’re not receiving any return traffic,
>> but because of that all sessions may end up stuck on the first thread.
>>
>> For udp transport, the listener is connection-less and bound to the main
>> thread. As a result, all incoming packets, even if they pertain to multiple
>> flows, are written to the listener’s buffer/fifo.
>>
>> Regards,
>> Florin
>>
>> On Jan 20
Re: [vpp-dev] #vpp-hoststack - Issue with UDP receiver application using VCL library
Hi Florin,
There is no drop on the interfaces. It is 100G card.
In UDP tx application, I am using 1460 bytes of buffer to send on select().
I am getting 5 Gbps throughput ,but if I start one more application then
total throughput goes down to 4 Gbps as both the sessions are on the same
thread.
I increased the tx buffer to 8192 bytes and then I can get 11 Gbps
throughput but again if I start one more application the throughput goes
down to 10 Gbps.
I found one issue in the code ( You must be aware of that) , the UDP send
MSS is hard-coded to 1460 ( /vpp/src/vnet/udp/udp.c file). So, the large
packets are getting fragmented.
udp_send_mss (transport_connection_t * t)
{
/* TODO figure out MTU of output interface */
return 1460;
}
if I change the MSS to 8192 then I am getting 17 Mbps throughput. But , if
i start one more application then throughput is going down to 13 Mbps.
It looks like the 17 Mbps is per core limit and since all the sessions are
pined to the same thread we can not get more throughput. Here, per core
throughput look good to me. Please let me know there is any way to use
multiple threads for UDP tx applications.
In your previous email you mentioned that we can use connected udp socket
in the UDP receiver. Can we do something similar for UDP tx ?
>From the hardware stats , it seems that UDP tx checksum offload is not
enabled/active which could impact the performance. I think, udp tx
checksum should be enabled by default if it is not disabled using
parameter "no-tx-checksum-offload".
Ethernet address b8:83:03:79:af:8c
Mellanox ConnectX-4 Family
carrier up full duplex mtu 9206
flags: admin-up pmd maybe-multiseg subif rx-ip4-cksum
rx: queues 5 (max 65535), desc 1024 (min 0 max 65535 align 1)
tx: queues 6 (max 65535), desc 1024 (min 0 max 65535 align 1)
pci: device 15b3:1017 subsystem 1590:0246 address :12:00.00 numa 0
max rx packet len: 65536
promiscuous: unicast off all-multicast on
vlan offload: strip off filter off qinq off
rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum vlan-filter
jumbo-frame scatter timestamp keep-crc
rx offload active: ipv4-cksum jumbo-frame scatter
tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum tcp-tso
outer-ipv4-cksum vxlan-tnl-tso gre-tnl-tso multi-segs
udp-tnl-tso ip-tnl-tso
tx offload active: multi-segs
rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4
ipv6-tcp-ex
ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other
ipv6-ex ipv6
rss active:ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4
ipv6-tcp-ex
ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other
ipv6-ex ipv6
tx burst function: (nil)
rx burst function: mlx5_rx_burst
thanks,
-Raj
On Mon, Jan 20, 2020 at 7:55 PM Florin Coras wrote:
> Hi Raj,
>
> Good to see progress. Check with “show int” the tx counters on the sender
> and rx counters on the receiver as the interfaces might be dropping
> traffic. One sender should be able to do more than 5Gbps.
>
> How big are the writes to the tx fifo? Make sure the tx buffer is some
> tens of kB.
>
> As for the issue with the number of workers, you’ll have to switch to udpc
> (connected udp), to ensure you have a separate connection for each ‘flow’,
> and to use accept in combination with epoll to accept the sessions udpc
> creates.
>
> Note that udpc currently does not work correctly with vcl and multiple vpp
> workers if vcl is the sender (not the receiver) and traffic is
> bidirectional. The sessions are all created on the first thread and once
> return traffic is received, they’re migrated to the thread selected by RSS
> hashing. VCL is not notified when that happens and it runs out of sync. You
> might not be affected by this, as you’re not receiving any return traffic,
> but because of that all sessions may end up stuck on the first thread.
>
> For udp transport, the listener is connection-less and bound to the main
> thread. As a result, all incoming packets, even if they pertain to multiple
> flows, are written to the listener’s buffer/fifo.
>
> Regards,
> Florin
>
> On Jan 20, 2020, at 3:50 PM, Raj Kumar wrote:
>
> Hi Florin,
> I changed my application as you suggested. Now, I am able to achieve 5
> Gbps with a single UDP stream. Overall, I can get ~20Gbps with multiple
> host application . Also, the TCP throughput is improved to ~28Gbps after
> tuning as mentioned in [1].
> On the similar topic; the UDP tx throughput is throttled to 5Gbps. Even if
> I run the multiple host applications the overall throughput is 5Gbps. I
> also tried by configuring multiple worker threads . But the problem is that
> all the application sessions are assigned to the
Re: [vpp-dev] #vpp-hoststack - Issue with UDP receiver application using VCL library
Hi Florin, I changed my application as you suggested. Now, I am able to achieve 5 Gbps with a single UDP stream. Overall, I can get ~20Gbps with multiple host application . Also, the TCP throughput is improved to ~28Gbps after tuning as mentioned in [1]. On the similar topic; the UDP tx throughput is throttled to 5Gbps. Even if I run the multiple host applications the overall throughput is 5Gbps. I also tried by configuring multiple worker threads . But the problem is that all the application sessions are assigned to the same worker thread. Is there any way to assign each session to a different worker thread? vpp# sh session verbose 2 Thread 0: no sessions [#1][U] fd0d:edc4::2001::203:58926->fd0d:edc4: Rx fifo: cursize 0 nitems 399 has_event 0 head 0 tail 0 segment manager 1 vpp session 0 thread 1 app session 0 thread 0 ooo pool 0 active elts newest 0 Tx fifo: cursize 399 nitems 399 has_event 1 head 1460553 tail 1460552 segment manager 1 vpp session 0 thread 1 app session 0 thread 0 ooo pool 0 active elts newest 4294967295 session: state: opened opaque: 0x0 flags: [#1][U] fd0d:edc4::2001::203:63413->fd0d:edc4: Rx fifo: cursize 0 nitems 399 has_event 0 head 0 tail 0 segment manager 2 vpp session 1 thread 1 app session 0 thread 0 ooo pool 0 active elts newest 0 Tx fifo: cursize 399 nitems 399 has_event 1 head 3965434 tail 3965433 segment manager 2 vpp session 1 thread 1 app session 0 thread 0 ooo pool 0 active elts newest 4294967295 session: state: opened opaque: 0x0 flags: Thread 1: active sessions 2 Thread 2: no sessions Thread 3: no sessions Thread 4: no sessions Thread 5: no sessions Thread 6: no sessions Thread 7: no sessions vpp# sh app client Connection App [#1][U] fd0d:edc4::2001::203:58926->udp6_tx_8092[shm] [#1][U] fd0d:edc4::2001::203:63413->udp6_tx_8093[shm] vpp# thanks, -Raj On Sun, Jan 19, 2020 at 8:50 PM Florin Coras wrote: > Hi Raj, > > The function used for receiving datagrams is limited to reading at most > the length of a datagram from the rx fifo. UDP datagrams are mtu sized, so > your reads are probably limited to ~1.5kB. On each epoll rx event try > reading from the session handle in a while loop until you get an > VPPCOM_EWOULDBLOCK. That might improve performance. > > Having said that, udp is lossy so unless you implement your own > congestion/flow control algorithms, the data you’ll receive might be full > of “holes”. What are the rx/tx error counters on your interfaces (check > with “sh int”). > > Also, with simple tuning like this [1], you should be able to achieve much > more than 15Gbps with tcp. > > Regards, > Florin > > [1] https://wiki.fd.io/view/VPP/HostStack/LDP/iperf > > On Jan 19, 2020, at 3:25 PM, Raj Kumar wrote: > > Hi Florin, > By using VCL library in an UDP receiver application, I am able to > receive only 2 Mbps traffic. On increasing the traffic, I see Rx FIFO full > error and application stopped receiving the traffic from the session > layer. Whereas, with TCP I can easily achieve 15Gbps throughput without > tuning any DPDK parameter. UDP tx also looks fine. From an host > application I can send ~5Gbps without any issue. > > I am running VPP( stable/2001 code) on RHEL8 server using Mellanox 100G > (MLNX5) adapters. > Please advise if I can use VCL library to receive high throughput UDP > traffic ( in Gbps). I would be running multiple instances of host > application to receive data ( ~50-60 Gbps). > > I also tried by increasing the Rx FIFO size to 16MB but did not help much. > The host application is just throwing the received packets , it is not > doing any packet processing. > > [root@orc01 vcl_test]# VCL_DEBUG=2 ./udp6_server_vcl > VCL<20201>: configured VCL debug level (2) from VCL_DEBUG! > VCL<20201>: allocated VCL heap = 0x7f39a17ab010, size 268435456 > (0x1000) > VCL<20201>: configured rx_fifo_size 400 (0x3d0900) > VCL<20201>: configured tx_fifo_size 400 (0x3d0900) > VCL<20201>: configured app_scope_local (1) > VCL<20201>: configured app_scope_global (1) > VCL<20201>: configured api-socket-name (/tmp/vpp-api.sock) > VCL<20201>: completed parsing vppcom config! > vppcom_connect_to_vpp:480: vcl<20201:0>: app (udp6_server) is connected to > VPP! > vppcom_app_create:1104: vcl<20201:0>: sending session enable > vppcom_app_create:1112: vcl<20201:0>: sending app attach > vppcom_app_create:1121: vcl<20201:0>: app_name 'udp6_server', > my_client_index 256 (0x100) > vppcom_epoll_create:2439: vcl<20201:0>: Created vep_idx 0 > vppcom_session_create:1179: vcl<20201:0>: created
Re: [vpp-dev] #vpp-hoststack - Issue with UDP receiver application using VCL library
0d:edc4::2001::203
tos 0x00, flow label 0x0, hop limit 64, payload length 1458
UDP: 56944 -> 8092
length 1458, checksum 0xb22d
00:09:53:445028: ethernet-input
frame: flags 0x3, hw-if-index 2, sw-if-index 2
IP6: b8:83:03:79:9f:e4 -> b8:83:03:79:af:8c 802.1q vlan 2001
00:09:53:445029: ip6-input
UDP: fd0d:edc4::2001::201 -> fd0d:edc4::2001::203
tos 0x00, flow label 0x0, hop limit 64, payload length 1458
UDP: 56944 -> 8092
length 1458, checksum 0xb22d
00:09:53:445031: ip6-lookup
fib 0 dpo-idx 6 flow hash: 0x
UDP: fd0d:edc4::2001::201 -> fd0d:edc4::2001::203
tos 0x00, flow label 0x0, hop limit 64, payload length 1458
UDP: 56944 -> 8092
length 1458, checksum 0xb22d
00:09:53:445032: ip6-local
UDP: fd0d:edc4::2001::201 -> fd0d:edc4::2001::203
tos 0x00, flow label 0x0, hop limit 64, payload length 1458
UDP: 56944 -> 8092
length 1458, checksum 0xb22d
00:09:53:445032: ip6-udp-lookup
UDP: src-port 56944 dst-port 8092
00:09:53:445033: udp6-input
UDP_INPUT: connection 0, disposition 5, thread 0
thanks,
-Raj
On Wed, Jan 15, 2020 at 4:09 PM Raj Kumar via Lists.Fd.Io wrote:
> Hi Florin,
> Yes, [2] patch resolved the IPv6/UDP receiver issue.
> Thanks! for your help.
>
> thanks,
> -Raj
>
> On Tue, Jan 14, 2020 at 9:35 PM Florin Coras
> wrote:
>
>> Hi Raj,
>>
>> First of all, with this [1], the vcl test app/client can establish a udpc
>> connection. Note that udp will most probably lose packets, so large
>> exchanges with those apps may not work.
>>
>> As for the second issue, does [2] solve it?
>>
>> Regards,
>> Florin
>>
>> [1] https://gerrit.fd.io/r/c/vpp/+/24332
>> [2] https://gerrit.fd.io/r/c/vpp/+/24334
>>
>> On Jan 14, 2020, at 12:59 PM, Raj Kumar wrote:
>>
>> Hi Florin,
>> Thanks! for the reply.
>>
>> I realized the issue with the non-connected case. For receiving
>> datagrams, I was using recvfrom() with DONOT_WAIT flag because of
>> that vppcom_session_recvfrom() api was failing. It expects either 0 or
>> MSG_PEEK flag.
>> if (flags == 0)
>> rv = vppcom_session_read (session_handle, buffer, buflen);
>> else if (flags & MSG_PEEK) 0x2
>> rv = vppcom_session_peek (session_handle, buffer, buflen);
>> else
>> {
>> VDBG (0, "Unsupport flags for recvfrom %d", flags);
>> return VPPCOM_EAFNOSUPPORT;
>> }
>>
>> I changed the flag to 0 in recvfrom() , after that UDP rx is working
>> fine but only for IPv4.
>>
>> I am facing a different issue with IPv6/UDP receiver. I am getting "no
>> listener for dst port" error.
>>
>> Please let me know if I am doing something wrong.
>> Here are the traces : -
>>
>> [root@orc01 testcode]# VCL_DEBUG=2 LDP_DEBUG=2
>> LD_PRELOAD=/opt/vpp/build-root/install-vpp-native/vpp/lib/libvcl_ldpreload.so
>> VCL_CONFIG=/etc/vpp/vcl.cfg ./udp6_rx
>> VCL<1164>: configured VCL debug level (2) from VCL_DEBUG!
>> VCL<1164>: allocated VCL heap = 0x7ff877439010, size 268435456
>> (0x1000)
>> VCL<1164>: configured rx_fifo_size 400 (0x3d0900)
>> VCL<1164>: configured tx_fifo_size 400 (0x3d0900)
>> VCL<1164>: configured app_scope_local (1)
>> VCL<1164>: configured app_scope_global (1)
>> VCL<1164>: configured api-socket-name (/tmp/vpp-api.sock)
>> VCL<1164>: completed parsing vppcom config!
>> vppcom_connect_to_vpp:549: vcl<1164:0>: app (ldp-1164-app) is connected
>> to VPP!
>> vppcom_app_create:1067: vcl<1164:0>: sending session enable
>> vppcom_app_create:1075: vcl<1164:0>: sending app attach
>> vppcom_app_create:1084: vcl<1164:0>: app_name 'ldp-1164-app',
>> my_client_index 0 (0x0)
>> ldp_init:209: ldp<1164>: configured LDP debug level (2) from env var
>> LDP_DEBUG!
>> ldp_init:282: ldp<1164>: LDP initialization: done!
>> ldp_constructor:2490: LDP<1164>: LDP constructor: done!
>> socket:974: ldp<1164>: calling vls_create: proto 1 (UDP), is_nonblocking 0
>> vppcom_session_create:1142: vcl<1164:0>: created session 0
>> bind:1086: ldp<1164>: fd 32: calling vls_bind: vlsh 0, addr
>> 0x7fff9a93efe0, len 28
>> vppcom_session_bind:1280: vcl<1164:0>: session 0 handle 0: binding to
>> local IPv6 address :: port 8092, proto UDP
>> vppcom_session_listen:1312: vcl<1164:0>: session 0: sending vpp listen
>> request...
>> vcl_session_bound_handler:610: vcl<1164:0>: session 0 [0x1]: listen
>> succeeded!
>> bind:1102: ldp<1164>
Re: [vpp-dev] #vpp-hoststack - Issue with UDP receiver application using VCL library
Hi Florin,
Yes, [2] patch resolved the IPv6/UDP receiver issue.
Thanks! for your help.
thanks,
-Raj
On Tue, Jan 14, 2020 at 9:35 PM Florin Coras wrote:
> Hi Raj,
>
> First of all, with this [1], the vcl test app/client can establish a udpc
> connection. Note that udp will most probably lose packets, so large
> exchanges with those apps may not work.
>
> As for the second issue, does [2] solve it?
>
> Regards,
> Florin
>
> [1] https://gerrit.fd.io/r/c/vpp/+/24332
> [2] https://gerrit.fd.io/r/c/vpp/+/24334
>
> On Jan 14, 2020, at 12:59 PM, Raj Kumar wrote:
>
> Hi Florin,
> Thanks! for the reply.
>
> I realized the issue with the non-connected case. For receiving
> datagrams, I was using recvfrom() with DONOT_WAIT flag because of
> that vppcom_session_recvfrom() api was failing. It expects either 0 or
> MSG_PEEK flag.
> if (flags == 0)
> rv = vppcom_session_read (session_handle, buffer, buflen);
> else if (flags & MSG_PEEK) 0x2
> rv = vppcom_session_peek (session_handle, buffer, buflen);
> else
> {
> VDBG (0, "Unsupport flags for recvfrom %d", flags);
> return VPPCOM_EAFNOSUPPORT;
> }
>
> I changed the flag to 0 in recvfrom() , after that UDP rx is working fine
> but only for IPv4.
>
> I am facing a different issue with IPv6/UDP receiver. I am getting "no
> listener for dst port" error.
>
> Please let me know if I am doing something wrong.
> Here are the traces : -
>
> [root@orc01 testcode]# VCL_DEBUG=2 LDP_DEBUG=2
> LD_PRELOAD=/opt/vpp/build-root/install-vpp-native/vpp/lib/libvcl_ldpreload.so
> VCL_CONFIG=/etc/vpp/vcl.cfg ./udp6_rx
> VCL<1164>: configured VCL debug level (2) from VCL_DEBUG!
> VCL<1164>: allocated VCL heap = 0x7ff877439010, size 268435456 (0x1000)
> VCL<1164>: configured rx_fifo_size 400 (0x3d0900)
> VCL<1164>: configured tx_fifo_size 400 (0x3d0900)
> VCL<1164>: configured app_scope_local (1)
> VCL<1164>: configured app_scope_global (1)
> VCL<1164>: configured api-socket-name (/tmp/vpp-api.sock)
> VCL<1164>: completed parsing vppcom config!
> vppcom_connect_to_vpp:549: vcl<1164:0>: app (ldp-1164-app) is connected to
> VPP!
> vppcom_app_create:1067: vcl<1164:0>: sending session enable
> vppcom_app_create:1075: vcl<1164:0>: sending app attach
> vppcom_app_create:1084: vcl<1164:0>: app_name 'ldp-1164-app',
> my_client_index 0 (0x0)
> ldp_init:209: ldp<1164>: configured LDP debug level (2) from env var
> LDP_DEBUG!
> ldp_init:282: ldp<1164>: LDP initialization: done!
> ldp_constructor:2490: LDP<1164>: LDP constructor: done!
> socket:974: ldp<1164>: calling vls_create: proto 1 (UDP), is_nonblocking 0
> vppcom_session_create:1142: vcl<1164:0>: created session 0
> bind:1086: ldp<1164>: fd 32: calling vls_bind: vlsh 0, addr
> 0x7fff9a93efe0, len 28
> vppcom_session_bind:1280: vcl<1164:0>: session 0 handle 0: binding to
> local IPv6 address :: port 8092, proto UDP
> vppcom_session_listen:1312: vcl<1164:0>: session 0: sending vpp listen
> request...
> vcl_session_bound_handler:610: vcl<1164:0>: session 0 [0x1]: listen
> succeeded!
> bind:1102: ldp<1164>: fd 32: returning 0
>
> vpp# sh app server
> Connection App Wrk
> [0:0][CT:U] :::8092->:::0 ldp-1164-app[shm] 0
> [#0][U] :::8092->:::0 ldp-1164-app[shm] 0
>
> vpp# sh err
>CountNode Reason
> 7 dpdk-input no error
> 2606 ip6-udp-lookup no listener for dst port
> 8arp-reply ARP replies sent
> 1 arp-disabled ARP Disabled on this
> interface
> 13ip6-glean neighbor solicitations
> sent
> 2606ip6-input valid ip6 packets
> 4 ip6-local-hop-by-hop Unknown protocol ip6
> local h-b-h packets dropped
> 2606 ip6-icmp-error destination unreachable
> response sent
> 40 ip6-icmp-input valid packets
> 1 ip6-icmp-input neighbor solicitations
> from source not on link
> 12 ip6-icmp-input neighbor solicitations
> for unknown targets
> 1 ip6-icmp-input neighbor advertisements
> sent
> 1 ip6-icmp-input neighbor advertisements
> received
> 40 ip6-icmp-input
Re: [vpp-dev] #vpp-hoststack - Issue with UDP receiver application using VCL library
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid Packet Types RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet RTE_PTYPE_L3_IPV6_EXT_UNKNOWN (0x00e0) IPv6 packet with or without extension headers RTE_PTYPE_L4_UDP (0x0200) UDP packet IP6: b8:83:03:79:9f:e4 -> b8:83:03:79:af:8c 802.1q vlan 2001 UDP: fd0d:edc4::2001::201 -> fd0d:edc4::2001::203 tos 0x00, flow label 0x0, hop limit 64, payload length 1458 UDP: 60593 -> 8092 length 1458, checksum 0x0964 00:23:39:401355: ethernet-input frame: flags 0x3, hw-if-index 2, sw-if-index 2 IP6: b8:83:03:79:9f:e4 -> b8:83:03:79:af:8c 802.1q vlan 2001 00:23:39:401356: ip6-input UDP: fd0d:edc4::2001::201 -> fd0d:edc4::2001::203 tos 0x00, flow label 0x0, hop limit 64, payload length 1458 UDP: 60593 -> 8092 length 1458, checksum 0x0964 00:23:39:401357: ip6-lookup fib 0 dpo-idx 5 flow hash: 0x UDP: fd0d:edc4::2001::201 -> fd0d:edc4::2001::203 tos 0x00, flow label 0x0, hop limit 64, payload length 1458 UDP: 60593 -> 8092 length 1458, checksum 0x0964 00:23:39:401361: ip6-local UDP: fd0d:edc4::2001::201 -> fd0d:edc4::2001::203 tos 0x00, flow label 0x0, hop limit 64, payload length 1458 UDP: 60593 -> 8092 length 1458, checksum 0x0964 00:23:39:401362: ip6-udp-lookup UDP: src-port 60593 dst-port 8092 (no listener) 00:23:39:401362: ip6-icmp-error UDP: fd0d:edc4::2001::201 -> fd0d:edc4::2001::203 tos 0x00, flow label 0x0, hop limit 64, payload length 1458 UDP: 60593 -> 8092 length 1458, checksum 0x0964 00:23:39:401363: error-drop rx:HundredGigabitEthernet12/0/0.2001 00:23:39:401364: drop ip6-input: valid ip6 packets vpp# Thanks, -Raj On Tue, Jan 14, 2020 at 1:44 PM Florin Coras wrote: > Hi Raj, > > Session layer does support connection-less transports but udp does not > raise accept notifications to vcl. UDPC might, but we haven’t tested udpc > with vcl in a long time so it might not work properly. > > What was the problem you were hitting in the non-connected case? > > Regards, > Florin > > > On Jan 14, 2020, at 7:13 AM, raj.gauta...@gmail.com wrote: > > > > Hi , > > I am trying some host application tests ( using LD_PRELOAD) . TCP rx > and tx both work fine. UDP tx also works fine. > > The issue is only with UDP rx . In some discussion it was mentioned > that session layer does not support connection-less transports so protocols > like udp still need to accept connections and only afterwards read from the > fifos. > > So, I changed the UDP receiver application to use listen() and accept() > before read() . But , I am still having issue to make it run. > > After I started, udp traffic from other server it seems to accept the > connection but never returns from the vppcom_session_accept() function. > > VPP release is 19.08. > > > > vpp# sh app server > > Connection App Wrk > > [0:0][CT:U] 0.0.0.0:8090->0.0.0.0:0 ldp-36646-app[shm]0 > > [#0][U] 0.0.0.0:8090->0.0.0.0:0 ldp-36646-app[shm]0 > > vpp# > > > > > > [root@orc01 testcode]# VCL_DEBUG=2 LDP_DEBUG=2 > LD_PRELOAD=/opt/vpp/build-root/install-vpp-native/vpp/lib/libvcl_ldpreload.so > VCL_CONFIG=/etc/vpp/vcl.cfg ./udp_rx > > VCL<36646>: configured VCL debug level (2) from VCL_DEBUG! > > VCL<36646>: allocated VCL heap = 0x7f77e5309010, size 268435456 > (0x1000) > > VCL<36646>: configured rx_fifo_size 400 (0x3d0900) > > VCL<36646>: configured tx_fifo_size 400 (0x3d0900) > > VCL<36646>: configured app_scope_local (1) > > VCL<36646>: configured app_scope_global (1) > > VCL<36646>: configured api-socket-name (/tmp/vpp-api.sock) > > VCL<36646>: completed parsing vppcom config! > > vppcom_connect_to_vpp:549: vcl<36646:0>: app (ldp-36646-app) is > connected to VPP! > > vppcom_app_create:1067: vcl<36646:0>: sending session enable > > vppcom_app_create:1075: vcl<36646:0>: sending app attach > > vppcom_app_create:1084: vcl<36646:0>: app_name 'ldp-36646-app', > my_client_index 0 (0x0) > > ldp_init:209: ldp<36646>: configured LDP debug level (2) from env var > LDP_DEBUG! > > ldp_init:282: ldp<36646>: LDP initialization: done! > > ldp_constructor:2490: LDP<36646>: LDP constructor: done! > > socket:974: ldp<36646>: calling vls_create: proto 1 (UDP), > is_nonblocking 0 > > vppcom_session_create:1142: vcl<36646:0>: created session 0 > > Socket successfully created.. > > bind:1086: ldp<36646>: fd 32: calling vls_bind: vlsh 0, addr > 0x7fff3f3c1040, len 16 &g
[vpp-dev] #vpp-hoststack - Issue with UDP receiver application using VCL library
Hi , I am trying some host application tests ( using LD_PRELOAD) . TCP rx and tx both work fine. UDP tx also works fine. The issue is only with UDP rx . In some discussion it was mentioned that session layer does not support connection-less transports so protocols like udp still need to accept connections and only afterwards read from the fifos. So, I changed the UDP receiver application to use listen() and accept() before read() . But , I am still having issue to make it run. After I started, udp traffic from other server it seems to accept the connection but never returns from the vppcom_session_accept() function. VPP release is 19.08. vpp# sh app server Connection App Wrk [0:0][CT:U] 0.0.0.0:8090->0.0.0.0:0 ldp-36646-app[shm] 0 [#0][U] 0.0.0.0:8090->0.0.0.0:0 ldp-36646-app[shm] 0 vpp# [root@orc01 testcode]# VCL_DEBUG=2 LDP_DEBUG=2 LD_PRELOAD=/opt/vpp/build-root/install-vpp-native/vpp/lib/libvcl_ldpreload.so VCL_CONFIG=/etc/vpp/vcl.cfg ./udp_rx VCL<36646>: configured VCL debug level (2) from VCL_DEBUG! VCL<36646>: allocated VCL heap = 0x7f77e5309010, size 268435456 (0x1000) VCL<36646>: configured rx_fifo_size 400 (0x3d0900) VCL<36646>: configured tx_fifo_size 400 (0x3d0900) VCL<36646>: configured app_scope_local (1) VCL<36646>: configured app_scope_global (1) VCL<36646>: configured api-socket-name (/tmp/vpp-api.sock) VCL<36646>: completed parsing vppcom config! vppcom_connect_to_vpp:549: vcl<36646:0>: app (ldp-36646-app) is connected to VPP! vppcom_app_create:1067: vcl<36646:0>: sending session enable vppcom_app_create:1075: vcl<36646:0>: sending app attach vppcom_app_create:1084: vcl<36646:0>: app_name 'ldp-36646-app', my_client_index 0 (0x0) ldp_init:209: ldp<36646>: configured LDP debug level (2) from env var LDP_DEBUG! ldp_init:282: ldp<36646>: LDP initialization: done! ldp_constructor:2490: LDP<36646>: LDP constructor: done! socket:974: ldp<36646>: calling vls_create: proto 1 (UDP), is_nonblocking 0 vppcom_session_create:1142: vcl<36646:0>: created session 0 Socket successfully created.. bind:1086: ldp<36646>: fd 32: calling vls_bind: vlsh 0, addr 0x7fff3f3c1040, len 16 vppcom_session_bind:1280: vcl<36646:0>: session 0 handle 0: binding to local IPv4 address 0.0.0.0 port 8090, proto UDP vppcom_session_listen:1312: vcl<36646:0>: session 0: sending vpp listen request... vcl_session_bound_handler:610: vcl<36646:0>: session 0 [0x1]: listen succeeded! bind:1102: ldp<36646>: fd 32: returning 0 Socket successfully binded.. listen:2005: ldp<36646>: fd 32: calling vls_listen: vlsh 0, n 5 vppcom_session_listen:1308: vcl<36646:0>: session 0 [0x1]: already in listen state! listen:2020: ldp<36646>: fd 32: returning 0 Server listening.. ldp_accept4:2043: ldp<36646>: listen fd 32: calling vppcom_session_accept: listen sid 0, ep 0x0, flags 0x3f3c0fc0 vppcom_session_accept:1478: vcl<36646:0>: discarded event: 0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15162): https://lists.fd.io/g/vpp-dev/message/15162 Mute This Topic: https://lists.fd.io/mt/69694900/21656 Mute #vpp-hoststack: https://lists.fd.io/mk?hashtag=vpp-hoststack=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Policer download bandwidth reduces on high rtt connection
Hello all, It looks like the input interface is not receiving all packets. iperf is sending 35176 packets but GigabitEthernet0/14/1 is showing only 15390 rx packets, which is approximately 56% packet loss. So this is not a policer issue, but some thing related to dpdk-input I guess. My startup config has dpdk section totally commented out. Is this a case of missing some parameters? [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-10.00 sec 47.7 MBytes 40.0 Mbits/sec 0.000 ms 0/35176 (0%) sender [SUM] 0.0-10.0 sec 13 datagrams received out-of-order [ 5] 0.00-10.00 sec 20.8 MBytes 17.4 Mbits/sec 0.131 ms 19827/35176 (56%) receiver DBGvpp# sh int GigabitEthernet0/14/1 Name IdxState MTU (L3/IP4/IP6/MPLS) Counter Count GigabitEthernet0/14/1 1 up 1500/0/0/0 rx packets 15390 rx bytes22443048 tx packets32 tx bytes2533 drops 12 punt 1 ip415386 Thanks and Regards, Raj On Mon, Dec 23, 2019 at 11:25 PM Raj via Lists.Fd.Io wrote: > > Hello all, > > In VPP I have configured policer to limit 50mbps bandwidth upload and > download to an IP. > > vpp# sh policer > Name "lap50_dw_20191223_212712" type 2r3c-4115 > cir 51200 eir 51456 cb 12800 eb 1536000 > rate type kbps, round type up > conform action transmit, exceed action transmit, violate action drop > > Template policer at 7ff7b20cb36c: dual rate, not color-aware > cir 709911 tok/period, pir 713461 tok/period, scale 11 > cur lim 26214400, cur bkt 26214400, ext lim 3145728000, ext bkt 3145728000 > last update 0 > --- > Name "lap50_up_20191223_212712" type 2r3c-4115 > cir 51200 eir 51456 cb 12800 eb 1536000 > rate type kbps, round type up > conform action transmit, exceed action transmit, violate action drop > > Template policer at 7ff7b20cb3ac: dual rate, not color-aware > cir 709911 tok/period, pir 713461 tok/period, scale 11 > cur lim 26214400, cur bkt 26214400, ext lim 3145728000, ext bkt 3145728000 > last update 0 > > When iperf is run over UDP and TCP from a server with a round trip > time of 24ms I get full 50mbps for TCP and UDP for both upload and > download. > > But when the same test is performed from a server with rtt of 162ms or > even 46ms, I get full upload speed, but download speed is around > 17mbps, for both TCP and UDP. > > The iperf server is running on remote server and in local machine I > used the following command: > > iperf3 -c -t 20 -u -b 40M -R. > > This is kind of baffling and would appreciate if there are any > pointers I can look to figure out what is happening. > > The output is as follows (here I am testing with 40mbps UDP in iperf) > > 24ms: > > [ ID] Interval Transfer Bitrate Jitter > Lost/Total Datagrams > [ 5] 0.00-1.00 sec 4.89 MBytes 41.0 Mbits/sec 0.014 ms 0/3613 (0%) > [ 5] 1.00-2.00 sec 4.76 MBytes 40.0 Mbits/sec 0.019 ms 0/3517 (0%) > [ 5] 2.00-3.00 sec 4.77 MBytes 40.0 Mbits/sec 0.017 ms 0/3521 (0%) > [ 5] 3.00-4.00 sec 4.77 MBytes 40.0 Mbits/sec 0.018 ms 0/3521 (0%) > [ 5] 4.00-5.00 sec 4.77 MBytes 40.0 Mbits/sec 0.017 ms 0/3521 (0%) > [ 5] 5.00-6.00 sec 4.77 MBytes 40.0 Mbits/sec 0.018 ms 0/3524 (0%) > [ 5] 6.00-7.00 sec 4.77 MBytes 40.0 Mbits/sec 0.021 ms 0/3519 (0%) > [ 5] 7.00-8.00 sec 4.77 MBytes 40.0 Mbits/sec 0.020 ms 0/3521 (0%) > [ 5] 8.00-9.00 sec 4.77 MBytes 40.0 Mbits/sec 0.024 ms 0/3522 (0%) > [ 5] 9.00-10.00 sec 4.76 MBytes 40.0 Mbits/sec 0.023 ms 0/3517 (0%) > [ 5] 10.00-11.00 sec 4.77 MBytes 40.0 Mbits/sec 0.020 ms 0/3523 (0%) > [ 5] 11.00-12.00 sec 4.77 MBytes 40.0 Mbits/sec 0.016 ms 0/3522 (0%) > [ 5] 12.00-13.00 sec 4.76 MBytes 40.0 Mbits/sec 0.020 ms 0/3518 (0%) > [ 5] 13.00-14.00 sec 4.77 MBytes 40.0 Mbits/sec 0.018 ms 0/3522 (0%) > [ 5] 14.00-15.00 sec 4.77 MBytes 40.0 Mbits/sec 0.018 ms 0/3523 (0%) > [ 5] 15.00-16.00 sec 4.77 MBytes 40.0 Mbits/sec 0.022 ms 0/3520 (0%) > [ 5] 16.00-17.00 sec 4.77 MBytes 40.0 Mbits/sec 0.020 ms 0/3522 (0%) > [ 5] 17.00-18.00 sec 4.77 MBytes 40.0 Mbits/sec 0.019 ms 0/3520 (0%) > [ 5] 18.00-19.00 sec 4.76 MBytes 40.0 Mbits/sec 0.025 ms 0/3518 (0%) > [ 5] 19.00-20.00 sec 4.77 MBytes 40.0 Mbits/sec 0.022 ms 0/3524 (0%) > - - - - - - - - - - - - - - - - - - - - - - - - - > [ I
[vpp-dev] Policer download bandwidth reduces on high rtt connection
00 sec 2.03 MBytes 17.0 Mbits/sec 0.129 ms 1993/3493 (57%) [ 5] 18.00-19.00 sec 2.04 MBytes 17.1 Mbits/sec 0.132 ms 2059/3566 (58%) [ 5] 19.00-20.00 sec 2.12 MBytes 17.8 Mbits/sec 0.127 ms 1970/3533 (56%) - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-20.00 sec 95.4 MBytes 40.0 Mbits/sec 0.000 ms 0/70434 (0%) sender [SUM] 0.0-20.0 sec 14 datagrams received out-of-order [ 5] 0.00-20.00 sec 41.3 MBytes 17.3 Mbits/sec 0.127 ms 39957/70434 (57%) receiver 162ms: [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-1.00 sec 2.04 MBytes 17.1 Mbits/sec 0.149 ms 1976/3484 (57%) [ 5] 1.00-2.00 sec 2.08 MBytes 17.4 Mbits/sec 0.149 ms 1993/3527 (57%) [ 5] 2.00-3.00 sec 2.05 MBytes 17.2 Mbits/sec 0.160 ms 2015/3527 (57%) [ 5] 3.00-4.00 sec 1.99 MBytes 16.7 Mbits/sec 0.138 ms 2035/3508 (58%) [ 5] 4.00-5.00 sec 2.03 MBytes 17.0 Mbits/sec 0.137 ms 2025/3525 (57%) [ 5] 5.00-6.00 sec 2.02 MBytes 16.9 Mbits/sec 0.147 ms 2047/3537 (58%) [ 5] 6.00-7.00 sec 2.02 MBytes 16.9 Mbits/sec 0.137 ms 2060/3552 (58%) [ 5] 7.00-8.00 sec 2.01 MBytes 16.9 Mbits/sec 0.142 ms 1989/3473 (57%) [ 5] 8.00-9.00 sec 2.04 MBytes 17.1 Mbits/sec 0.152 ms 2068/3572 (58%) [ 5] 9.00-10.00 sec 2.02 MBytes 17.0 Mbits/sec 0.144 ms 2000/3493 (57%) [ 5] 10.00-11.00 sec 2.07 MBytes 17.3 Mbits/sec 0.136 ms 2006/3531 (57%) [ 5] 11.00-12.00 sec 2.07 MBytes 17.4 Mbits/sec 0.141 ms 2011/3540 (57%) [ 5] 12.00-13.00 sec 2.02 MBytes 16.9 Mbits/sec 0.133 ms 1968/3458 (57%) [ 5] 13.00-14.00 sec 2.03 MBytes 17.1 Mbits/sec 0.139 ms 2026/3528 (57%) [ 5] 14.00-15.00 sec 2.01 MBytes 16.9 Mbits/sec 0.136 ms 2031/3516 (58%) [ 5] 15.00-16.00 sec 2.07 MBytes 17.4 Mbits/sec 0.143 ms 2021/3551 (57%) [ 5] 16.00-17.00 sec 2.06 MBytes 17.3 Mbits/sec 0.138 ms 1972/3495 (56%) [ 5] 17.00-18.00 sec 2.07 MBytes 17.4 Mbits/sec 0.143 ms 2009/3537 (57%) [ 5] 18.00-19.00 sec 2.05 MBytes 17.2 Mbits/sec 0.165 ms 2050/3563 (58%) [ 5] 19.00-20.00 sec 2.06 MBytes 17.3 Mbits/sec 0.142 ms 1952/3473 (56%) - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-20.00 sec 96.0 MBytes 40.3 Mbits/sec 0.000 ms 0/70904 (0%) sender [SUM] 0.0-20.0 sec 67 datagrams received out-of-order [ 5] 0.00-20.00 sec 41.2 MBytes 17.3 Mbits/sec 0.106 ms 40505/70904 (57%) receiver Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14957): https://lists.fd.io/g/vpp-dev/message/14957 Mute This Topic: https://lists.fd.io/mt/69236177/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] QinQ and dot1ad any
On Tue, Dec 17, 2019 at 7:43 PM John Lo (loj) wrote: > Without exact match on a L3 sub-interface, VPP has no mechanism to know what > VLAN tags > to use for packet output, such as ARP request packets or IP packets, on that > sub-interface. For a big QinQ network (like say about 8 - 10 S-VLAN each with 4K C-VLANs) , the number of interface can become huge. Will that hit some internal VPP limits or will it become a performance problem? Is it a good idea to have a mechanism to provide this information to VPP via a control plane application, like whats is being done for PPPoE, so that only outer VLAN interface needs to be created, while inner VLAN interface will be provided by the control plane application? Is this a road worth traveling? Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14918): https://lists.fd.io/g/vpp-dev/message/14918 Mute This Topic: https://lists.fd.io/mt/68757125/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] QinQ and dot1ad any
Thanks a lot for the clarification! Raj On Tue, Dec 17, 2019 at 7:43 PM John Lo (loj) wrote: > > Hi Raj, > > A sub-interface with "dot1q inner any" can only work with L2 forwarding via > cross-connect or bridging where packets are forwarded using MAC header > without any changes to MAC header nor VLAN IDs in VLAN tags. > > For L3 IP forwarding, any VLAN tags on a packet must be exact match to a > sub-interface which means both outer and inner VLAN tag IDs must be > exact-matched to specific values defined of that sub-interface. Without > exact match on a L3 sub-interface, VPP has no mechanism to know what VLAN > tags to use for packet output, such as ARP request packets or IP packets, on > that sub-interface. Thus, sub-interface with "inner-dot1q any" is not an > exact match sub-interface by definition since no match is present on inner > tag. > > I suppose the CLI: > >> create sub-interfaces GigabitEthernet3/0/3 50 dot1ad 50 inner-dot1q any > >> exact-match > should have been rejected as exact match cannot be supported on the > sub-interface. This is something we should ideally fix in the CLI to avoid > any confusion with the meaning of exact match. > > Regards, > John > > -Original Message- > From: vpp-dev@lists.fd.io On Behalf Of Raj > Sent: Tuesday, December 17, 2019 4:39 AM > To: vpp-dev > Subject: [vpp-dev] QinQ and dot1ad any > > Hello all, > > When an interface is created using a command like: > > create sub-interfaces GigabitEthernet3/0/3 50 dot1ad 50 inner-dot1q any > exact-match > > I can see that dual tagged packets with outer vlan 50 will be accepted by > that interface. > > But how does a packet goes out from this interface? Is there a means by which > I can say that a given IP is with a specific S_VLAN:C_VLAN? > > During testing VPP could receive an ARP packet, but when it sends it out, > entire Ethernet header was missing. I guess there must be some means to add > adjacency information so that correct headers can be added. > > The trace is as follows: > > > 00:01:04:749110: dpdk-input > GigabitEthernet3/0/3 rx queue 0 > buffer 0x4c7d: current data 0, length 60, free-list 0, clone-count 0, > totlen-nifb 0, trace 0x1 > ext-hdr-valid > l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 60 > buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr > 0x26b31fc0 > packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid > ARP: 40:7b:1b:00:11:aa -> ff:ff:ff:ff:ff:ff 802.1ad vlan 50 802.1ad vlan 51 > request, type ethernet/IP4, address size 6/4 > 40:7b:1b:00:11:aa/192.168.25.1 -> 00:00:00:00:00:00/192.168.25.2 > 00:01:04:749112: ethernet-input > frame: flags 0x3, hw-if-index 1, sw-if-index 1 > ARP: 40:7b:1b:00:11:aa -> ff:ff:ff:ff:ff:ff 802.1ad vlan 50 802.1ad vlan 51 > 00:01:04:749113: arp-input > request, type ethernet/IP4, address size 6/4 > 40:7b:1b:00:11:aa/192.168.25.1 -> 00:00:00:00:00:00/192.168.25.2 > 00:01:04:749116: GigabitEthernet3/0/3-output > GigabitEthernet3/0/3.50 > 0x1235: 00:02:40:7b:1b:00 -> 00:01:08:00:06:04 > 00:01:04:749116: GigabitEthernet3/0/3-tx > GigabitEthernet3/0/3 tx queue 0 > buffer 0x4c7d: current data 22, length 38, free-list 0, clone-count 0, > totlen-nifb 0, trace 0x1 > ext-hdr-valid > l4-cksum-computed l4-cksum-correct vlan-2-deep l2-hdr-offset > 0 l3-hdr-offset 22 > PKT MBUF: port 0, nb_segs 1, pkt_len 38 > buf_len 2176, data_len 38, ol_flags 0x180, data_off 150, phys_addr > 0x26b31fc0 > packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid > 0x1235: 00:02:40:7b:1b:00 -> 00:01:08:00:06:04 > > btw, if inner VLAN is specified, everything works fine. > > > Thanks and Regards, > > Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14917): https://lists.fd.io/g/vpp-dev/message/14917 Mute This Topic: https://lists.fd.io/mt/68757125/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] QinQ and dot1ad any
Hello all, When an interface is created using a command like: create sub-interfaces GigabitEthernet3/0/3 50 dot1ad 50 inner-dot1q any exact-match I can see that dual tagged packets with outer vlan 50 will be accepted by that interface. But how does a packet goes out from this interface? Is there a means by which I can say that a given IP is with a specific S_VLAN:C_VLAN? During testing VPP could receive an ARP packet, but when it sends it out, entire Ethernet header was missing. I guess there must be some means to add adjacency information so that correct headers can be added. The trace is as follows: 00:01:04:749110: dpdk-input GigabitEthernet3/0/3 rx queue 0 buffer 0x4c7d: current data 0, length 60, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1 ext-hdr-valid l4-cksum-computed l4-cksum-correct PKT MBUF: port 0, nb_segs 1, pkt_len 60 buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr 0x26b31fc0 packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0 Packet Offload Flags PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid ARP: 40:7b:1b:00:11:aa -> ff:ff:ff:ff:ff:ff 802.1ad vlan 50 802.1ad vlan 51 request, type ethernet/IP4, address size 6/4 40:7b:1b:00:11:aa/192.168.25.1 -> 00:00:00:00:00:00/192.168.25.2 00:01:04:749112: ethernet-input frame: flags 0x3, hw-if-index 1, sw-if-index 1 ARP: 40:7b:1b:00:11:aa -> ff:ff:ff:ff:ff:ff 802.1ad vlan 50 802.1ad vlan 51 00:01:04:749113: arp-input request, type ethernet/IP4, address size 6/4 40:7b:1b:00:11:aa/192.168.25.1 -> 00:00:00:00:00:00/192.168.25.2 00:01:04:749116: GigabitEthernet3/0/3-output GigabitEthernet3/0/3.50 0x1235: 00:02:40:7b:1b:00 -> 00:01:08:00:06:04 00:01:04:749116: GigabitEthernet3/0/3-tx GigabitEthernet3/0/3 tx queue 0 buffer 0x4c7d: current data 22, length 38, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1 ext-hdr-valid l4-cksum-computed l4-cksum-correct vlan-2-deep l2-hdr-offset 0 l3-hdr-offset 22 PKT MBUF: port 0, nb_segs 1, pkt_len 38 buf_len 2176, data_len 38, ol_flags 0x180, data_off 150, phys_addr 0x26b31fc0 packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0 Packet Offload Flags PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 0x1235: 00:02:40:7b:1b:00 -> 00:01:08:00:06:04 btw, if inner VLAN is specified, everything works fine. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14912): https://lists.fd.io/g/vpp-dev/message/14912 Mute This Topic: https://lists.fd.io/mt/68757125/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Two potential bugs in nat/out2in
Hello All,
I want to share two observations I had while chasing a crash in VPP.
To the best of my knowledge, this is what happens in
src/plugins/nat/nat_det_out2in.c:icmp_match_out2in_det()
- First it will lookup the destination addr and get the snat_det_map if found.
- If no mapping found, check whether that destination address is an
Interface addr
- if not:
nat_log_info ("unknown dst address: %U",
format_ip4_address, >dst_address);
goto out;
In this case we are not setting the 'dont_translate' flag to true. Is
there any specific reason for not doing so? Could this be a bug?
In src/plugins/nat/out2in.c:icmp_out2in ()
Here 'snat_session_key_t sm0' is not initialized before it is used.
Since 'snat_session_key_t sm0' is not initialized and dont_translate
flag is false in the icmp_match_out2in_det() callback function,
address, port, protocol and fib_index will be set to uninitialized
values, which could be garbage.
This leads to packets being routed to unexpected locations, if
fib_index happens to be a valid value and vpp crashes
(assert(pool_elt_at_index(ip4_main.v4_fibs, index)) while trying to
fetch the pool with the garbage fib_index.
Is leaving snat_session_key_t sm0 uninitialized intentional, or should
it be initialized?
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#14587): https://lists.fd.io/g/vpp-dev/message/14587
Mute This Topic: https://lists.fd.io/mt/55922440/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Looking for memif performance numbers and configuration
On Mon, Nov 11, 2019 at 5:20 PM Jerome Tollet (jtollet) wrote: > > Did you have a look at > https://docs.fd.io/csit/rls1908/report/vpp_performance_tests/throughput_speedup_multi_core/container_memif.html# > ? Thanks for this link. I hadn't seen this before. raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14558): https://lists.fd.io/g/vpp-dev/message/14558 Mute This Topic: https://lists.fd.io/mt/52470809/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Looking for memif performance numbers and configuration
Hello all, I am looking to see if any one has attempted to connect two VPP using memif and what pps numbers is seen over memif interface? It would be great if the VPP config too can be shares so that I can recreate it locally. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14556): https://lists.fd.io/g/vpp-dev/message/14556 Mute This Topic: https://lists.fd.io/mt/52470809/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Connecting two vpp using memif caused severe performance drop
Hi all,
I am testing the performance of memif to connect multiple VPP. In my
simple test, I am finding that with each memif connections, overall
packet throughput gets drastically reduced.
1. I configured VPP and did a performance test.
command
```
set int ip address TenGigabitEthernet82/0/0 100.96.0.1/16
set int ip address TenGigabitEthernet85/0/1 172.30.1.2/29
set int state TenGigabitEthernet82/0/0 up
set int state TenGigabitEthernet85/0/1 up
```
Tested download from a client 100.96.2.1
The download speed of a 3GB file was 9.30Gb/s
2. Connected 2 VPP instances using memif
in VPP1
```
set int ip address TenGigabitEthernet82/0/0 100.96.0.1/16
set int state TenGigabitEthernet82/0/0 up
create interface memif id 1 socket-id 0 master
set int state memif0/1 up
set int ip address memif0/1 10.10.3.1/24
ip route add 0.0.0.0/0 via memif0/1 10.10.3.3
```
ran in main-core 1
in VPP2
```
set int state TenGigabitEthernet85/0/1 up
set int ip address TenGigabitEthernet85/0/1 172.30.1.2/29
create interface memif id 1 socket-id 0 slave
set int state memif0/1 up
set int ip address memif0/1 10.10.3.3/24
ip route add 0.0.0.0/0 via TenGigabitEthernet85/0/1 172.30.1.1
ip route add 100.96.0.0/16 via memif0/1 10.10.3.1
```
ran in main-core 2
Tested download from a client 100.96.2.1
The download speed of a 3GB file was 7.83Gb/s
3. connected 3 VPP instances
```
+--++--++--+
| VPP1 || VPP2 || VPP3 |
+--++--++--+
```
in VPP1
```
set int ip address TenGigabitEthernet82/0/0 100.96.0.1/16
set int state TenGigabitEthernet82/0/0 up
comment{create interface memif id 1 socket-id 0 master}
create memif socket id 2 filename /tmp/vppsh/memif.sock
create interface memif id 0 socket-id 2 master
set int state memif2/0 up
set int ip address memif2/0 10.10.2.1/24
ip route add 0.0.0.0/0 via memif2/0 10.10.2.2
```
ran in main-core 1
in VPP2
```
create memif socket id 2 filename /tmp/vppsh/memif.sock
create interface memif id 0 socket-id 2 slave
set int state memif2/0 up
set int ip address memif2/0 10.10.2.2/24
create interface memif id 1 socket-id 0 master
set int state memif0/1 up
set int ip address memif0/1 10.10.3.2/24
ip route add 100.96.0.0/16 via memif2/0 10.10.2.1
ip route add 0.0.0.0/0 via memif0/1 10.10.3.3
```
ran in main-core 2
in VPP3
```
set int state TenGigabitEthernet85/0/1 up
set int ip address TenGigabitEthernet85/0/1 172.30.1.2/29
create interface memif id 1 socket-id 0 slave
set int state memif0/1 up
set int ip address memif0/1 10.10.3.3/24
ip route add 10.10.2.0/24 via memif0/1 10.10.3.2
ip route add 0.0.0.0/0 via TenGigabitEthernet85/0/1 172.30.1.1
ip route add 100.96.0.0/16 via memif0/1 10.10.3.2
```
ran in main-core 3
tested download from a client 100.96.2.1
The download speed of a 3GB file was 3.99Gb/s
As it can be seen, the performance drops from 9.3 to 7.83 to 3.99, as
number of memif interconnect increases.
Is this the expected behavior or did I miss some tuning?
The CPU is Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz, total ram is
16GB CPU pinning is not done. vpp version is: 19.01.3-rc0
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#14545): https://lists.fd.io/g/vpp-dev/message/14545
Mute This Topic: https://lists.fd.io/mt/47761827/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP core dump with PPPoE
On Thu, Oct 17, 2019 at 8:45 PM Neale Ranns (nranns) wrote: > I think your analysis is spot on. Thank you! > How you're a VPP PPPoE expert __ do you have some suggestions on a fix? Thanks for taking a look at this Neale! I have some ideas for a fix to this problem. Alternative 1. During the packet processing time, get the session information by using the adj->rewrite_header.sw_if_index . With this approach fixup_data need not be passed. + pppoe_main_t *pem = _main; + + u32 sw_if_index = adj->rewrite_header.sw_if_index; + u32 session_id = pem->session_index_by_sw_if_index[sw_if_index]; + t = pool_elt_at_index (pem->sessions, session_id); Pros : Without any functin prototype change we can arrive at the solution. Cons : While processing every pppoe packet, we have to get the session data. Alternative 2. pppoe build rewrite is done based on rewrite data(adj->rewrite_header). Here we have a sw_if_index(adj->rewrite_header.sw_if_index). How is this is different from the session's encap_if_index? I tried to understand this from code, but couldn't figure out, if both are the same. If both are same: Fix would be : At the time processing pppoe packet, just use the 'adj->rewrite_header.sw_if_index'. Alternative 3. As Neale suggested, just push the session's encap_if_index, instead of passing the const void *. But this requires some architectural changes, I believe. Because const void *fixup_data is depended on the function pointer 'adj->sub_type.midchain.fixup_func'. This can bind only during the run time. I have one question regarding pool memory creation: On each pppoe session addition VPP is resizing the vector pool pem->sessions. Is it ok if I allocate a fixed number of session initially? I am trying to understand the architectural rational behind adding one element to the vector pool at a time, as opposed to allocating a bulk number initially. If it is ok to allocate a large pool initially (preferably taking the max_sessions from a conf file), we can side step this issue. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14232): https://lists.fd.io/g/vpp-dev/message/14232 Mute This Topic: https://lists.fd.io/mt/34298895/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP core dump with PPPoE
Hello all, I could do some more analysis of this issue and I think I have identified what could be wrong here. During pppoe session creation time frame, as we mentioned earlier, the below function will get invoked from pppoe_update_adj() adj_nbr_midchain_update_rewrite(adj_index_t adj_index,adj_midchain_fixup_t fixup,const void *fixup_data,...) Note the 3rd arg 'fixup_data', here we are passing the current session address as fixup_data When subsequent sessions are added this memory address will get altered as we are resizing the vector pool pem->sessions. Hence the above stored fixup_data (adj->sub_type.midchain.fixup_data) address is no longer valid, and that address could have been already freed. I think that is the reason why we see memory corruption. It would be great if some one with far better knowledge of VPP internals could take a look at this and confirm this. Thanks and Regards, Raj On Fri, Oct 11, 2019 at 12:17 PM Ni, Hongjun wrote: > > Hi Raj, > > I tried to reproduce your issue on VPP 20.01 as per your steps for some > times, but cannot reproduce it. > > From your description, please set a breakpoint in > vnet_pppoe_add_del_session(). > And to see what happened when you create your second pppoe session with > traffic in first pppoe session. > > Thanks, > Hongjun > > -Original Message- > From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Raj > Sent: Monday, September 30, 2019 11:54 PM > To: vpp-dev > Subject: Re: [vpp-dev] VPP core dump with PPPoE > > Hello all, > > I did some more debugging to find out when and where exactly the > pppoe_session_t get corrupted. Added couple of log entries as shown below to > log pppoe session id when a session is created as well as when packets from > north traverses to south. I have tried this in VPP 19.08, 19.04 and 19.01 > with same results. > > vnet [21892]: pppoe_update_adj:195: New_Session pppoe01 session id 20923 vnet > [21892]: pppoe_update_adj:195: New_Session pppoe01 session id 35666 > vnet [21892]: pppoe_fixup:169: New_Packet pppoe01 session id 35666 > vnet [21892]: pppoe_update_adj:195: New_Session pppoe01 session id 58191 > > The sequence when corruption happens seems to be: > > 1. A new session is created > 2. A packet for the newly created session traverses from north to south 3. > Next session is created - and vpp crashes. > > Digging deeper, I added watch for all newly created sessions using the > following gdb script > > b /root/build-1901/src/vnet/ip/ip4_forward.c:2444 > commands 1 > watch -l ((pppoe_session_t*)adj0->sub_type.midchain.fixup_data).session_id > bt > continue > end > > gdb, running with this script, bails out with following message: > > Thread 1 "vpp_main" hit Hardware watchpoint 2: -location > ((pppoe_session_t*)adj0->sub_type.midchain.fixup_data).session_id > > Old value = 35666 > New value = 4883 > __memset_avx2_unaligned_erms () at > ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:203 > 203 ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: No > such file or directory. > (gdb) > > It is interesting to note that 4883 is 0x1313 > > Back trace shows the path it took to reach here: > > > (gdb) bt > #0 __memset_avx2_unaligned_erms () at > ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:203 > #1 0x761a4179 in mspace_put (msp=0x7fffb4df7010, > p_arg=0x7fffb592d9c8) at /root/build-1901/src/vppinfra/dlmalloc.c:4294 > #2 0x7618ea39 in clib_mem_free (p=0x7fffb592d9c8) at > /root/build-1901/src/vppinfra/mem.h:215 > #3 0x7618edd8 in vec_resize_allocate_memory (v=0x7fffb592da00, > length_increment=1, data_bytes=312, header_bytes=56, data_align=64) at > /root/build-1901/src/vppinfra/vec.c:96 > #4 0x7fffb0aa4a29 in _vec_resize_inline (v=0x7fffb592da00, > length_increment=1, data_bytes=256, header_bytes=48, data_align=64) at > /root/build-1901/src/vppinfra/vec.h:147 > #5 0x7fffb0aa9ca4 in vnet_pppoe_add_del_session (a=0x7fffb6703950, > sw_if_indexp=0x7fffb67038e8) at > /root/build-1901/src/plugins/pppoe/pppoe.c:335 > #6 0x7fffb0aaadec in pppoe_add_del_session_command_fn > (vm=0x768e3400 , input=0x7fffb6703ee0, > cmd=0x7fffb65dd73c) at /root/build-1901/src/plugins/pppoe/pppoe.c:554 > #7 0x76617db0 in vlib_cli_dispatch_sub_commands > (vm=0x768e3400 , cm=0x768e3600 > , input=0x7fffb6703ee0, parent_command_index=21) at > /root/build-1901/src/vlib/cli.c:644 > > This do not occur if traffic is not initiated, and there is no packet flow > through the system. It would be great if some one who understands this code > to confirm if my analysis is correct and give some pointers to figure out > >
Re: [vpp-dev] VPP core dump with PPPoE
Hello all, I did some more debugging to find out when and where exactly the pppoe_session_t get corrupted. Added couple of log entries as shown below to log pppoe session id when a session is created as well as when packets from north traverses to south. I have tried this in VPP 19.08, 19.04 and 19.01 with same results. vnet [21892]: pppoe_update_adj:195: New_Session pppoe01 session id 20923 vnet [21892]: pppoe_update_adj:195: New_Session pppoe01 session id 35666 vnet [21892]: pppoe_fixup:169: New_Packet pppoe01 session id 35666 vnet [21892]: pppoe_update_adj:195: New_Session pppoe01 session id 58191 The sequence when corruption happens seems to be: 1. A new session is created 2. A packet for the newly created session traverses from north to south 3. Next session is created - and vpp crashes. Digging deeper, I added watch for all newly created sessions using the following gdb script b /root/build-1901/src/vnet/ip/ip4_forward.c:2444 commands 1 watch -l ((pppoe_session_t*)adj0->sub_type.midchain.fixup_data).session_id bt continue end gdb, running with this script, bails out with following message: Thread 1 "vpp_main" hit Hardware watchpoint 2: -location ((pppoe_session_t*)adj0->sub_type.midchain.fixup_data).session_id Old value = 35666 New value = 4883 __memset_avx2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:203 203 ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: No such file or directory. (gdb) It is interesting to note that 4883 is 0x1313 Back trace shows the path it took to reach here: (gdb) bt #0 __memset_avx2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:203 #1 0x761a4179 in mspace_put (msp=0x7fffb4df7010, p_arg=0x7fffb592d9c8) at /root/build-1901/src/vppinfra/dlmalloc.c:4294 #2 0x7618ea39 in clib_mem_free (p=0x7fffb592d9c8) at /root/build-1901/src/vppinfra/mem.h:215 #3 0x7618edd8 in vec_resize_allocate_memory (v=0x7fffb592da00, length_increment=1, data_bytes=312, header_bytes=56, data_align=64) at /root/build-1901/src/vppinfra/vec.c:96 #4 0x7fffb0aa4a29 in _vec_resize_inline (v=0x7fffb592da00, length_increment=1, data_bytes=256, header_bytes=48, data_align=64) at /root/build-1901/src/vppinfra/vec.h:147 #5 0x7fffb0aa9ca4 in vnet_pppoe_add_del_session (a=0x7fffb6703950, sw_if_indexp=0x7fffb67038e8) at /root/build-1901/src/plugins/pppoe/pppoe.c:335 #6 0x7fffb0aaadec in pppoe_add_del_session_command_fn (vm=0x768e3400 , input=0x7fffb6703ee0, cmd=0x7fffb65dd73c) at /root/build-1901/src/plugins/pppoe/pppoe.c:554 #7 0x76617db0 in vlib_cli_dispatch_sub_commands (vm=0x768e3400 , cm=0x768e3600 , input=0x7fffb6703ee0, parent_command_index=21) at /root/build-1901/src/vlib/cli.c:644 This do not occur if traffic is not initiated, and there is no packet flow through the system. It would be great if some one who understands this code to confirm if my analysis is correct and give some pointers to figure out 1. Why, when a new session is created, the data of an old session is changed to 0x1313 2. What debugging steps can I take next to figure out why this happens. Thanks and Regards, Raj On Sat, Sep 28, 2019 at 6:09 PM Raj via Lists.Fd.Io wrote: > > Hello all, > > I have done some more tests to pinpoint the exact condition of the > crash. What I could figure out was that the crash happens when memory > is being allocated for pppoe_session_t while packets are flowing > through pppoe interface. > > Here is what I did to arrive at this conclusion: > > 1. Configure VPP without any default route (to ensure packets do not > hit north interface from south) > 2. Provision 100 PPPoE clients - No crash observed > 3. Deprovision all 100 PPPoE clients > 4. Configure default route > 5. Provision 100 PPPoE clients again, and start a ping to an external > IP from each client - No Crash observed > 6. Provision 50 more PPPoE clients - VPP crashes. > > Based on this test, and from what I could understand from the code, my > guess is that there is some memory corruption happening inside the > pppoe_session_t when memory is being allocated for it when there is > packets traversing through PPPoE interface. > > Thanks and Regards, > > Raj > > > On Thu, Sep 26, 2019 at 7:15 PM Raj via Lists.Fd.Io > wrote: > > > > Hello all, > > > > I am observing a VPP crash when approximately 20 - 50 PPPoE clients > > are connecting and traffic is flowing through them. This crash was > > reproducible every time I tried. > > > > I did some debugging and here is what I could find out so far: > > > > If I understand correctly, when a incoming packet from north side is > > being sent to PPPoE interface, pppoe_fixup() is called to update > > pppoe0->length, and t->encap_if_index. Length
Re: [vpp-dev] VPP core dump with PPPoE
Hello all, I have done some more tests to pinpoint the exact condition of the crash. What I could figure out was that the crash happens when memory is being allocated for pppoe_session_t while packets are flowing through pppoe interface. Here is what I did to arrive at this conclusion: 1. Configure VPP without any default route (to ensure packets do not hit north interface from south) 2. Provision 100 PPPoE clients - No crash observed 3. Deprovision all 100 PPPoE clients 4. Configure default route 5. Provision 100 PPPoE clients again, and start a ping to an external IP from each client - No Crash observed 6. Provision 50 more PPPoE clients - VPP crashes. Based on this test, and from what I could understand from the code, my guess is that there is some memory corruption happening inside the pppoe_session_t when memory is being allocated for it when there is packets traversing through PPPoE interface. Thanks and Regards, Raj On Thu, Sep 26, 2019 at 7:15 PM Raj via Lists.Fd.Io wrote: > > Hello all, > > I am observing a VPP crash when approximately 20 - 50 PPPoE clients > are connecting and traffic is flowing through them. This crash was > reproducible every time I tried. > > I did some debugging and here is what I could find out so far: > > If I understand correctly, when a incoming packet from north side is > being sent to PPPoE interface, pppoe_fixup() is called to update > pppoe0->length, and t->encap_if_index. Length and encap_if_index is > taken from adj0->sub_type.midchain.fixup_data > > My observation is that while clients are connecting and traffic is > flowing for connected clients, adj0->sub_type.midchain.fixup_data > appears to hold incorrect data, at some point in time, during the > test. What we have seen is the incorrect data > (adj0->sub_type.midchain.fixup_data) is observed for clients which are > already provisioned for some time and which had packets flowing > through them. > > I figured this out by using gdb and inspecting > adj0->sub_type.midchain.fixup_data, after typecasting it into > pppoe_session_t > > In the structure, I could see that session_id, client_ip and encap_idx > are incorrect. I did not check other values in the structure. > > I also added code to log this fields in pppoe_fixup() and logs too > shows incorrect data in the fields. > > Example logs taken just before crash: > > vnet[12988]: pppoe_fixup:243: 40:7b:1b: 0:12:38 -> 2:42: a: 1: 0: 2 , type > 8864 > vnet[12988]: pppoe_fixup:271: pppoe session id 4883, client_ip > 0x13131313 encap idx 0x13131313 > > First log prints out packet headers, to verify that data in packet is > as expected and is correct. Second log prints values in pppoe_session > data, and it can be seen that the values are obviously incorrect. At > this point the packet is sent out through the south interface. Again > after some time the TX index values become some thing similar to > 1422457436 and VPP core dumps. > > We have tested the following scenarios: > > 1. Add PPPoE clients without sending out any traffic: There is no > crash observed. > 2. Add n number of PPPoE clients, load traffic [No adding or removal > or clients while traffic is on, see next scenario]: There is no crash > observed > 3. Load traffic as soon as each client connects: VPP crash observed. > > Another observation is that encap_if_index is available in two places > inside pppoe_fixup: > > 1. adj->rewrite_header.sw_if_index > 2. t->encap_if_index > > t->encap_if_index is used for updating TX, and this gets corrupted, > while adj->rewrite_header.sw_if_index has the correct index. > > I can check and get back if you need any additional information. Let > me know if a bug report is to be created for this. > > Environment: > > vpp# show version verbose > Version: v19.08.1-59~ga2aa83ca9-dirty > Compiled by: root > Compile host: build-02 > Compile date: Thu Sep 26 16:44:00 IST 2019 > Compile location: /root/build-1908 > Compiler: GCC 7.4.0 > Current PID: 7802 > > Operating system: Ubuntu 18.04 amd64 > > startup.conf and associated exec file is attached. > > There is a small patch to stock VPP to disable > ETHERNET_ERROR_L3_MAC_MISMATCH, which is attached. I have also > attached output of show show hardware and gdb bt output. I have the > core file and its matching VPP debs, and can be shared if needed. > > In the bt the incorrect value of index can be seen in bt #5: > > #5 0x7fba88e9ce0b in vlib_increment_combined_counter > (n_bytes=, n_packets=1, index=538976288, > thread_index=0, cm=0x7fba481f46a0) at > /root/build-1908/src/vlib/counter.h:229 > > Thanks and Regards, > > Raj >
[vpp-dev] VPP core dump with PPPoE
Hello all, I am observing a VPP crash when approximately 20 - 50 PPPoE clients are connecting and traffic is flowing through them. This crash was reproducible every time I tried. I did some debugging and here is what I could find out so far: If I understand correctly, when a incoming packet from north side is being sent to PPPoE interface, pppoe_fixup() is called to update pppoe0->length, and t->encap_if_index. Length and encap_if_index is taken from adj0->sub_type.midchain.fixup_data My observation is that while clients are connecting and traffic is flowing for connected clients, adj0->sub_type.midchain.fixup_data appears to hold incorrect data, at some point in time, during the test. What we have seen is the incorrect data (adj0->sub_type.midchain.fixup_data) is observed for clients which are already provisioned for some time and which had packets flowing through them. I figured this out by using gdb and inspecting adj0->sub_type.midchain.fixup_data, after typecasting it into pppoe_session_t In the structure, I could see that session_id, client_ip and encap_idx are incorrect. I did not check other values in the structure. I also added code to log this fields in pppoe_fixup() and logs too shows incorrect data in the fields. Example logs taken just before crash: vnet[12988]: pppoe_fixup:243: 40:7b:1b: 0:12:38 -> 2:42: a: 1: 0: 2 , type 8864 vnet[12988]: pppoe_fixup:271: pppoe session id 4883, client_ip 0x13131313 encap idx 0x13131313 First log prints out packet headers, to verify that data in packet is as expected and is correct. Second log prints values in pppoe_session data, and it can be seen that the values are obviously incorrect. At this point the packet is sent out through the south interface. Again after some time the TX index values become some thing similar to 1422457436 and VPP core dumps. We have tested the following scenarios: 1. Add PPPoE clients without sending out any traffic: There is no crash observed. 2. Add n number of PPPoE clients, load traffic [No adding or removal or clients while traffic is on, see next scenario]: There is no crash observed 3. Load traffic as soon as each client connects: VPP crash observed. Another observation is that encap_if_index is available in two places inside pppoe_fixup: 1. adj->rewrite_header.sw_if_index 2. t->encap_if_index t->encap_if_index is used for updating TX, and this gets corrupted, while adj->rewrite_header.sw_if_index has the correct index. I can check and get back if you need any additional information. Let me know if a bug report is to be created for this. Environment: vpp# show version verbose Version: v19.08.1-59~ga2aa83ca9-dirty Compiled by: root Compile host: build-02 Compile date: Thu Sep 26 16:44:00 IST 2019 Compile location: /root/build-1908 Compiler: GCC 7.4.0 Current PID: 7802 Operating system: Ubuntu 18.04 amd64 startup.conf and associated exec file is attached. There is a small patch to stock VPP to disable ETHERNET_ERROR_L3_MAC_MISMATCH, which is attached. I have also attached output of show show hardware and gdb bt output. I have the core file and its matching VPP debs, and can be shared if needed. In the bt the incorrect value of index can be seen in bt #5: #5 0x7fba88e9ce0b in vlib_increment_combined_counter (n_bytes=, n_packets=1, index=538976288, thread_index=0, cm=0x7fba481f46a0) at /root/build-1908/src/vlib/counter.h:229 Thanks and Regards, Raj vpp# show version verbose Version: v19.08.1-59~ga2aa83ca9-dirty Compiled by: root Compile host: build-02 Compile date: Thu Sep 26 16:44:00 IST 2019 Compile location: /root/build-1908 Compiler: GCC 7.4.0 Current PID: 7802 vpp# show hardware NameIdx Link Hardware GigabitEthernet3/0/1 1 up GigabitEthernet3/0/1 Link speed: 1 Gbps Ethernet address 40:7b:1b:00:12:33 Intel e1000 carrier up full duplex mtu 9206 flags: pmd maybe-multiseg tx-offload intel-phdr-cksum rx-ip4-cksum rx: queues 1 (max 8), desc 1024 (min 32 max 4096 align 8) tx: queues 1 (max 8), desc 1024 (min 32 max 4096 align 8) pci: device 8086:1521 subsystem 15bb:0008 address :03:00.01 numa 0 max rx packet len: 16383 promiscuous: unicast off all-multicast off vlan offload: strip off filter off qinq off rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum vlan-filter jumbo-frame scatter keep-crc rx offload active: ipv4-cksum jumbo-frame scatter tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-cksum tcp-tso multi-segs tx offload active: udp-cksum tcp-cksum multi-segs rss avail: ipv4-tcp ipv4-udp ipv4 ipv6-tcp-ex ipv6-udp-ex ipv6-tcp ipv6-udp ipv6-ex ipv6 rss active:none tx burst functio
[vpp-dev] Checking for enabled feature
Hello all,
I am in need to find out whether a particular feature is enabled on an
interface during run time.
So if I understand correctly we have to get the corresponding 'feature
arc index '. Then get the config index by using
'config_index_by_sw_if_index' then get the config_pool and then
iterate through the features list to check whether the given feature
is there or not.
What I would like to know, is this is the correct approach or is there
any optimized and a better performing alternative for achieving the
same
current_config_index =
vec_elt (cm[feature_arc].config_index_by_sw_if_index, sw_if_index);
cfg_index = vcm->config_pool_index_by_user_index[current_config_index];
cfg = pool_elt_at_index (vcm->config_pool, cfg_index);
for (i = 0; i < vec_len (cfg->features); i++)
{
feat = cfg->features + i;
node_index = feat->node_index;
n = vlib_get_node (vm, node_index);
if (0 == memcmp (n->name, input_feat,
sizeof("ip4-policer-classify") - 1))
return 1;
}
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#13392): https://lists.fd.io/g/vpp-dev/message/13392
Mute This Topic: https://lists.fd.io/mt/32241517/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Routing of ipv6 packets over PPPoE
Hello all, I have PPPoE working successfully over IPv4, and able to ping outside. IPv6 negotiation by control plane is successful and an IPv6 session is created using the following command: create pppoe session client-ip client-ip fe80::60bb:6a:38cc:6b03 session-id 25740 client-mac 08:00:27:44:cd:23 vpp# show pppoe session ... [1] sw-if-index 5 client-ip fe80::60bb:6a:38cc:6b03 session-id 25740 encap-if-index 1 decap-fib-index 0 local-mac 08:00:27:47:6e:4b client-mac 08:00:27:44:cd:23 So far its fine. Next I need to allocate a block of IPv6 address to the PPPoE client using DHCP-PD. When a block of IP address is allocated to a PPPoE client, how does the routing work? How will VPP route the packets, ingress from north, belonging to a network, to correct PPPoE encapsulation, including MAC and PPPoE ID? I guess I am missing some routing command here. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12912): https://lists.fd.io/g/vpp-dev/message/12912 Mute This Topic: https://lists.fd.io/mt/31456403/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Using Prometheus Plugin
Hi all, Any one with experience in using Prometheus plugin for VPP stats monitoring? Any help in getting started with the Prometheus plugin is much appreciated. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12854): https://lists.fd.io/g/vpp-dev/message/12854 Mute This Topic: https://lists.fd.io/mt/31341940/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Using wildcard-ip4-arp-publisher-process
Hi all, I am trying to get ARP events from VPP and found that there is a 'wildcard-ip4-arp-publisher-process' process which could be used. To use this, I registered a client using API `vpp.api.want_ip4_arp_events` using 0 as IP address argument. I could see that registration is happening and `wc_arp_set_publisher_node` function was getting invoked, as it should. But the node was not reporting any ARP events to the client. Looking through the code, the main function in the node, `wc_arp_process (vlib_main_t vm, vlib_node_runtime_t rt, vlib_frame_t * f)` is not getting invoked. I also registered another client, using `vpp.api.want_ip4_arp_events` but using a valid IP addres as IP address in argument. Here I'm able to get the ARP events in the client process from ip-route-resolver-process, as expected. I just need to figure out why wildcard-ip4-arp-publisher-process is not reporting ARP events, though ip-route-resolver-process is doing so. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12812): https://lists.fd.io/g/vpp-dev/message/12812 Mute This Topic: https://lists.fd.io/mt/31223473/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Classify session weird behavior
Hello all,
I found a work around to this:
{
"classify-table": [
{
"name": "table0",
"classify-session": [
{
"match":
"00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:64:45:01:04:00:00:00:00:00:00:00:00:00:25:05:30:01:00",
"advance": 0,
"hit_next": "deny",
"opaque_index": 1
}
],
"active_sessions": 1,
"skip_n_vectors": 0,
"nbuckets": 2,
"mask":
"00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:ff:ff:ff:ff:00:00:00:00:00:00:00:00:00:00:00:00:00:00",
"miss_next": "permit"
}
]
}
I can create a classify table like this, with skip_n_vectors as zero,
but with a longer mask and it works fine.
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12704): https://lists.fd.io/g/vpp-dev/message/12704
Mute This Topic: https://lists.fd.io/mt/30810146/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Classify session weird behavior
On Thu, Mar 28, 2019 at 4:38 PM Michal Cmarada -X (mcmarada - PANTHEON
TECHNOLOGIES at Cisco) wrote:
>
> Raj: If you have more data available after testing, please provide them here.
> Thanks
My observation is when I use skip of 1, cli works fine while RESTCONF do not.
Config using CLI:
vpp# classify table mask l3 ip4 src skip 1
vpp# configure policer name policy1 cir 500 eir 0 cb 5000 eb 15000
rate kbps round closest type 1r3c conform-action transmit
exceed-action mark-and-transmit AF22 violate-action drop
vpp# classify session policer-hit-next policy1 exceed-color
table-index 0 match l3 ip4 src 192.168.1.2
vpp# sh classify tables verbose
TableIdx Sessions NextTbl NextNode
0 1-1-1
Heap: total: 2.06M, used: 1.27K, free: 2.06M, trimmable: 2.06M
no traced allocations
nbuckets 2, skip 1 match 1 flag 0 offset 0
mask
linear-search buckets 0
[1]: heap offset 1136, elts 2, normal
0: [1136]: next_index 0 advance 0 opaque 1 action 0 metadata 0
k: c0a80102
hits 0, last_heard 0.00
1 active elements
1 free lists
0 linear-search buckets
The `k` value is what is expected.
Config using RESTCONF:
PUT
https://{{machine}}/restconf/config/vpp-classifier:vpp-classifier/classify-table/table0
json
{
"classify-table": [
{
"name": "table0",
"nbuckets": "2",
"memory_size": "1073857",
"miss_next": "permit",
"skip_n_vectors": "1",
"mask": "00:00:00:00:00:00:00:00:00:00:ff:ff:ff:ff:00:00"
}
]
}
adding limiting for src ip
PUT
https://{{machine}}/restconf/config/vpp-classifier:vpp-classifier/classify-table/table0/classify-session/00:00:00:00:00:00:00:00:00:00:c0:a8:1e:8e:00:01
json
{
"classify-session": [
{
"policer_hit_next": "policy1",
"color_classfier": "exceed-color",
"match": "00:00:00:00:00:00:00:00:00:00:c0:a8:1e:8e:00:01"
}
]
}
vpp# sh policer
Name "policy1" type 1r3c cir 500 eir 0 cb 5000 eb 15000
rate type kbps, round type closest
conform action transmit, exceed action mark-and-transmit CS0, violate
action drop
Template policer at 7fd3f972de5c: single rate, not color-aware
cir 825955 tok/period, pir 1 tok/period, scale 18
cur lim 131072, cur bkt 131072, ext lim 393216, ext bkt 393216
last update 0
---
vpp#
vpp#
vpp# sh classify tables verbose
TableIdx Sessions NextTbl NextNode
0 1-1-1
Heap: total: 1.06M, used: 1.27K, free: 1.06M, trimmable: 1.06M
no traced allocations
nbuckets 2, skip 1 match 1 flag 0 offset 0
mask
linear-search buckets 0
[0]: heap offset 1136, elts 2, normal
0: [1136]: next_index 0 advance 0 opaque 1 action 0 metadata 0
k:
hits 0, last_heard 0.00
1 active elements
1 free lists
0 linear-search buckets
As it can be seen `k` is all zeros here.
@Michal Cmarada: I am curious as to why a skip value of 12 was chosen?
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12660): https://lists.fd.io/g/vpp-dev/message/12660
Mute This Topic: https://lists.fd.io/mt/30810146/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] DHCPv6 proxy not working over VLAN interface
Hi all,
I am trying to assign v6 address to clients using DHCP PD. I have ISC
Bind configured to provide v6 prefixes and configured dhcpv6 proxy
using the following command.
vppctl set dhcpv6 proxy server 2001:xxx::10d::1 src-address
2001:xxx::10d::700
It is working for clients from non VLAN interface but fails for
clients in VLAN interface. It took some time to figure out but
ultimately the fix looked trivial.
diff --git a/src/vnet/dhcp/dhcp6_proxy_node.c b/src/vnet/dhcp/dhcp6_proxy_node.c
index 7d157ad35..40dc220d6 100644
--- a/src/vnet/dhcp/dhcp6_proxy_node.c
+++ b/src/vnet/dhcp/dhcp6_proxy_node.c
@@ -775,7 +775,7 @@ dhcpv6_proxy_to_client_input (vlib_main_t * vm,
{
u32 *vlan_tag = (u32 *) (mac0 + 1);
u32 tmp;
- tmp = (si0->sub.id << 16) | 0x0800;
+ tmp = (si0->sub.id << 16) | 0x86dd;
*vlan_tag = clib_host_to_net_u32 (tmp);
}
With this patch applied, DHCPv6 proxy is working correctly over VLANs.
I have couple of observations as I was testing the this feature.
1. Trace of proxy to client packets does not show any output interface
nodes and stops abruptly at dhcpv6-proxy-to-client node
14:49:51:731269: ip6-local
UDP: 2001:xxx::10d::1 -> 2001:xxx::10d::700
tos 0x00, flow label 0x0, hop limit 64, payload length 219
UDP: 547 -> 547
length 219, checksum 0xa3ef
14:49:51:731271: ip6-udp-lookup
UDP: src-port 547 dst-port 547
14:49:51:731274: dhcpv6-proxy-to-client
DHCPV6 proxy: sent to client from 2001:xxx::700::1
original_sw_if_index: 10, sw_if_index: 10
While the proxy to server packet trace shows next node as ip6-lookup
and it goes till TenGigabitEthernet85/0/1-tx
14:49:51:730918: dhcpv6-proxy-to-server
DHCPV6 proxy: sent to server 2001:xxx::10d::1
original_sw_if_index: 10, sw_if_index: 10
14:49:51:730922: ip6-lookup
fib 0 dpo-idx 4 flow hash: 0x
UDP: 2001:xxx::10d::700 -> 2001:xxx::10d::1
tos 0x00, flow label 0x0, hop limit 32, payload length 163
UDP: 546 -> 547
2. There is a comment in the code which shows a TODO item is pending
in the code:
/* $$$ consider adding a dynamic next to the graph node, for performance */
I could not figure out what the OP meant by this comment, but I can
attempt to do it if some one here can explain what needs to be done.
Is the abrupt ending of the trace and the above TODO related?
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12615): https://lists.fd.io/g/vpp-dev/message/12615
Mute This Topic: https://lists.fd.io/mt/30706811/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Status of PPPoE Plugin
I commented out the offending parts and now PPPoE is working fine.
diff --git a/src/vnet/ethernet/node.c b/src/vnet/ethernet/node.c
index 53d5b4eb0..7c5f673e0 100755
--- a/src/vnet/ethernet/node.c
+++ b/src/vnet/ethernet/node.c
@@ -429,14 +429,14 @@ ethernet_input_inline (vlib_main_t * vm,
}
else
{
+ /*if (!ethernet_address_cast (e0->dst_address) &&
(hi->hw_address != 0) &&
!eth_mac_equal ((u8 *) e0, hi->hw_address))
error0 = ETHERNET_ERROR_L3_MAC_MISMATCH;
if (!ethernet_address_cast (e1->dst_address) &&
(hi->hw_address != 0) &&
!eth_mac_equal ((u8 *) e1, hi->hw_address))
+error1 = ETHERNET_ERROR_L3_MAC_MISMATCH;*/
vlib_buffer_advance (b0, sizeof (ethernet_header_t));
determine_next_node (em, variant, 0, type0, b0,
, );
@@ -653,10 +653,10 @@ ethernet_input_inline (vlib_main_t * vm,
}
else
{
+ /*if (!ethernet_address_cast (e0->dst_address) &&
(hi->hw_address != 0) &&
!eth_mac_equal ((u8 *) e0, hi->hw_address))
+error0 = ETHERNET_ERROR_L3_MAC_MISMATCH;*/
vlib_buffer_advance (b0, sizeof (ethernet_header_t));
determine_next_node (em, variant, 0, type0, b0,
, );
I am sure this is not a patch which will be accepted upstream. I am
not sure how to _correctly_ fix this, so that it will be accepted by
upstream. If some pointers are given I can submit a patch to get PPPoE
working correctly again in VPP.
Thanks and Regards,
Raj
On Tue, Mar 19, 2019 at 1:09 PM Raj via Lists.Fd.Io
wrote:
>
> Another approach that can be used is to send all PPP/PPPoE control
> packets encapsulated in NSH, and Control plane can process it and
> return the packets to VPP which will decapsulate the NSH headers and
> pass the packets on to PPPoE client.
>
> Thanks and Regards,
>
> Raj
>
> On Mon, Mar 18, 2019 at 7:44 PM Raj via Lists.Fd.Io
> wrote:
> >
> > On Thu, Dec 20, 2018 at 6:17 AM Ni, Hongjun wrote:
> >
> > > For the issue you mentioned, Alp Arslan in the To list will submit a
> > > patch to fix it.
> >
> > I can take a stab at this and prepare a patch to get PPPoE working
> > again in VPP. I understand that this error happens because VPP will
> > only allow packets whose destination MAC is the same as the interface
> > MAC or bcast/mcast MAC. How should I go about fixing this? Would
> > turning off off this check for TAP interface work? Any better way to
> > achieve this result?
> >
> > Thanks and Regards,
> >
> > Raj
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
>
> View/Reply Online (#12588): https://lists.fd.io/g/vpp-dev/message/12588
> Mute This Topic: https://lists.fd.io/mt/28791570/157026
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [rajlistu...@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12593): https://lists.fd.io/g/vpp-dev/message/12593
Mute This Topic: https://lists.fd.io/mt/28791570/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Status of PPPoE Plugin
Another approach that can be used is to send all PPP/PPPoE control packets encapsulated in NSH, and Control plane can process it and return the packets to VPP which will decapsulate the NSH headers and pass the packets on to PPPoE client. Thanks and Regards, Raj On Mon, Mar 18, 2019 at 7:44 PM Raj via Lists.Fd.Io wrote: > > On Thu, Dec 20, 2018 at 6:17 AM Ni, Hongjun wrote: > > > For the issue you mentioned, Alp Arslan in the To list will submit a patch > > to fix it. > > I can take a stab at this and prepare a patch to get PPPoE working > again in VPP. I understand that this error happens because VPP will > only allow packets whose destination MAC is the same as the interface > MAC or bcast/mcast MAC. How should I go about fixing this? Would > turning off off this check for TAP interface work? Any better way to > achieve this result? > > Thanks and Regards, > > Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12588): https://lists.fd.io/g/vpp-dev/message/12588 Mute This Topic: https://lists.fd.io/mt/28791570/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Enabling events
On Mon, Mar 18, 2019 at 7:25 PM Neale Ranns (nranns) wrote: > If you're looking for VPP to send your agent/app a notification across the > binary API when an ARP and/or ND entry > is created then your app needs to call; want_ip4_arp_events and/or > want_ip6_nd_events respectively. Thanks! raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12582): https://lists.fd.io/g/vpp-dev/message/12582 Mute This Topic: https://lists.fd.io/mt/30439598/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Crash while configuring ABF
Will do that and report back.
Thanks and Regards,
Raj
On Mon, Mar 18, 2019 at 7:45 PM Neale Ranns (nranns)
wrote:
>
>
> Hi Raj,
>
>
>
> ABF, which is a feature that runs in the L3 path, has not (to my knowledge
> anyway) been tested with MACIP ACLs – this ACL type is usually applied to
> L2 traffic. Try an L3 ACL instead (i.e. use acl_add_replace to create the
> ACL, not macip_acl_rule).
>
>
>
> Regards,
>
> neale
>
>
>
>
>
> *De : * au nom de Raj
> *Date : *lundi 18 mars 2019 à 15:08
> *À : *vpp-dev
> *Objet : *Re: [vpp-dev] Crash while configuring ABF
>
>
>
> Hi,
>
>
>
> Tested again in 18.10, same result.
>
>
>
> Configuration:
>
>
>
> sh version
> vpp v18.10-22~g13f5dcf91 built by raj on vpp-dev-01 at Mon Mar 18 18:38:14
> IST 2019
> DBGvpp#
> DBGvpp#
> DBGvpp# set int state GigabitEthernet86/0/3 up
> DBGvpp# set int ip address GigabitEthernet86/0/3 xxx.xx.223.14/29
> DBGvpp# set int ip address GigabitEthernet86/0/3 2001:470::xxx::600/64
> DBGvpp# ip route add 0.0.0.0/0 via xxx.xx.223.9 GigabitEthernet86/0/3
> DBGvpp# ip route add ::/0 via 2001:470::xxx::1 GigabitEthernet86/0/3
> DBGvpp# set int state GigabitEthernet86/0/2 up
> DBGvpp# set int ip address GigabitEthernet86/0/2 100.69.1.1/24
> DBGvpp# set int ip address GigabitEthernet86/0/2 2001:xxx::600::1/56
> DBGvpp# set int ip addr GigabitEthernet86/0/0 192.168.20.1/29
> DBGvpp# set int state GigabitEthernet86/0/0 up
> DBGvpp# sh acl-plugin macip acl
> MACIP acl_index: 0, count: 1 (true len 1) tag {} is free pool slot: 0
> ip4_table_index 5, ip6_table_index 5, l2_table_index 5
> out_ip4_table_index -1, out_ip6_table_index -1, out_l2_table_index -1
> rule 0: ipv4 action 1 ip 100.69.1.4/32 mac 00:00:00:00:00:00 mask
> 00:00:00:00:00:00
> DBGvpp#
> DBGvpp# abf policy add id 0 acl 0 via 192.168.20.2 GigabitEthernet86/0/0
> DBGvpp# abf attach ip4 policy 0 priority 1 GigabitEthernet86/0/2
>
>
>
> API
>
> PUT https://
> {{machine}}/restconf/config/ietf-access-control-list:access-lists/acl/vpp-acl:vpp-macip-acl/macip-acl-2
> body
>
> {
> "acl": [
> {
> "acl-name": "macip-acl-2",
> "acl-type": "vpp-acl:vpp-macip-acl",
> "access-list-entries": {
> "ace": [
> {
> "rule-name": "macip-rule-1",
> "matches": {
> "vpp-macip-ace-nodes": {
> "source-ipv4-network": "100.69.1.4/32",
> "source-mac-address": "00:00:00:00:00:00",
> "source-mac-address-mask": "00:00:00:00:00:00"
> }
> },
> "actions": {
> "permit": [null]
> }
> }
>
> ]
> }
> }
> ]
> }
>
>
>
> GDB output:
>
>
>
> Thread 1 "vpp_main" received signal SIGABRT, Aborted.
> __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
> 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
> (gdb)
> (gdb) bt
> #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
> #1 0x75b40801 in __GI_abort () at abort.c:79
> #2 0xce54 in os_panic () at
> /home/raj/vpp-build/src/vpp/vnet/main.c:325
> #3 0x75f2dadf in debugger () at
> /home/raj/vpp-build/src/vppinfra/error.c:84
> #4 0x75f2df1a in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x7fffb4ecce50 "%s:%d (%s) assertion `%s' fails") at
> /home/raj/vpp-build/src/vppinfra/error.c:143
> #5 0x7fffb4ec690a in hash_multi_acl_match_5tuple
> (p_acl_main=0x7fffb4eb7a00 , lc_index=0,
> pkt_5tuple=0x7fffb55ffb50, is_ip6=0, action=0x7fffb55ffad7 "",
> acl_pos_p=0x7fffb55ffae0,
> acl_match_p=0x7fffb55ffadc, rule_match_p=0x7fffb55ffae4,
> trace_bitmap=0x7fffb55ffae8) at
> /home/raj/vpp-build/src/plugins/acl/public_inlines.h:651
> #6 0x7fffb4ec6b49 in acl_plugin_match_5tuple_inline
> (p_acl_main=0x7fffb4eb7a00 , lc_index=0,
> pkt_5tuple=0x7fffb55ffb50, is_ip6=0, r_action=0x7fffb55ffad7 "",
> r_acl_pos_p=0x7fffb55ffae0,
> r_acl_match_p=0x7fffb55ffadc, r_rule_match_p=0x7fffb55ffae4,
> trace_bitmap=0x7fffb55ffae8) at
> /home/raj/vpp-build/src/plugins/acl/public_inlines.h:690
> #7 0x7fffb4ec93fe in abf_input_inline (vm=0x7694b240
> , node=0x7fffb537fb80, frame=0x7fffb5ec1640,
> fproto=FIB_PROTOCOL_IP4)
> at /home/raj/vpp
Re: [vpp-dev] Status of PPPoE Plugin
On Thu, Dec 20, 2018 at 6:17 AM Ni, Hongjun wrote: > For the issue you mentioned, Alp Arslan in the To list will submit a patch to > fix it. I can take a stab at this and prepare a patch to get PPPoE working again in VPP. I understand that this error happens because VPP will only allow packets whose destination MAC is the same as the interface MAC or bcast/mcast MAC. How should I go about fixing this? Would turning off off this check for TAP interface work? Any better way to achieve this result? Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12579): https://lists.fd.io/g/vpp-dev/message/12579 Mute This Topic: https://lists.fd.io/mt/28791570/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Crash while configuring ABF
Hi,
Tested again in 18.10, same result.
Configuration:
sh version
vpp v18.10-22~g13f5dcf91 built by raj on vpp-dev-01 at Mon Mar 18 18:38:14
IST 2019
DBGvpp#
DBGvpp#
DBGvpp# set int state GigabitEthernet86/0/3 up
DBGvpp# set int ip address GigabitEthernet86/0/3 xxx.xx.223.14/29
DBGvpp# set int ip address GigabitEthernet86/0/3 2001:470::xxx::600/64
DBGvpp# ip route add 0.0.0.0/0 via xxx.xx.223.9 GigabitEthernet86/0/3
DBGvpp# ip route add ::/0 via 2001:470::xxx::1 GigabitEthernet86/0/3
DBGvpp# set int state GigabitEthernet86/0/2 up
DBGvpp# set int ip address GigabitEthernet86/0/2 100.69.1.1/24
DBGvpp# set int ip address GigabitEthernet86/0/2 2001:xxx::600::1/56
DBGvpp# set int ip addr GigabitEthernet86/0/0 192.168.20.1/29
DBGvpp# set int state GigabitEthernet86/0/0 up
DBGvpp# sh acl-plugin macip acl
MACIP acl_index: 0, count: 1 (true len 1) tag {} is free pool slot: 0
ip4_table_index 5, ip6_table_index 5, l2_table_index 5
out_ip4_table_index -1, out_ip6_table_index -1, out_l2_table_index -1
rule 0: ipv4 action 1 ip 100.69.1.4/32 mac 00:00:00:00:00:00 mask
00:00:00:00:00:00
DBGvpp#
DBGvpp# abf policy add id 0 acl 0 via 192.168.20.2 GigabitEthernet86/0/0
DBGvpp# abf attach ip4 policy 0 priority 1 GigabitEthernet86/0/2
API
PUT https://
{{machine}}/restconf/config/ietf-access-control-list:access-lists/acl/vpp-acl:vpp-macip-acl/macip-acl-2
body
{
"acl": [
{
"acl-name": "macip-acl-2",
"acl-type": "vpp-acl:vpp-macip-acl",
"access-list-entries": {
"ace": [
{
"rule-name": "macip-rule-1",
"matches": {
"vpp-macip-ace-nodes": {
"source-ipv4-network": "100.69.1.4/32",
"source-mac-address": "00:00:00:00:00:00",
"source-mac-address-mask": "00:00:00:00:00:00"
}
},
"actions": {
"permit": [null]
}
}
]
}
}
]
}
GDB output:
Thread 1 "vpp_main" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb)
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x75b40801 in __GI_abort () at abort.c:79
#2 0xce54 in os_panic () at
/home/raj/vpp-build/src/vpp/vnet/main.c:325
#3 0x75f2dadf in debugger () at
/home/raj/vpp-build/src/vppinfra/error.c:84
#4 0x75f2df1a in _clib_error (how_to_die=2, function_name=0x0,
line_number=0, fmt=0x7fffb4ecce50 "%s:%d (%s) assertion `%s' fails") at
/home/raj/vpp-build/src/vppinfra/error.c:143
#5 0x7fffb4ec690a in hash_multi_acl_match_5tuple
(p_acl_main=0x7fffb4eb7a00 , lc_index=0,
pkt_5tuple=0x7fffb55ffb50, is_ip6=0, action=0x7fffb55ffad7 "",
acl_pos_p=0x7fffb55ffae0,
acl_match_p=0x7fffb55ffadc, rule_match_p=0x7fffb55ffae4,
trace_bitmap=0x7fffb55ffae8) at
/home/raj/vpp-build/src/plugins/acl/public_inlines.h:651
#6 0x7fffb4ec6b49 in acl_plugin_match_5tuple_inline
(p_acl_main=0x7fffb4eb7a00 , lc_index=0,
pkt_5tuple=0x7fffb55ffb50, is_ip6=0, r_action=0x7fffb55ffad7 "",
r_acl_pos_p=0x7fffb55ffae0,
r_acl_match_p=0x7fffb55ffadc, r_rule_match_p=0x7fffb55ffae4,
trace_bitmap=0x7fffb55ffae8) at
/home/raj/vpp-build/src/plugins/acl/public_inlines.h:690
#7 0x7fffb4ec93fe in abf_input_inline (vm=0x7694b240
, node=0x7fffb537fb80, frame=0x7fffb5ec1640,
fproto=FIB_PROTOCOL_IP4)
at /home/raj/vpp-build/src/plugins/abf/abf_itf_attach.c:569
#8 0x7fffb4ec9662 in abf_input_ip4 (vm=0x7694b240
, node=0x7fffb537fb80, frame=0x7fffb5ec1640) at
/home/raj/vpp-build/src/plugins/abf/abf_itf_attach.c:625
#9 0x766c6ad4 in dispatch_node (vm=0x7694b240
, node=0x7fffb537fb80, type=VLIB_NODE_TYPE_INTERNAL,
dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fffb5ec1640,
last_time_stamp=1155022497203244) at
/home/raj/vpp-build/src/vlib/main.c:989
#10 0x766c708d in dispatch_pending_node (vm=0x7694b240
, pending_frame_index=1,
last_time_stamp=1155022497203244) at
/home/raj/vpp-build/src/vlib/main.c:1139
#11 0x766c8c8b in vlib_main_or_worker_loop (vm=0x7694b240
, is_main=1) at /home/raj/vpp-build/src/vlib/main.c:1555
#12 0x766c9464 in vlib_main_loop (vm=0x7694b240
) at /home/raj/vpp-build/src/vlib/main.c:1629
#13 0x766ca037 in vlib_main (vm=0x7694b240 ,
input=0x7fffb55fffb0) at /home/raj/vpp-build/src/vlib/main.c:1820
#14 0x7672148a in thread0 (arg=140737330328128) at
/home/raj/vpp-build/src/vlib/unix/main.c:607
#15 0x75f5085c in clib_calljmp () from
/home/raj/vpp-build/build-root/install-vpp_debug-nat
Re: [vpp-dev] Crash while configuring ABF
Thanks Andrew for looking into it. In the crash I sent, I missed adding ACL. But I was working with 18.10 previously and there I am sure I added ACL, but there was still some thing amiss. I am now doing the testing once more in 18.10 and will get back with result. Thanks and Regards, Raj On Mon, Mar 18, 2019 at 3:41 PM Andrew Yourtchenko wrote: > > Raj, > > yeah so looking further at it - the abf plugin does not handle the > error code correctly for the case when the ACL does not exist, and > rather than erroring out, just keeps going. Hence the assert later on > when it tries to perform the lookup in the context that has not been > properly setup.. > > --a > > On 3/18/19, Raj wrote: > > Hello all, > > > > I am testing ABF functionality and was able to crash VPP. > > > > I was using v19.04-rc0~458-g53ba544d7. Configured VPP using following > > commands: > > > > set int state GigabitEthernet86/0/3 up > > set int ip address GigabitEthernet86/0/3 xxx.xx.223.14/29 > > set int ip address GigabitEthernet86/0/3 2001:470::xxx::600/64 > > > > ip route add 0.0.0.0/0 via xxx.xx.223.9 GigabitEthernet86/0/3 > > ip route add ::/0 via 2001:470::xxx::1 GigabitEthernet86/0/3 > > > > set int state GigabitEthernet86/0/2 up > > set int ip address GigabitEthernet86/0/2 100.69.1.1/24 > > set int ip address GigabitEthernet86/0/2 2001:470::yyy::1/56 > > > > set int ip addr GigabitEthernet86/0/0 192.168.20.1/29 > > set int state GigabitEthernet86/0/0 up > > > > abf policy add id 0 acl 0 via 192.168.20.2 GigabitEthernet86/0/0 > > abf attach ip4 policy 0 priority 1 GigabitEthernet86/0/2 > > > > Running VPP under gdb, the back trace is: > > > > Thread 1 "vpp_main" received signal SIGABRT, Aborted. > > __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 > > 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. > > (gdb) > > (gdb) > > (gdb) bt > > #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 > > #1 0x75912801 in __GI_abort () at abort.c:79 > > #2 0xc4c8 in os_panic () at > > /home/raj/vpp/src/vpp/vnet/main.c:335 > > #3 0x75d0125f in debugger () at > > /home/raj/vpp/src/vppinfra/error.c:84 > > #4 0x75d0169a in _clib_error (how_to_die=2, > > function_name=0x0, line_number=0, fmt=0x7fffb4860d40 "%s:%d (%s) > > assertion `%s' fails") at /home/raj/vpp/src/vppinfra/error.c:143 > > #5 0x7fffb485a720 in hash_multi_acl_match_5tuple > > (p_acl_main=0x7fffb48513e0 , lc_index=0, > > pkt_5tuple=0x7fffb6bffac0, is_ip6=0, action=0x7fffb6bffa47 "", > > acl_pos_p=0x7fffb6bffa50, acl_match_p=0x7fffb6bffa4c, > > rule_match_p=0x7fffb6bffa54, trace_bitmap=0x7fffb6bffa58) at > > /home/raj/vpp/src/plugins/acl/public_inlines.h:636 > > #6 0x7fffb485a95f in acl_plugin_match_5tuple_inline > > (p_acl_main=0x7fffb48513e0 , lc_index=0, > > pkt_5tuple=0x7fffb6bffac0, is_ip6=0, r_action=0x7fffb6bffa47 "", > > r_acl_pos_p=0x7fffb6bffa50, r_acl_match_p=0x7fffb6bffa4c, > > r_rule_match_p=0x7fffb6bffa54, trace_bitmap=0x7fffb6bffa58) at > > /home/raj/vpp/src/plugins/acl/public_inlines.h:675 > > #7 0x7fffb485d214 in abf_input_inline (vm=0x76512600 > > , node=0x7fffb5f36980, frame=0x7fffb6dfc400, > > fproto=FIB_PROTOCOL_IP4) > > at /home/raj/vpp/src/plugins/abf/abf_itf_attach.c:569 > > #8 0x7fffb485d478 in abf_input_ip4 (vm=0x76512600 > > , node=0x7fffb5f36980, frame=0x7fffb6dfc400) at > > /home/raj/vpp/src/plugins/abf/abf_itf_attach.c:625 > > #9 0x76280614 in dispatch_node (vm=0x76512600 > > , node=0x7fffb5f36980, type=VLIB_NODE_TYPE_INTERNAL, > > dispatch_state=VLIB_NODE_STATE_POLLING, > > frame=0x7fffb6dfc400, last_time_stamp=1082092493908744) at > > /home/raj/vpp/src/vlib/main.c:1209 > > #10 0x76280dd5 in dispatch_pending_node (vm=0x76512600 > > , pending_frame_index=2, > > last_time_stamp=1082092493908744) at > > /home/raj/vpp/src/vlib/main.c:1376 > > #11 0x76282b01 in vlib_main_or_worker_loop (vm=0x76512600 > > , is_main=1) at /home/raj/vpp/src/vlib/main.c:1820 > > #12 0x7628337c in vlib_main_loop (vm=0x76512600 > > ) at /home/raj/vpp/src/vlib/main.c:1922 > > #13 0x76284109 in vlib_main (vm=0x76512600 > > , input=0x7fffb6bfffb0) at > > /home/raj/vpp/src/vlib/main.c:2111 > > #14 0x762e13c0 in thread0 (arg=140737325901312) at > > /home/raj/vpp/src/vlib/unix/main.c:612 > > #15 0x75d24544
[vpp-dev] Crash while configuring ABF
Hello all, I am testing ABF functionality and was able to crash VPP. I was using v19.04-rc0~458-g53ba544d7. Configured VPP using following commands: set int state GigabitEthernet86/0/3 up set int ip address GigabitEthernet86/0/3 xxx.xx.223.14/29 set int ip address GigabitEthernet86/0/3 2001:470::xxx::600/64 ip route add 0.0.0.0/0 via xxx.xx.223.9 GigabitEthernet86/0/3 ip route add ::/0 via 2001:470::xxx::1 GigabitEthernet86/0/3 set int state GigabitEthernet86/0/2 up set int ip address GigabitEthernet86/0/2 100.69.1.1/24 set int ip address GigabitEthernet86/0/2 2001:470::yyy::1/56 set int ip addr GigabitEthernet86/0/0 192.168.20.1/29 set int state GigabitEthernet86/0/0 up abf policy add id 0 acl 0 via 192.168.20.2 GigabitEthernet86/0/0 abf attach ip4 policy 0 priority 1 GigabitEthernet86/0/2 Running VPP under gdb, the back trace is: Thread 1 "vpp_main" received signal SIGABRT, Aborted. __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) (gdb) (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x75912801 in __GI_abort () at abort.c:79 #2 0xc4c8 in os_panic () at /home/raj/vpp/src/vpp/vnet/main.c:335 #3 0x75d0125f in debugger () at /home/raj/vpp/src/vppinfra/error.c:84 #4 0x75d0169a in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7fffb4860d40 "%s:%d (%s) assertion `%s' fails") at /home/raj/vpp/src/vppinfra/error.c:143 #5 0x7fffb485a720 in hash_multi_acl_match_5tuple (p_acl_main=0x7fffb48513e0 , lc_index=0, pkt_5tuple=0x7fffb6bffac0, is_ip6=0, action=0x7fffb6bffa47 "", acl_pos_p=0x7fffb6bffa50, acl_match_p=0x7fffb6bffa4c, rule_match_p=0x7fffb6bffa54, trace_bitmap=0x7fffb6bffa58) at /home/raj/vpp/src/plugins/acl/public_inlines.h:636 #6 0x7fffb485a95f in acl_plugin_match_5tuple_inline (p_acl_main=0x7fffb48513e0 , lc_index=0, pkt_5tuple=0x7fffb6bffac0, is_ip6=0, r_action=0x7fffb6bffa47 "", r_acl_pos_p=0x7fffb6bffa50, r_acl_match_p=0x7fffb6bffa4c, r_rule_match_p=0x7fffb6bffa54, trace_bitmap=0x7fffb6bffa58) at /home/raj/vpp/src/plugins/acl/public_inlines.h:675 #7 0x7fffb485d214 in abf_input_inline (vm=0x76512600 , node=0x7fffb5f36980, frame=0x7fffb6dfc400, fproto=FIB_PROTOCOL_IP4) at /home/raj/vpp/src/plugins/abf/abf_itf_attach.c:569 #8 0x7fffb485d478 in abf_input_ip4 (vm=0x76512600 , node=0x7fffb5f36980, frame=0x7fffb6dfc400) at /home/raj/vpp/src/plugins/abf/abf_itf_attach.c:625 #9 0x76280614 in dispatch_node (vm=0x76512600 , node=0x7fffb5f36980, type=VLIB_NODE_TYPE_INTERNAL, dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fffb6dfc400, last_time_stamp=1082092493908744) at /home/raj/vpp/src/vlib/main.c:1209 #10 0x76280dd5 in dispatch_pending_node (vm=0x76512600 , pending_frame_index=2, last_time_stamp=1082092493908744) at /home/raj/vpp/src/vlib/main.c:1376 #11 0x76282b01 in vlib_main_or_worker_loop (vm=0x76512600 , is_main=1) at /home/raj/vpp/src/vlib/main.c:1820 #12 0x7628337c in vlib_main_loop (vm=0x76512600 ) at /home/raj/vpp/src/vlib/main.c:1922 #13 0x76284109 in vlib_main (vm=0x76512600 , input=0x7fffb6bfffb0) at /home/raj/vpp/src/vlib/main.c:2111 #14 0x762e13c0 in thread0 (arg=140737325901312) at /home/raj/vpp/src/vlib/unix/main.c:612 #15 0x7ffff5d24544 in clib_calljmp () from /home/raj/vpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.19.04 #16 0x7fffd170 in ?? () #17 0x762e188d in vlib_unix_main (argc=43, argv=0x5586f500) at /home/raj/vpp/src/vlib/unix/main.c:681 #18 0xbf0c in main (argc=43, argv=0x5586f500) at /home/raj/vpp/src/vpp/vnet/main.c:274 Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12567): https://lists.fd.io/g/vpp-dev/message/12567 Mute This Topic: https://lists.fd.io/mt/30471670/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Enabling events
Thanks, I will cross check and get back. Thanks and Regards, Raj On Fri, Mar 15, 2019 at 6:40 PM Dave Barach (dbarach) wrote: > > You're sure that links are up, ipv6 is enabled on the interfaces and all that > good stuff, right? > > If "show ip6 neighbor" shows nothing, you may be going nowhere near the ip6 > neighbor code. > > Try setting a gdb breakpoint in ip6_neighbor_syslog(...). If you don't get > there, you're not going to see much in the event log... > > D. > > -Original Message----- > From: vpp-dev@lists.fd.io On Behalf Of Raj > Sent: Friday, March 15, 2019 8:33 AM > To: vpp-dev > Subject: Re: [vpp-dev] Enabling events > > On Fri, Mar 15, 2019 at 5:44 PM Dave Barach (dbarach) > wrote: > > > > We don't log many (if any) events by default. > > Thanks for the reply Dave. > > I am specifically looking for events related to ARP and v6 ND. For example > the commit at > https://git.flirble.org/fdio/vpp/commit/1edfba9a6394128ee5fad2b413e9e0a05972ef48 > talks about these events.Do these events gets generated automatically? > > Thanks and Regards, > > Raj > > > -Original Message- > > From: vpp-dev@lists.fd.io On Behalf Of Raj > > Sent: Friday, March 15, 2019 8:02 AM > > To: vpp-dev > > Subject: [vpp-dev] Enabling events > > > > Hi all, > > > > I have started VPP, but the number of events always is zero. > > > > vpp# sh event-logger > > 0 of 1024 events in buffer, logger running vpp# event-logger save > > myevent Saving 0 of 1024 events to /tmp/myevent > > > > So I am wondering if I have to specifically enable event logging or are > > events logged automatically. > > > > Also can I view events in real time, rather than saving in a file and > > viewing it offline? > > > > Thanks and Regards, > > > > Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12561): https://lists.fd.io/g/vpp-dev/message/12561 Mute This Topic: https://lists.fd.io/mt/30439598/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Enabling events
On Fri, Mar 15, 2019 at 5:44 PM Dave Barach (dbarach) wrote: > > We don't log many (if any) events by default. Thanks for the reply Dave. I am specifically looking for events related to ARP and v6 ND. For example the commit at https://git.flirble.org/fdio/vpp/commit/1edfba9a6394128ee5fad2b413e9e0a05972ef48 talks about these events.Do these events gets generated automatically? Thanks and Regards, Raj > -Original Message- > From: vpp-dev@lists.fd.io On Behalf Of Raj > Sent: Friday, March 15, 2019 8:02 AM > To: vpp-dev > Subject: [vpp-dev] Enabling events > > Hi all, > > I have started VPP, but the number of events always is zero. > > vpp# sh event-logger > 0 of 1024 events in buffer, logger running vpp# event-logger save myevent > Saving 0 of 1024 events to /tmp/myevent > > So I am wondering if I have to specifically enable event logging or are > events logged automatically. > > Also can I view events in real time, rather than saving in a file and viewing > it offline? > > Thanks and Regards, > > Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12559): https://lists.fd.io/g/vpp-dev/message/12559 Mute This Topic: https://lists.fd.io/mt/30439598/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Enabling events
Hi all, I have started VPP, but the number of events always is zero. vpp# sh event-logger 0 of 1024 events in buffer, logger running vpp# event-logger save myevent Saving 0 of 1024 events to /tmp/myevent So I am wondering if I have to specifically enable event logging or are events logged automatically. Also can I view events in real time, rather than saving in a file and viewing it offline? Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12557): https://lists.fd.io/g/vpp-dev/message/12557 Mute This Topic: https://lists.fd.io/mt/30439598/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] recipe for target 'g2-find-source' failed
On Thu, Mar 14, 2019 at 7:19 PM Dave Barach (dbarach) wrote: > > This is a doc bug. Where exactly is the broken documentation? https://fdio-vpp.readthedocs.io/en/latest/gettingstarted/developers/eventviewer.html#g2-graphical-event-viewer > g2 has at least one worthwhile new feature in master/latest - automated > anomaly detection - and the master/latest version will cheerfully work with > event-logs from an 18.10 vpp image. So, I'd be tempted to pull master/latest > and build it. Will check out master and try. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12543): https://lists.fd.io/g/vpp-dev/message/12543 Mute This Topic: https://lists.fd.io/mt/30427633/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] recipe for target 'g2-find-source' failed
Hello all, According to docs I can compile g2 using make g2-install. But in 18.10 sources I get this [raj@vpp-dev-01 build-root (stable-1810)]$ make g2-install Arch for platform 'native' is native Finding source for g2 Package g2 not found with path /home/raj/vpp/build-root/.. Makefile:773: recipe for target 'g2-find-source' failed make: *** [g2-find-source] Error 1 Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12538): https://lists.fd.io/g/vpp-dev/message/12538 Mute This Topic: https://lists.fd.io/mt/30427633/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP stats
Hi Ole,
I tried various combination of stats and statseg in config file but no
joy. For example this conf gives error in 18.10. though by manual it
seems ok.
stats { socket-name /run/vpp/stats.sock }
statseg { size 128M }
It would be great if you can point to the documentation for 18.10 or a
sample conf for 18.10
Thanks and Regards,
Raj
On Wed, Mar 6, 2019 at 8:31 PM Ole Troan wrote:
>
> Hi Raj,
>
> statseg { … }
>
> There is a C API in vpp-api/client/stat_client.h you can use.
> Or a higher level Go, Python or C++ binding too.
>
> Cheers,
> Ole
>
> > On 6 Mar 2019, at 14:27, Raj wrote:
> >
> > Hi all,
> >
> > I am trying to get the stats from VPP SHM. If I understand correctly,
> > I need to open VPP stats Unix domain socket and read from the
> > corresponding memory mapped segment.
> >
> > My problem is that the stats socket file is missing. I added the
> > following line in startup.conf but of no avail.
> >
> > stats { socket-name /run/vpp/stats.sock }
> >
> > Should I have to do any thing else to get the sock file and possibly
> > enable SHM logging?
> >
> > Thanks and Regards,
> >
> > Raj
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> >
> > View/Reply Online (#12443): https://lists.fd.io/g/vpp-dev/message/12443
> > Mute This Topic: https://lists.fd.io/mt/30284996/675193
> > Group Owner: vpp-dev+ow...@lists.fd.io
> > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [otr...@employees.org]
> > -=-=-=-=-=-=-=-=-=-=-=-
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12482): https://lists.fd.io/g/vpp-dev/message/12482
Mute This Topic: https://lists.fd.io/mt/30284996/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP core dump
On Wed, Mar 6, 2019 at 10:42 PM Andrew Yourtchenko wrote: > Unfortunately I can’t give you a step by step way to isolate that trigger > because it can be most everything potentially. Can you see if you can reproduce the crash by running the attached script ? In my environment, this script can reliably produce the crash. Thanks and Regards, Raj vppset.sh Description: application/shellscript -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12481): https://lists.fd.io/g/vpp-dev/message/12481 Mute This Topic: https://lists.fd.io/mt/30283387/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP stats
Hi all,
I am trying to get the stats from VPP SHM. If I understand correctly,
I need to open VPP stats Unix domain socket and read from the
corresponding memory mapped segment.
My problem is that the stats socket file is missing. I added the
following line in startup.conf but of no avail.
stats { socket-name /run/vpp/stats.sock }
Should I have to do any thing else to get the sock file and possibly
enable SHM logging?
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12443): https://lists.fd.io/g/vpp-dev/message/12443
Mute This Topic: https://lists.fd.io/mt/30284996/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP core dump
On Wed, Mar 6, 2019 at 2:52 PM Andrew Yourtchenko wrote: > > Sounds like a memory corruption. > I am out of office for another week, so in the meantime if you might collect > few postmortem dumps with reproductions, I will look at it when I return. Sure, I will get some dumps with problem reproduced. In the mean time, if you like me to check any thing specific I can do it. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12442): https://lists.fd.io/g/vpp-dev/message/12442 Mute This Topic: https://lists.fd.io/mt/30283387/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP core dump
Hello all,
I am getting a core dump when adding MACIP ACL using API (using
honeycomb). My observation is that I can reproduce this core dump
reliably if I add about 300 MACIP ACL. I am on v18.10-27~ga0005702c
I did some debugging and my observations is:
In the function:
void
vl_msg_api_handler_with_vm_node (api_main_t * am,
void *the_msg, vlib_main_t * vm,
vlib_node_runtime_t * node)
{
...
...
/*
* Special-case, so we can e.g. bounce messages off the vnet
* main thread without copying them...
*/
if (!(am->message_bounce[id]))
vl_msg_api_free (the_msg);
...
}
Control is reaching the special-case, and core dump is happening in
vl_msg_api_free function.
Code flow is:
void_mem_api_handle_msg_i()
->vl_msg_api_free (the_msg);
->clib_mem_free (rv);
->mspace_put (heap, p);
->mspace_free (msp, object_header);
->ok_magic(fm)
->return (m->magic == mparams.magic); /* here it dumps */
Following is my gdb session transcript:
(gdb) bt
#0 0x75fd9f98 in ok_magic (m=0x13131313cdbec9ad) at
/home/raj/vpp/src/vppinfra/dlmalloc.c:1618
#1 0x75fe271a in mspace_free (msp=0x130044010,
mem=0x1301c4ca0) at /home/raj/vpp/src/vppinfra/dlmalloc.c:4456
#2 0x75fe1b9d in mspace_put (msp=0x130044010,
p_arg=0x1301c4ca4) at /home/raj/vpp/src/vppinfra/dlmalloc.c:4291
#3 0x77b916a4 in clib_mem_free (p=0x1301c4ca4) at
/home/raj/vpp/src/vppinfra/mem.h:215
#4 0x77b922f6 in vl_msg_api_free (a=0x1301c4cb4) at
/home/raj/vpp/src/vlibmemory/memory_shared.c:291
#5 0x77bc325c in vl_msg_api_handler_with_vm_node
(am=0x77dd3d20 , the_msg=0x1301c4cb4, vm=0x76952240
, vm=0x76952240 , node=0x7fffb
at /home/raj/vpp/src/vlibmemory/memory_api.c:692
#7 0x77b8ff23 in vl_mem_api_handle_msg_main
(vm=0x76952240 , node=0x7fffb5264000) at
/home/raj/vpp/
#8 0x77baded4 in vl_api_clnt_process (vm=0x76952240
, node=0x7fffb5264000, f=0x0) at /home/raj/vpp/
#9 0x766ce32a in vlib_process_bootstrap (_a=140736236354592)
at /home/raj/vpp/src/vlib/main.c:1232
#10 0x75f5784c in clib_calljmp () from
/home/raj/vpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.18.10
#11 0x7fffb55ffbf0 in ?? ()
#12 0x766ce455 in vlib_process_startup (vm=0xd52f22e80133b900,
p=0x, f=0x7fffb5264000) at /home/raj/vpp/sr
#13 0x0086 in ?? ()
#14 0x76952350 in vlib_global_main () from
/home/raj/vpp/build-root/install-vpp_debug-native/vpp/lib/libvlib.so.18.10
#15 0x0003612097f3543e in ?? ()
#16 0x7fffb5264000 in ?? ()
n ?? ()
#18 0x7fffb5ccf56c in ?? ()
#19 0x0011 in ?? ()
#20 0x7fffb5ccf668 in ?? ()
#21 0x7fffb5264000 in ?? ()
#22 0x7fffb79d8294 in ?? ()
#23 0x in ?? ()
(gdb) f 2
#2 0x75fe1b9d in mspace_put (msp=0x130044010,
p_arg=0x1301c4ca4) at /home/raj/vpp/src/vppinfra/dlmalloc.c:4291
4291 mspace_free (msp, object_header);
(gdb) p msp
$1 = (mspace) 0x130044010
(gdb) p *msp
Attempt to dereference a generic pointer.
(gdb) p *(mstate)msp
$2 = {smallmap = 4096, treemap = 32768, dvsize = 0, topsize =
15069712, least_addr = 0x130044000 "", dv = 0x0, top = 0x1301e4da0,
tri
release_checks = 4086, magic = 3735935678, smallbins = {0x0, 0x0,
0x130044058, 0x130044058, 0x130044068, 0x130044068, 0x130044078,
0x130044088, 0x130044098, 0x130044098, 0x1300440a8, 0x1300440a8,
0x1300440b8, 0x1300440b8, 0x1300440c8, 0x1300440c8, 0x13005c5b0,
0x1300440e8, 0x1300440f8, 0x1300440f8, 0x130044108, 0x130044108,
0x1300652c0, 0x1300652c0, 0x130044128, 0x130044128, 0x130044138,
0x130044148, 0x130044158, 0x130044158, 0x130044168, 0x130044168,
0x130044178, 0x130044178, 0x130044188, 0x130044188, 0x1301c4ce0,
0x1300441a8, 0x1300441b8, 0x1300441b8, 0x1300441c8, 0x1300441c8,
0x1300441d8, 0x1300441d8, 0x1300441e8, 0x1300441e8, 0x1300441f8,
0x130044208, 0x130044218, 0x130044218, 0x130044228, 0x130044228,
0x130044238, 0x130044238, 0x130044248, 0x130044248}, treebins =
0x1301c5cc0, 0x0 }, footprint = 16777216,
max_footprint = 16777216, footprint_limit = 0, mflags = 15, mutex = 0
size = 16777216, next = 0x0, sflags = 8}, extp = 0x0, exts = 0}
(gdb) f 5
#5 0x77bc325c in vl_msg_api_handler_with_vm_node
(am=0x77dd3d20 , the_msg=0x1301c4cb4, vm=0x76952240
...}
(gdb) f 0
#0 0x75fd9f98 in ok_magic (m=0x13131313cdbec9ad) at
/home/raj/vpp/src/vppinfra/dlmalloc.c:1618
1618return (m->magic == mparams.magic);
(gdb) p m->magic
Cannot access memory at address 0x13131313cdbec9ed
(gdb) f 1
#1 0x75fe271a in mspace_free (msp=0x130044010,
mem=0x1301c4ca0) at /home/raj/vpp/src/vppinfra/dlmalloc.c:4456
4456if (!ok_magic(fm)) {
(gdb) p *(mstate)msp
$24 = {smallmap = 4096, treemap = 32768, dvsize = 0, topsize =
15069712, least_addr = 0x1
[vpp-dev] Inserting a node into ICMP path
Hello all,
To get a better understanding of DPO and feature arc, I am attempting
to insert a node "ip4-my-node" before ip4-load-balance. So the ICMP
flow would be like
ip4-icmp-input->ip4-icmp-echo-request->my_node->ip4-load-balance->ip4-rewrite->..
For achieving this, I have created a DPO instance similar to
load-balance and in the ip4-icmp-echo-request node added the
following:
.next_nodes = {
-[0] = "ip4-load-balance",
+[0] = "ip4-my-node",
But after changing this, the flow is still happening from
ip4-icmp-echo-request->ip4-load-balance, and packets are not hitting
ip4-my-node.
Do I have to make any other change apart from what I have already done
insert the ip4-my-node into the path?
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12353): https://lists.fd.io/g/vpp-dev/message/12353
Mute This Topic: https://lists.fd.io/mt/30151551/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Configuring NAT and Policing together
On Wed, Feb 6, 2019 at 11:05 AM Matus Fabian -X (matfabia - PANTHEON
TECHNOLOGIES at Cisco) wrote:
> You need to update next_nodes in VLIB_REGISTER_NODE (snat_out2in_node) too
This was already one. Did some debugging using GDB and figured out
that the issue could be in the node ip4_policer_classify node.
Earlier 'runs_before' was updated with 'nat44-in2out' in
'ip4-policer-classify' node. With this I was hitting ASSERT
(next_index < n->n_next_nodes) for the 'ip4-policer-classify' node.
(gdb) print *(nm->nodes_by_type[VLIB_NODE_TYPE_INTERNAL])
$12 = {cacheline0 = 0x7fffb533da40 "\210\317m\366\377\177",
function = 0x766dcf88 , errors = 0x7fffb51d7b2c,
clocks_since_last_overflow = 0, max_clock = 0, max_clock_n = 0,
calls_since_last_overflow = 0, vectors_since_last_overflow = 0,
next_frame_index = 6,
node_index = 0, input_main_loops_per_call = 0,
main_loop_count_last_dispatch = 0,
main_loop_vector_stats = {0, 0}, flags = 0, state = 0, n_next_nodes = 0,
cached_next_index = 0, thread_index = 0, runtime_data = 0x7fffb533da86 ""}
(gdb) print *n
$13 = {cacheline0 = 0x7fffb534c5c0 "W\270\314\366\377\177",
function = 0x76ccb857 , errors = 0x7fffb538159c,
clocks_since_last_overflow = 89712, max_clock = 22476, max_clock_n = 1,
calls_since_last_overflow = 11, vectors_since_last_overflow = 11,
next_frame_index = 1945, node_index = 539, input_main_loops_per_call = 0,
main_loop_count_last_dispatch = 335304799, main_loop_vector_stats =
{1, 0}, flags = 0,
state = 0, n_next_nodes = 4, cached_next_index = 2, thread_index = 0,
runtime_data = 0x7fffb534c606 ""}
But when I added the feature 'nat44-out2in' also in the .runs_before
of 'ip4-policer-classify', the issue was fixed. No Assertions and I
could see the functionality is working fine.
diff --git a/src/vnet/ip/ip4_forward.c b/src/vnet/ip/ip4_forward.c
index 9dac828a..e9a99006 100644
--- a/src/vnet/ip/ip4_forward.c
+++ b/src/vnet/ip/ip4_forward.c
@@ -753,7 +753,7 @@ VNET_FEATURE_INIT (ip4_policer_classify, static) =
{
.arc_name = "ip4-unicast",
.node_name = "ip4-policer-classify",
- .runs_before = VNET_FEATURES ("ipsec-input-ip4"),
+ .runs_before = VNET_FEATURES
("ipsec-input-ip4","nat44-in2out","nat44-out2in"),
};
What I would like to know is whether this is the correct fix for this
ASSERT failure, and if yes I am not sure how exactly it works.
Does adding the feature 'nat44-out2in' in the .runs_before of
'ip4-policer-classify' has a side effect of incrementing the value of
n_next_nodes? If not how adding the above patch resolve the Assert
failure?
With regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12226): https://lists.fd.io/g/vpp-dev/message/12226
Mute This Topic: https://lists.fd.io/mt/29379239/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Understanding vlib_frame_t
On Mon, Feb 4, 2019 at 9:19 PM Dave Barach via Lists.Fd.Io wrote: > "use the Force and read the Source..." Thanks :) Will do! Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12179): https://lists.fd.io/g/vpp-dev/message/12179 Mute This Topic: https://lists.fd.io/mt/29606187/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Understanding vlib_frame_t
Thanks Dave and Damjan, If I understand correctly a node can add elements to a frame and the pending frame thus formed are put into the graph dispatcher's run-queue. Checking the structure of vlib_pending_frame_t, there is a param 'next_frame_index. Does it mean that we can have a list of pending frames in a node at a time (max 256 packets in each frame). If there is a pending frame, is it processed just like a normal frame? Reading through the code, I get a feel that pending frames are not processed the way normal frames are, and there are some timer wheel involved, but could not figure out the exact mechanism. Thanks and Regards, Raj On Fri, Feb 1, 2019 at 8:21 PM Dave Barach (dbarach) wrote: > > Next frames are the physical manifestations of graph arcs. Pending frames are > graph scheduler run-queue entries. > > -Original Message- > From: vpp-dev@lists.fd.io On Behalf Of Raj > Sent: Friday, February 1, 2019 9:47 AM > To: vpp-dev > Subject: Re: [vpp-dev] Understanding vlib_frame_t > > Thanks Damjan and Dave. These small explanations will greatly help in making > sure what I understand from reading code is correct. > > Just a couple of followup questions, I hope these will help those reading the > archives later too in understanding the nuances of VPP architecture. > > vlib_get_buffer() translates the buffer index into buffer pointer. > Where exactly is the buffer memory getting allocated, when packets are coming > in via dpdk-input? Is this allocation is done by DPDK or this is done by VPP. > If I understand correctly vlib_buffer_t is pointing to this location. > > Also as I understand, a graph nodes gets a vector. This vector is a frame > which holds a set of 4 Byte buffer indices which points to the the actual > buffer, containing packets. Node process all these packets (max 256) and > transmit to the next node. While reading code in this area, after processing > this there is reference to next frame and pending frame. I could not figure > out what these are and for what they are being used. > > Thanks and Regards, > > Raj > > On Thu, Jan 31, 2019 at 9:31 PM Dave Barach (dbarach) > wrote: > > > > In vlib – vpp’s basic vector library - frames comprise a single vector, a > > set of scalar variables, and some flags. > > > > > > > > At node creation time, each graph node registration indicates the size of a > > vector element, and the total size of scalar data. Vlib_frame_t vectors > > include up to VLIB_FRAME_SIZE (256) vector elements of the indicated size. > > > > > > > > In the vpp/vnet use case, vector elements are 32-bit buffer handles, easily > > converted to vlib_buffer_t pointers. Most graph nodes do not use scalar > > data. > > > > > > > > An sample use case for per-frame scalar data: send all these packets in the > > vector to a certain ip address, where the ip address could be stored in 4 > > or 16 octets’ worth of scalar data. > > > > > > > > HTH... Dave > > > > > > > > -Original Message- > > From: vpp-dev@lists.fd.io On Behalf Of Raj > > Sent: Thursday, January 31, 2019 9:27 AM > > To: vpp-dev > > Subject: [vpp-dev] Understanding vlib_frame_t > > > > > > > > Hello all, > > > > > > > > Continuing my reading of VPP code, I have a query regarding packet handling. > > > > > > > > When a node gets a vector of packets to process, if I understand correctly, > > this would be a vlib_frame_t instance. From this vlib_frame_t instance > > arguments[0] is the offset to first packet, arguments[1] is the offset to > > second packet and so on, upto xx number of packets. Is this understanding > > correct? if yes, how much is the maximum size of the packet buffer? > > > > > > > > Also what is the significance of scalar_size/vector_size in vlib_frame_t? > > Does it mean the number of frames/size of frames? > > > > > > > > in vlib/main.c there is a statement vec_resize (nm->pending_frames, 32). > > In here, what is pending_frames and what is the significance of number 32? > > > > > > > > I know I have been asking too many questions over here and very grateful to > > all who answer. > > > > > > > > Thanks a lot! > > > > > > > > Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12157): https://lists.fd.io/g/vpp-dev/message/12157 Mute This Topic: https://lists.fd.io/mt/29606187/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Packet copy in plugins/nsim
Thanks Florin! Raj On Fri, Feb 1, 2019 at 2:07 AM Florin Coras wrote: > > Hi Raj, > > Partly. Check “show buffers” > > Florin > > > On Jan 31, 2019, at 1:04 AM, Raj wrote: > > > > Hi Florin, > > > > On Wed, Jan 30, 2019 at 10:01 PM Florin Coras > > wrote: > > > >> Because if it can work with all of its buffers cached, without flushing > >> them to main memory, it can access and therefore ‘process’ them faster. > > > > Understood. Is there any means to see how many buffers are in use > > currently, and if they are cached? > > > >> It's two copies vs more buffer memory, not bigger buffers. Flushing and > >> reading buffers to/from main memory are pretty much copy operations. > >> Nsim typically needs a lot of buffer memory, therefore instead of trashing > >> the cache and potentially impacting other vpp components, it directly > >> stores everything in memory. > > > > That's clear. > > > > Thanks! > > > > Raj > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > > > View/Reply Online (#12088): https://lists.fd.io/g/vpp-dev/message/12088 > > Mute This Topic: https://lists.fd.io/mt/29581029/675152 > > Group Owner: vpp-dev+ow...@lists.fd.io > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [fcoras.li...@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12127): https://lists.fd.io/g/vpp-dev/message/12127 Mute This Topic: https://lists.fd.io/mt/29581029/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Understanding vlib_frame_t
Thanks Damjan and Dave. These small explanations will greatly help in making sure what I understand from reading code is correct. Just a couple of followup questions, I hope these will help those reading the archives later too in understanding the nuances of VPP architecture. vlib_get_buffer() translates the buffer index into buffer pointer. Where exactly is the buffer memory getting allocated, when packets are coming in via dpdk-input? Is this allocation is done by DPDK or this is done by VPP. If I understand correctly vlib_buffer_t is pointing to this location. Also as I understand, a graph nodes gets a vector. This vector is a frame which holds a set of 4 Byte buffer indices which points to the the actual buffer, containing packets. Node process all these packets (max 256) and transmit to the next node. While reading code in this area, after processing this there is reference to next frame and pending frame. I could not figure out what these are and for what they are being used. Thanks and Regards, Raj On Thu, Jan 31, 2019 at 9:31 PM Dave Barach (dbarach) wrote: > > In vlib – vpp’s basic vector library - frames comprise a single vector, a set > of scalar variables, and some flags. > > > > At node creation time, each graph node registration indicates the size of a > vector element, and the total size of scalar data. Vlib_frame_t vectors > include up to VLIB_FRAME_SIZE (256) vector elements of the indicated size. > > > > In the vpp/vnet use case, vector elements are 32-bit buffer handles, easily > converted to vlib_buffer_t pointers. Most graph nodes do not use scalar data. > > > > An sample use case for per-frame scalar data: send all these packets in the > vector to a certain ip address, where the ip address could be stored in 4 or > 16 octets’ worth of scalar data. > > > > HTH... Dave > > > > -Original Message- > From: vpp-dev@lists.fd.io On Behalf Of Raj > Sent: Thursday, January 31, 2019 9:27 AM > To: vpp-dev > Subject: [vpp-dev] Understanding vlib_frame_t > > > > Hello all, > > > > Continuing my reading of VPP code, I have a query regarding packet handling. > > > > When a node gets a vector of packets to process, if I understand correctly, > this would be a vlib_frame_t instance. From this vlib_frame_t instance > arguments[0] is the offset to first packet, arguments[1] is the offset to > second packet and so on, upto xx number of packets. Is this understanding > correct? if yes, how much is the maximum size of the packet buffer? > > > > Also what is the significance of scalar_size/vector_size in vlib_frame_t? > Does it mean the number of frames/size of frames? > > > > in vlib/main.c there is a statement vec_resize (nm->pending_frames, 32). In > here, what is pending_frames and what is the significance of number 32? > > > > I know I have been asking too many questions over here and very grateful to > all who answer. > > > > Thanks a lot! > > > > Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12126): https://lists.fd.io/g/vpp-dev/message/12126 Mute This Topic: https://lists.fd.io/mt/29606187/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Understanding vlib_frame_t
Hello all, Continuing my reading of VPP code, I have a query regarding packet handling. When a node gets a vector of packets to process, if I understand correctly, this would be a vlib_frame_t instance. From this vlib_frame_t instance arguments[0] is the offset to first packet, arguments[1] is the offset to second packet and so on, upto xx number of packets. Is this understanding correct? if yes, how much is the maximum size of the packet buffer? Also what is the significance of scalar_size/vector_size in vlib_frame_t? Does it mean the number of frames/size of frames? in vlib/main.c there is a statement vec_resize (nm->pending_frames, 32). In here, what is pending_frames and what is the significance of number 32? I know I have been asking too many questions over here and very grateful to all who answer. Thanks a lot! Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12093): https://lists.fd.io/g/vpp-dev/message/12093 Mute This Topic: https://lists.fd.io/mt/29606187/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Packet copy in plugins/nsim
Hello Florin, Thanks a lot for your reply. On Tue, Jan 29, 2019 at 9:15 PM Florin Coras wrote: > > VPP typically works best if configured with a relatively small number of > buffers. I am trying to understand why this would be the case. > On the other hand, nsim can induce a large amount of delay, so the buffers > cannot be used to temporarily store the data. If my understanding is correct, the comparison is between two packet copy per packet and performance degradation due to bigger buffers. How would they compare? Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12065): https://lists.fd.io/g/vpp-dev/message/12065 Mute This Topic: https://lists.fd.io/mt/29581029/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Configuring NAT and Policing together
Hi,
I am checking whether policing feature is enabled or not in NAT with
the following approach
---
Did an extern declaration for policer_classify_main_t in nat/out2in.c
if (pcm->classify_table_index_by_sw_if_index[0][sw_if_index0] != ~0)
{
/* feature is enabled */
next0 = SNAT_OUT2IN_NEXT_POLICER_CLASSIFY;
vlib_node_add_next (vm, ip4_policer_classify_node->index,
next0);
}
---
I have a feeling that this is not the best/proper way, but I could not
figure out any thing better. Please suggest is there any other
optimized/proper way to check an interface feature is enabled or not?
But when the packets starts flowing, VPP is getting aborted !!
DBGvpp# Aborted
Makefile:473: recipe for target 'run' failed
make: *** [run] Error 134
=> following is the gdb trace for the same <==
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x7f7b08a7542a in __GI_abort () at abort.c:89
#2 0x55d570fd6f5e in os_exit (code=1) at
/home//vpp/src/vpp/vnet/main.c:349
#3 0x7f7b0a1e7cac in unix_signal_handler (signum=11,
si=0x7f7ac8fff670, uc=0x7f7ac8fff540)
at /home//vpp/src/vlib/unix/main.c:157
#4
#5 0x7f7ac3f2606d in svs_input_inline (vm=0x7f7b0a413240
, node=0x7f7ac95a3080,
frame=0x7f7aca271d00, fproto=FIB_PROTOCOL_IP4) at
/home//vpp/src/plugins/svs/svs.c:294
#6 0x7f7ac3f262af in svs_input_ip4 (vm=0x7f7b0a413240
, node=0x7f7ac95a3080,
frame=0x7f7aca271d00) at /home//vpp/src/plugins/svs/svs.c:339
#7 0x7f7b0a18eae4 in dispatch_node (vm=0x7f7b0a413240
, node=0x7f7ac95a3080,
type=VLIB_NODE_TYPE_INTERNAL,
dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7f7aca271d00,
last_time_stamp=12587373585573984) at /home//vpp/src/vlib/main.c:989
#8 0x7f7b0a18f09d in dispatch_pending_node (vm=0x7f7b0a413240
,
pending_frame_index=3, last_time_stamp=12587373585573984) at
/home//vpp/src/vlib/main.c:1139
#9 0x7f7b0a190c9c in vlib_main_or_worker_loop (vm=0x7f7b0a413240
, is_main=1)
at /home//vpp/src/vlib/main.c:1555
#10 0x7f7b0a191478 in vlib_main_loop (vm=0x7f7b0a413240 )
at /home//vpp/src/vlib/main.c:1629
#11 0x7f7b0a19204b in vlib_main (vm=0x7f7b0a413240
, input=0x7f7ac8b0)
at /home//vpp/src/vlib/main.c:1820
#12 0x7f7b0a1e94ba in thread0 (arg=140166429749824) at
/home//vpp/src/vlib/unix/main.c:607
#13 0x7f7b098128a4 in clib_calljmp ()
from
/home//vpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.18.10
#14 0x7ffcc1a6b2d0 in ?? ()
#15 0x7f7b0a1e996e in vlib_unix_main (argc=2, argv=0x7ffcc1a6c5c8)
at /home//vpp/src/vlib/unix/main.c:676
#16 0x55d570fd6927 in main (argc=2, argv=0x7ffcc1a6c5c8) at
/home//vpp/src/vpp/vnet/main.c:264
Thanks and Regards,
Raj
On Wed, Jan 23, 2019 at 8:05 PM Matus Fabian -X (matfabia - PANTHEON
TECHNOLOGIES at Cisco) wrote:
>
> Hi,
>
> You should go from nat44-out2in to ip4-policer-classify only if it is
> configured on given interface (check if sw_if_index0 in nat44-out2in has
> configured/enabled policer), I think this may be reason of ASSERT.
>
> Matus
>
>
> -Original Message-----
> From: vpp-dev@lists.fd.io On Behalf Of Raj
> Sent: Wednesday, January 23, 2019 3:05 PM
> To: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Configuring NAT and Policing together
>
> Hi Matus,
>
> Thanks for the code fragment it helped me get a better understanding of what
> I have to do, and have modified the code. But occasionally VPP hits an ASSERT
> at:
>
> DBGvpp# 0: /vpp/src/vlib/node_funcs.h:296
> (vlib_node_runtime_get_next_frame) assertion `next_index <
> n->n_next_nodes' fails
> Aborted
>
> The approach I had followed was to get the index of policer classify node and
> setting that as the next node of 'nat44-out2in'
> ,'nat44-out2in-reass' and 'nat44-out2in-fast'.
>
> This is the partial diff of how we got the index of ip4-policer-classify and
> setting the next node. (full diff is attached).
>
> --- a/src/plugins/nat/out2in.c
> +++ b/src/plugins/nat/out2in.c
> @@ -1110,6 +1113,15 @@ snat_out2in_node_fn (vlib_main_t * vm,
>
> proto0 = ip_proto_to_snat_proto (ip0->protocol);
>
> + ip4_policer_classify_node =
> +vlib_get_node_by_name (vm, (u8 *) "ip4-policer-classify");
> + if (ip4_policer_classify_node)
> +{
> + next0 = SNAT_OUT2IN_NEXT_POLICER_CLASSIFY;
> + vlib_node_add_next (vm, ip4_policer_classify_node->index,
> + next0);
> +}
> +
> if (PREDICT_FALSE (proto0 == ~0))
> {
>
Re: [vpp-dev] Configuring NAT and Policing together
Hi Matus,
Thanks for the code fragment it helped me get a better understanding
of what I have to do, and have modified the code. But occasionally VPP
hits an ASSERT at:
DBGvpp# 0: /vpp/src/vlib/node_funcs.h:296
(vlib_node_runtime_get_next_frame) assertion `next_index <
n->n_next_nodes' fails
Aborted
The approach I had followed was to get the index of policer classify
node and setting that as the next node of 'nat44-out2in'
,'nat44-out2in-reass' and 'nat44-out2in-fast'.
This is the partial diff of how we got the index of
ip4-policer-classify and setting the next node. (full diff is
attached).
--- a/src/plugins/nat/out2in.c
+++ b/src/plugins/nat/out2in.c
@@ -1110,6 +1113,15 @@ snat_out2in_node_fn (vlib_main_t * vm,
proto0 = ip_proto_to_snat_proto (ip0->protocol);
+ ip4_policer_classify_node =
+vlib_get_node_by_name (vm, (u8 *) "ip4-policer-classify");
+ if (ip4_policer_classify_node)
+{
+ next0 = SNAT_OUT2IN_NEXT_POLICER_CLASSIFY;
+ vlib_node_add_next (vm, ip4_policer_classify_node->index,
+ next0);
+}
+
if (PREDICT_FALSE (proto0 == ~0))
{
if (nat_out2in_sm_unknown_proto (sm, b0, ip0, rx_fib_index0))
I hope the approach followed is the correct one, but I could not
figure out why the ASSERT is happening.
Thanks and Regards,
Raj
On Tue, Jan 22, 2019 at 8:10 PM Matus Fabian -X (matfabia - PANTHEON
TECHNOLOGIES at Cisco) wrote:
>
> nat44-out2in node:
> u32 next0 = SNAT_OUT2IN_NEXT_LOOKUP;
> <...>
> vlib_validate_buffer_enqueue_x1 (vm, node, next_index, to_next,
> n_left_to_next, bi0, next0);
>
> whatever you specify in VNET_FEATURE_INIT runs_before is ignored for
> nat44-out2in, normally when you want continue to nex node in feature arc you
> use vnet_feature_next(), but this is not possible in NAT (nat44-out2in is not
> always configured as interface feature, e.g. worker handoff in case of
> multithreading or combined in-out NAT interface).
>
> Matus
>
> -Original Message-
> From: Raj
> Sent: Tuesday, January 22, 2019 3:22 PM
> To: Matus Fabian -X (matfabia - PANTHEON TECHNOLOGIES at Cisco)
>
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Configuring NAT and Policing together
>
> On Tue, Jan 22, 2019 at 7:44 PM Matus Fabian -X (matfabia - PANTHEON
> TECHNOLOGIES at Cisco) wrote:
> > I don't think it is working way you wanted since nat44-out2in goes directly
> > to ip4-lookup instead of continue in feature arc to ip4-policer-classify.
>
> Yes, you were right. My conclusion was premature. I still do not quite
> understand VNET_FEATURE_INIT to route the traffic the way I want. A sample
> code fragment would be very helpful.
>
> Thanks and Regards,
>
> Raj
diff --git a/src/plugins/nat/out2in.c b/src/plugins/nat/out2in.c
index 8c013d9b..401bcacc 100755
--- a/src/plugins/nat/out2in.c
+++ b/src/plugins/nat/out2in.c
@@ -34,6 +34,7 @@
#include
#include
+
typedef struct
{
u32 sw_if_index;
@@ -103,6 +104,7 @@ typedef enum
SNAT_OUT2IN_NEXT_LOOKUP,
SNAT_OUT2IN_NEXT_ICMP_ERROR,
SNAT_OUT2IN_NEXT_REASS,
+ SNAT_OUT2IN_NEXT_POLICER_CLASSIFY,
SNAT_OUT2IN_N_NEXT,
} snat_out2in_next_t;
@@ -1086,6 +1088,7 @@ snat_out2in_node_fn (vlib_main_t * vm,
snat_session_t *s0 = 0;
clib_bihash_kv_8_8_t kv0, value0;
u8 identity_nat0;
+ vlib_node_t *ip4_policer_classify_node = NULL;
/* speculatively enqueue b0 to the current next frame */
bi0 = from[0];
@@ -1110,6 +1113,15 @@ snat_out2in_node_fn (vlib_main_t * vm,
proto0 = ip_proto_to_snat_proto (ip0->protocol);
+ ip4_policer_classify_node =
+vlib_get_node_by_name (vm, (u8 *) "ip4-policer-classify");
+ if (ip4_policer_classify_node)
+{
+ next0 = SNAT_OUT2IN_NEXT_POLICER_CLASSIFY;
+ vlib_node_add_next (vm, ip4_policer_classify_node->index,
+ next0);
+}
+
if (PREDICT_FALSE (proto0 == ~0))
{
if (nat_out2in_sm_unknown_proto (sm, b0, ip0, rx_fib_index0))
@@ -1295,6 +1307,7 @@ VLIB_REGISTER_NODE (snat_out2in_node) = {
[SNAT_OUT2IN_NEXT_LOOKUP] = "ip4-lookup",
[SNAT_OUT2IN_NEXT_ICMP_ERROR] = "ip4-icmp-error",
[SNAT_OUT2IN_NEXT_REASS] = "nat44-out2in-reass",
+[SNAT_OUT2IN_NEXT_POLICER_CLASSIFY] = "ip4-policer-classify",
},
};
/* *INDENT-ON* */
@@ -1343,6 +1356,8 @@ nat44_out2in_reass_node_fn (vlib_main_t * vm,
u16 old_port0, new_port0;
ip_csum_t sum0;
u8 identity_nat0;
+ vlib_node_t *ip4_policer_classify_node = NULL;
+
/* speculatively enqu
Re: [vpp-dev] Configuring NAT and Policing together
On Tue, Jan 22, 2019 at 7:44 PM Matus Fabian -X (matfabia - PANTHEON TECHNOLOGIES at Cisco) wrote: > I don't think it is working way you wanted since nat44-out2in goes directly > to ip4-lookup instead of continue in feature arc to ip4-policer-classify. Yes, you were right. My conclusion was premature. I still do not quite understand VNET_FEATURE_INIT to route the traffic the way I want. A sample code fragment would be very helpful. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11971): https://lists.fd.io/g/vpp-dev/message/11971 Mute This Topic: https://lists.fd.io/mt/29379239/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Configuring NAT and Policing together
Hi Matus,
This is the patch I have made and looks like its working.
diff --git a/src/plugins/nat/nat.c b/src/plugins/nat/nat.c
index 540d3bf8..ac735e9a 100755
--- a/src/plugins/nat/nat.c
+++ b/src/plugins/nat/nat.c
@@ -47,6 +47,7 @@ VNET_FEATURE_INIT (ip4_snat_in2out, static) = {
VNET_FEATURE_INIT (ip4_snat_out2in, static) = {
.arc_name = "ip4-unicast",
.node_name = "nat44-out2in",
+ .runs_before = VNET_FEATURES ("ip4-policer-classify"),
.runs_after = VNET_FEATURES ("acl-plugin-in-ip4-fa",
"ip4-dhcp-client-detect"),
};
Its just first time hacking on VPP code, so do let me know if this is
the right way or there is a better way.
Thanks and Regards,
Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#11969): https://lists.fd.io/g/vpp-dev/message/11969
Mute This Topic: https://lists.fd.io/mt/29379239/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Configuring NAT and Policing together
Hi Matus,
We were looking to modify the flow so that the south->north path looks
like ip4-input-no-checksum-> ip4-policer-classify -> nat44-in2out ->
ip4-lookup and north->south path should be ip4-input-no-checksum ->
nat44-in2out -> ip4-policer-classify -> ip4-lookup
With your suggested modifications we were able to get the desired flow
for south->north path, but with a small change, by putting
"nat44-out2in" in .runs_after. The code looks like:
.runs_before = VNET_FEATURES ("ipsec-input-ip4","nat44-in2out"),
.runs_after = VNET_FEATURES ("nat44-out2in"),
But in north->south direction ip4-policer-classify node is being skipped.
Also I did not fully understand this statement: "It would be possible
to add additional static graph arc from nat node to
ip4-policer-classify and decide on per-packet basis where to send
packet since you don't know at compile time whether policer is
configured on interface.".
Thanks and Regards,
Raj
On Mon, Jan 21, 2019 at 6:04 PM Matus Fabian -X (matfabia - PANTHEON
TECHNOLOGIES at Cisco) wrote:
>
> Hi,
>
> You can use ip4-policer-classify before NAT node. Add nat44-in2out or
> nat44-out2in to ip4_policer_classify runs_before list
> VNET_FEATURE_INIT (ip4_policer_classify, static) =
> {
> .arc_name = "ip4-unicast",
> .node_name = "ip4-policer-classify",
> .runs_before = VNET_FEATURES ("ipsec4-input-feature", "nat44-in2out",
> "nat44-out2in"),
> };
>
> NAT code can't continue in feature arc using vnet_feature_next in some cases.
> It would be possible to add additional static graph arc from nat node to
> ip4-policer-classify and decide on per-packet basis where to send packet
> since you don't know at compile time whether policer is configured on
> interface.
>
> Matus
>
>
> -Original Message-
> From: vpp-dev@lists.fd.io On Behalf Of Raj
> Sent: Monday, January 21, 2019 1:00 PM
> To: vpp-dev@lists.fd.io
> Subject: [vpp-dev] Configuring NAT and Policing together
>
> Hello all,
>
> I am trying to configure NAT and VPP run together, but its not working.
>
> My configuration is as follows:
>
> version: vpp v18.10-release built by root on 41f0552eeae3
>
> Interfaces:
>
> GigabitEthernet1/0/0 (up):
> L3 100.69.1.1/24
> L3 2001:xxx:xxx:600::1/56
> GigabitEthernet1/0/1 (up):
> L3 xxx.79.223.14/29
> L3 2001:xxx:xxx:10d::600/64
>
> Policer config with default route:
>
> configure policer name policy1 cir 500 eir 0 cb 5000 eb 15000 rate kbps round
> closest type 1r3c conform-action transmit exceed-action mark-and-transmit
> AF22 violate-action drop configure policer name policy2 cir 750 eir 0 cb 7500
> eb 2 rate kbps round closest type 1r3c conform-action transmit
> exceed-action mark-and-transmit AF22 violate-action drop classify table mask
> l3 ip4 src classify table mask l3 ip4 dst classify session policer-hit-next
> policy1 exceed-color table-index 0 match l3 ip4 src 100.69.1.4 classify
> session policer-hit-next policy2 exceed-color table-index 1 match l3 ip4 dst
> 100.69.1.4 set policer classify interface GigabitEthernet1/0/0 ip4-table 0
> set policer classify interface GigabitEthernet1/0/1 ip4-table 1 ip route add
> 0.0.0.0/0 via xxx.79.223.9 GigabitEthernet1/0/1 ip route add ::/0 via
> 2001:xxx::10d::1 GigabitEthernet1/0/1
>
> At this point, if I do a wget at 100.69.1.4 to download from xxx.79.223.9,
> the speed is about 1mbps, but ranging from about 1.5mbps to 831kbps
>
> /dev/null 14%[===> ] 75.30M 1.18Mb/s
>
> The packet trace show:
>
> 100.69.1.4 -> xxx.79.223.9
>
> 01:10:21:269382: dpdk-input
> GigabitEthernet1/0/0 rx queue 0
> 01:10:21:269383: ip4-input-no-checksum
> 01:10:21:269384: ip4-policer-classify
> 01:10:21:269384: ip4-lookup
> 01:10:21:269384: ip4-rewrite
> 01:10:21:269384: GigabitEthernet1/0/1-output
> 01:10:21:269385: GigabitEthernet1/0/1-tx
>
>
> xxx.79.223.9 -> 100.69.1.4
>
> 01:10:21:268964: dpdk-input
> GigabitEthernet1/0/1 rx queue 0
> 01:10:21:268970: ip4-input-no-checksum
> 01:10:21:268973: ip4-policer-classify
> 01:10:21:268974: ip4-lookup
> 01:10:21:268975: ip4-rewrite
> 01:10:21:268976: GigabitEthernet1/0/0-output
> 01:10:21:268976: GigabitEthernet1/0/0-tx
>
> Now adding NAT using the commands:
>
> nat44 add interface address GigabitEthernet1/0/1 set interface nat44 in
> GigabitEthernet1/0/0 out GigabitEthernet1/0/1
>
> Policer stops working at this point.
>
> traces show:
>
> 100.69.1.4 -> xxx.79.223.9
>
> 01:23:19:656284: dpdk-input
> GigabitEthernet1/0/0 rx queue 0
> 01:23:19:656285: ip4-
[vpp-dev] Configuring NAT and Policing together
Hello all, I am trying to configure NAT and VPP run together, but its not working. My configuration is as follows: version: vpp v18.10-release built by root on 41f0552eeae3 Interfaces: GigabitEthernet1/0/0 (up): L3 100.69.1.1/24 L3 2001:xxx:xxx:600::1/56 GigabitEthernet1/0/1 (up): L3 xxx.79.223.14/29 L3 2001:xxx:xxx:10d::600/64 Policer config with default route: configure policer name policy1 cir 500 eir 0 cb 5000 eb 15000 rate kbps round closest type 1r3c conform-action transmit exceed-action mark-and-transmit AF22 violate-action drop configure policer name policy2 cir 750 eir 0 cb 7500 eb 2 rate kbps round closest type 1r3c conform-action transmit exceed-action mark-and-transmit AF22 violate-action drop classify table mask l3 ip4 src classify table mask l3 ip4 dst classify session policer-hit-next policy1 exceed-color table-index 0 match l3 ip4 src 100.69.1.4 classify session policer-hit-next policy2 exceed-color table-index 1 match l3 ip4 dst 100.69.1.4 set policer classify interface GigabitEthernet1/0/0 ip4-table 0 set policer classify interface GigabitEthernet1/0/1 ip4-table 1 ip route add 0.0.0.0/0 via xxx.79.223.9 GigabitEthernet1/0/1 ip route add ::/0 via 2001:xxx::10d::1 GigabitEthernet1/0/1 At this point, if I do a wget at 100.69.1.4 to download from xxx.79.223.9, the speed is about 1mbps, but ranging from about 1.5mbps to 831kbps /dev/null 14%[===> ] 75.30M 1.18Mb/s The packet trace show: 100.69.1.4 -> xxx.79.223.9 01:10:21:269382: dpdk-input GigabitEthernet1/0/0 rx queue 0 01:10:21:269383: ip4-input-no-checksum 01:10:21:269384: ip4-policer-classify 01:10:21:269384: ip4-lookup 01:10:21:269384: ip4-rewrite 01:10:21:269384: GigabitEthernet1/0/1-output 01:10:21:269385: GigabitEthernet1/0/1-tx xxx.79.223.9 -> 100.69.1.4 01:10:21:268964: dpdk-input GigabitEthernet1/0/1 rx queue 0 01:10:21:268970: ip4-input-no-checksum 01:10:21:268973: ip4-policer-classify 01:10:21:268974: ip4-lookup 01:10:21:268975: ip4-rewrite 01:10:21:268976: GigabitEthernet1/0/0-output 01:10:21:268976: GigabitEthernet1/0/0-tx Now adding NAT using the commands: nat44 add interface address GigabitEthernet1/0/1 set interface nat44 in GigabitEthernet1/0/0 out GigabitEthernet1/0/1 Policer stops working at this point. traces show: 100.69.1.4 -> xxx.79.223.9 01:23:19:656284: dpdk-input GigabitEthernet1/0/0 rx queue 0 01:23:19:656285: ip4-input-no-checksum 01:23:19:656285: nat44-in2out 01:23:19:656285: ip4-lookup 01:23:19:656286: ip4-rewrite 01:23:19:656286: GigabitEthernet1/0/1-output 01:23:19:656286: GigabitEthernet1/0/1-tx xxx.79.223.9 -> xxx.79.223.14 01:23:19:656289: dpdk-input GigabitEthernet1/0/1 rx queue 0 01:23:19:656290: ip4-input-no-checksum 01:23:19:656290: nat44-out2in 01:23:19:656290: ip4-lookup 01:23:19:656290: ip4-rewrite 01:23:19:656290: GigabitEthernet1/0/0-output 01:23:19:656291: GigabitEthernet1/0/0-tx The traces show that when NAT is enabled, policer nodes are not getting traversed. Ideally 100.69.1.4 -> xxx.79.223.9 should have ip4-input-no-checksum -> ip4-policer-classify -> nat44-in2out -> ip4-lookup and xxx.79.223.9 -> 100.69.1.4 should have ip4-input-no-checksum -> nat44-in2out -> ip4-policer-classify -> ip4-lookup Is such a configuration possible? How can I configure VPP for it? Is there any incompatibility between NAT and Policer? Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11963): https://lists.fd.io/g/vpp-dev/message/11963 Mute This Topic: https://lists.fd.io/mt/29379239/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Status of PPPoE Plugin
Hello Hongjun and Alp Arslan, Just checking if there is any update on this patch. with regards, raj On Thu, Dec 20, 2018 at 6:17 AM Ni, Hongjun wrote: > Hi Raj, > > For the issue you mentioned, Alp Arslan in the To list will submit a patch to > fix it. > > To Alp Arslan, > Thank you for your great help! > -Original Message- > From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Raj > Sent: Tuesday, December 18, 2018 8:11 PM > To: vpp-dev@lists.fd.io > Subject: [vpp-dev] Status of PPPoE Plugin > > Hello all, > > I am trying out PPPoE with VPP and while seaching for its configuration I > found the following thread > https://lists.fd.io/g/vpp-dev/message/10844 which kind of suggests that the > currently PPPoE is broken and that the code must be moved to tapv2. > > The problem people get is that packets from CP is getting dropped and I quote: > > https://lists.fd.io/g/vpp-dev/message/11199> > Since PPPoE control packet is special, which destination MAC is the PPPoE > client's MAC. Need to submit a patch to identify it and not perform L3 MAC > filter in ethernet-input-inline() function. > > > Just checking if this patch is available or is there any other way to get > PPPoE working in latest version? > > Thanks and Regards, > > Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11931): https://lists.fd.io/g/vpp-dev/message/11931 Mute This Topic: https://lists.fd.io/mt/28791570/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Understanding policer configuration
Hello all, I am trying to set up policing in VPP, using RESTCONF. The postman collections provided by hc2vpp gives the parameters and requests that needs to be made to configure policing. While configuring I do not understand what classifier table and classifier session is. classify table has a mask parameter and session has a match parameter, what do they do? The example shows how to configure policing for one IP address, what needs to be done to configure it for another IP address or a network? Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11894): https://lists.fd.io/g/vpp-dev/message/11894 Mute This Topic: https://lists.fd.io/mt/28993544/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Using prefetch and buffer_enqueue
On Wed, Jan 9, 2019 at 4:18 PM Damjan Marion wrote: >> On 9 Jan 2019, at 09:53, Raj wrote: >> 1. static_always_inline void >> vlib_buffer_enqueue_to_next (vlib_main_t * vm, vlib_node_runtime_t * node, >>u32 * buffers, u16 * nexts, uword count) >> > > This function simply gets array of buffer indices and array of next indices, > and enqueues those buffers to be processed by those next nodes... Is this function only used in the case of feature arc, where some packets satisfies certain criteria and moves to a different node? Basically from where do we get the array of next indices? Also in the code certain macros CLIB_HAVE_VEC128, CLIB_HAVE_VEC256 and CLIB_HAVE_VEC512 are defined. I could not figure out what these do. >> type can be either LOAD or STORE. What is the purpose of these two >> operations? > On x86, there was only load prefetch for a long time, recently store prefetch > is introduced > with Skylake microarchitecture. You can find explanation here: Based on code walkthrough, one thing I have observed is where ever .function is defined, static binding happens. In other nodes, dynamic binding based on the macro VLIB_NODE_FN happens. So If I understand correctly, we have to use STORE in the latter case. But we can use LOAD as well. There is no hard and fast rule. Please correct me if I'm wrong. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11893): https://lists.fd.io/g/vpp-dev/message/11893 Mute This Topic: https://lists.fd.io/mt/28982509/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Using prefetch and buffer_enqueue
Hello all, While going through the VPP code, I have not fully understood what the following functions mean and when should they be used? 1. static_always_inline void vlib_buffer_enqueue_to_next (vlib_main_t * vm, vlib_node_runtime_t * node, u32 * buffers, u16 * nexts, uword count) I could see following comments about this function /* * Send the packets along their respective next-node graph arcs * Considerable locality of reference is expected, most if not all * packets in the inbound vector will traverse the same next-node * arc */ I tried reading through the code but could not figure out what this function does, and how this is implemented. I see that the code has optimizations for various CPU, but not could not figure out what those optimizations are. 2. #define vlib_prefetch_buffer_header(b, type ) CLIB_PREFETCH (b, 64, type) This gets expanded to: #define _CLIB_PREFETCH(n,size,type) \ if ((size) > (n)*CLIB_CACHE_LINE_BYTES) \ __builtin_prefetch (_addr + (n)*CLIB_CACHE_LINE_BYTES, \ CLIB_PREFETCH_##type, \ /* locality */ 3); type can be either LOAD or STORE. What is the purpose of these two operations? I am sure I am missing some thing with respect to understanding prefetch. I assume LOAD will prefetch data from memory, and STORE performs its inverse operation? Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11878): https://lists.fd.io/g/vpp-dev/message/11878 Mute This Topic: https://lists.fd.io/mt/28982509/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Mellanox ConnectX-4 Lx cards binding with DPDK, but not recognized by VPP #vpp
*Setup:* ·Platform – GNU/Linux ·Kernel – 4.4.0-131-generic ·Processor – x86_64 ·OS - Ubuntu 16.04 *MLNX_OFED Driver Version :* ·4.1-1.0.2.0 Followed link : https://community.mellanox.com/s/article/how-to-build-vpp-fd-io--160--development-environment-with-mellanox-dpdk-pmd-for-connectx-4-and-connectx-5 Installation Successful and bind to DPDK with vfio-pci, but not recognized by VPP # vppctl sh pci address Sock VID:PID Link Speed Driver Product Name Vital Product Data :02:00.0 15b3:1015 8.0 GT/s x4 vfio-pci :02:00.1 15b3:1015 8.0 GT/s x4 vfio-pci :03:00.0 15b3:1015 8.0 GT/s x4 vfio-pci :03:00.1 15b3:1015 8.0 GT/s x4 vfio-pci :04:00.0 8086:1539 2.5 GT/s x1 igb :05:00.0 8086:1539 2.5 GT/s x1 igb 0 000:06:00.0 8086:1539 2.5 GT/s x1 igb :07:00.0 8086:1539 2.5 GT/s x1 igb :08:00.0 8086:1539 2.5 GT/s x1 igb #vppctl sh int Name Idx State Counter Count local0 0 down A -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11876): https://lists.fd.io/g/vpp-dev/message/11876 Mute This Topic: https://lists.fd.io/mt/28982352/21656 Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP coding techniques
On Thu, Jan 3, 2019 at 4:44 PM Damjan Marion wrote: > From VPP perspective sample plugin might be good start, if you have > any particular questions, drop email here and we will try to explain... Thanks a lot. Will get back if there are questions. Thanks and Regards, Raj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11826): https://lists.fd.io/g/vpp-dev/message/11826 Mute This Topic: https://lists.fd.io/mt/28903866/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Simple Rate Limit and QoS #vpp
Found some mail list links with reference to configuring policing in VPP https://jira.fd.io/browse/HC2VPP-39 https://www.mail-archive.com/vpp-dev@lists.fd.io/msg06792.html https://lists.fd.io/g/vpp-dev/message/3579 Based on the above links, I have used the following commands to configure policing. configure policer name policy1 cir 5 eir 5 cb 5 eb 5 rate kbps round up type 2r3c-4115 conform-action transmit exceed-action drop violate-action drop classify table mask l2 ignore-tag1 l3 ip4 src classify session policer-hit-next policy1 exceed-color table-index 0 match l2 ignore-tag1 l3 ip4 src 100.69.1.4 set policer classify interface TenGigabitEthernet82/0/0 ip4-table 0 TenGigabitEthernet82/0/0 is the LAN interface of the VPP where 100.69.1.4 is connected. While I do not fully understand the command I had given I assume it to police the traffic rate at 5kbps. But that is not working. I am able to download at the full interface speed of 100.69.1.4 Any pointers to get this working? Thanks and Regards, Raj On Tue, Jan 1, 2019 at 2:56 PM Raj via Lists.Fd.Io wrote: > > Same here, some example for QoS/Rate limiting would be nice to get started. > > Thanks and Regards, > > Raj > > On Sat, Dec 29, 2018 at 9:39 AM carlito nueno wrote: > > > > > > I am looking for rate limiting (bandwidth/traffic shaping) as well. > > > > > > Vakili, Did you figure it out? > > > > Thanks. > > On Sat, Sep 8, 2018 at 12:16 AM wrote: > >> > >> Simple Rate Limit and QoS > >> Hi dears. Three questions please: > >> 1: How can I configure an interface to let pass limited rate (Bandwidth > >> management) in VPP > >> 2: Can I give rage of IPs to assign limit rates? > >> 3: Is there any way to not restart vpp or reload interface configuration > >> for any configuration? I need to save and run without restart. > >> > >> Thanks alot > >> Vakili -=-=-=-=-=-=-=-=-=-=-=- > >> Links: You receive all messages sent to this group. > >> > >> View/Reply Online (#10441): https://lists.fd.io/g/vpp-dev/message/10441 > >> Mute This Topic: https://lists.fd.io/mt/25362068/675621 > >> Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480478 > >> Group Owner: vpp-dev+ow...@lists.fd.io > >> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [carlitonu...@gmail.com] > >> -=-=-=-=-=-=-=-=-=-=-=- > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > > > View/Reply Online (#11793): https://lists.fd.io/g/vpp-dev/message/11793 > > Mute This Topic: https://lists.fd.io/mt/25362068/157026 > > Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=3532616 > > Group Owner: vpp-dev+ow...@lists.fd.io > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [rajlistu...@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > > View/Reply Online (#11809): https://lists.fd.io/g/vpp-dev/message/11809 > Mute This Topic: https://lists.fd.io/mt/25362068/157026 > Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=3532616 > Group Owner: vpp-dev+ow...@lists.fd.io > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [rajlistu...@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11822): https://lists.fd.io/g/vpp-dev/message/11822 Mute This Topic: https://lists.fd.io/mt/25362068/21656 Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
