Re: [vpp-dev] Question about vlib_next_frame_change_ownership

[Kingwel Xie]

Thanks for the clarify. Makes sense.


 原始邮件 
主题： Re: [vpp-dev] Question about vlib_next_frame_change_ownership
来自： Damjan Marion 
发至： 2019年1月25日 下午10:56
抄送： Dave Barach 

Just to avoid potential confusion, my recent ethernet-input change mandates that
frames are not aggregated when they are coming from input nodes. Reason for 
that is much faster
processing of the frame when we know in advance that all packets in frame share 
same sw_if_index.

In other words, input node calls vlib_get_new_next_frame() instead of 
vlib_get_next_frame()
and it stores flags and sw_if_index inside frame scalar data

At the moment this is only case in current codebase which doesn't aggregate...


On 25 Jan 2019, at 14:03, Dave Barach via Lists.Fd.Io 
mailto:dbarach=cisco@lists.fd.io>> wrote:

Dear Kingwei,

On a per-thread basis, only one input node is active at a time. In the 2x 
active input node case you sketched, the second input node will take frame 
ownership of the ethernet input frame - which should be pending but not yet 
dispatched - and add more buffer indices to it. It’s possible that the first 
frame will fill and that a second frame will be allocated and [partially] 
filled; that’s not the typical case.

One lap around the track later, the first input node will take back ownership.

Let’s say that input node 1 adds 50 packets to the ethernet-input frame, and 
input node 2 adds another 50 packets. The frame ownership change dance yields a 
vector of size 100 in ethernet-input. My guess is that the increase in 
efficiency from ethernet-input forward in the graph more than compensates for 
the fixed cost of the frame change ownership dance. This is to confirm what you 
wrote.

I usually call this effect “input aggregation.” Also holds true for the handoff 
node, especially when handing off frames from multiple threads to a single 
thread.

The alternative: dispatch two smaller frames instead of one big one. Doing that 
might not be awful if all input nodes produced a fair number of packets. The 
situation becomes awful when the 2..N input nodes produce very different 
numbers of packets, e.g. 99 and 1. Anyhow, the code doesn’t work that way and 
isn’t going to work that way, so I digress...

Thanks... Dave

From: Kingwel Xie mailto:kingwel@ericsson.com>>
Sent: Friday, January 25, 2019 5:13 AM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: RE: [vpp-dev] Question about vlib_next_frame_change_ownership

Hi Dave,

After checking the code and some debug sessions, I realized where the bug is – 
crypto-input, which is calling vlib_trace_buffer() earlier than 
get_next_frame/put_next_frame. Therefore, the next_frame->flag is overwritten 
by get_next_frame/change_owenrship. I’ve made a patch for it: 
https://gerrit.fd.io/r/17079  appreciate your comments.

Again, about vlib_next_frame_change_ownership, as I understand, this mechanism 
will always try to enqueue buffers to the owner’s frame_index, so please 
consider this scenario:

We have 2 input nodes running at the same time – dpdk-input and avf-input, they 
will try to compete with each other to enqueue to ethernet-input. Consequently, 
bad side, memcpy per frame as we discussed, and, good side, assume they both 
have 10 buffers per frame, then ethernet-input will have chance to get 20 
buffers for batch processing, good for performance. Please confirm if this is 
expected behavior of frame ownership.

Regards,
Kingwel


From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Dave Barach via 
Lists.Fd.Io
Sent: Friday, January 25, 2019 4:52 AM
To: Kingwel Xie mailto:kingwel@ericsson.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Question about vlib_next_frame_change_ownership

The vpp packet trace which I extracted from your dispatch trace seems exactly 
as I would have expected. See below. In a pg test like this one using a 
loopback interface, anything past loopN-tx is irrelevant. The ipsec packet 
turns into an ARP request for 18.1.0.241.

In non-cyclic graph cases, we don’t end up changing frame ownership at all. In 
this case, you’re doing a double lookup. One small memcpy per frame is a 
trivial cost, especially when one remembers that the cost is amortized over all 
the packets in the frame.

Until you produce a repeatable demonstration of the claimed issue, there’s 
nothing that we can do.

Thanks... Dave

VPP Buffer Trace
Trace:
Trace: 00:00:53:959410: pg-input
Trace:   stream ipsec0, 100 bytes, 0 sw_if_index
Trace:   current data 0, length 100, buffer-pool 0, clone-count 0, trace 0x0
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959426: ip4-input
Trace:   UDP: 

Re: [vpp-dev] How do I get the "dpdk-shared" in VPP ?

[Burt Silverman]

I tried the procedure here. The name of the directory "build" is arbitrary;
I used "burt." Things worked (i.e., no build errors) when I located burt
under vpp but when I tried locating it under ~, cmake complained that I was
not inside a git space (which is true), due to the pack.cmake file.

Burt
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12020): https://lists.fd.io/g/vpp-dev/message/12020
Mute This Topic: https://lists.fd.io/mt/29529767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] RFC: buffer manager rework

[Florin Coras]

Awesome! Looking forward to using it in the host stack ;-)

Florin

> On Jan 25, 2019, at 9:08 AM, Damjan Marion via Lists.Fd.Io 
>  wrote:
> 
> 
> I am very close to the finish line with buffer management rework patch, and 
> would like to 
> ask people to take a look before it is merged.
> 
> https://gerrit.fd.io/r/16638
> 
> It significantly improves performance of buffer alloc free and introduces 
> numa awareness.
> On my skylake platinum 8180 system, with native AVF driver observed 
> performance improvement is:
> 
> - single core, 2 threads, ipv4 base forwarding test, CPU running at 2.5GHz 
> (TB off):
> 
> old code - dpdk buffer manager: 20.4 Mpps
> old code - old native buffer manager: 19.4 Mpps
> new code: 24.9 Mpps
> 
> With DPDK drivers performance stays same as DPDK is maintaining own internal 
> buffer cache. 
> So major perf gain should be observed in native code like: vhost-user, memif, 
> AVF, host stack.
> 
> user facing changes:
> to change number of buffers:
>  old startup.conf:
>dpdk { num-mbufs  } 
>  new startup.conf:
>buffers { buffers-per-numa }
> 
> Internal changes:
> - free lists are deprecated
> - buffer metadata is always initialised.
> - first 64-bytes of metadata are initialised on free, so buffer alloc is very 
> fast
> - DPDK mempools are not used anymore, we register custom mempool ops, and 
> dpdk is taking buffers from VPP
> - to support such operation plugin can request external header space - in 
> case of DPDK it stores rte_mbuf + rte_mempool_objhdr
> 
> I'm still running some tests so possible minor changes are possible, but 
> nothing major expected.
> 
> -- 
> Damjan
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#12016): https://lists.fd.io/g/vpp-dev/message/12016
> Mute This Topic: https://lists.fd.io/mt/29539221/675152
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [fcoras.li...@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12018): https://lists.fd.io/g/vpp-dev/message/12018
Mute This Topic: https://lists.fd.io/mt/29539221/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] RFC: buffer manager rework

[Damjan Marion via Lists.Fd.Io]

I am very close to the finish line with buffer management rework patch, and 
would like to 
ask people to take a look before it is merged.

https://gerrit.fd.io/r/16638

It significantly improves performance of buffer alloc free and introduces numa 
awareness.
On my skylake platinum 8180 system, with native AVF driver observed performance 
improvement is:

- single core, 2 threads, ipv4 base forwarding test, CPU running at 2.5GHz (TB 
off):

old code - dpdk buffer manager: 20.4 Mpps
old code - old native buffer manager: 19.4 Mpps
new code: 24.9 Mpps

With DPDK drivers performance stays same as DPDK is maintaining own internal 
buffer cache. 
So major perf gain should be observed in native code like: vhost-user, memif, 
AVF, host stack.

user facing changes:
to change number of buffers:
  old startup.conf:
dpdk { num-mbufs  } 
  new startup.conf:
buffers { buffers-per-numa }

Internal changes:
 - free lists are deprecated
 - buffer metadata is always initialised.
 - first 64-bytes of metadata are initialised on free, so buffer alloc is very 
fast
 - DPDK mempools are not used anymore, we register custom mempool ops, and dpdk 
is taking buffers from VPP
 - to support such operation plugin can request external header space - in case 
of DPDK it stores rte_mbuf + rte_mempool_objhdr

I'm still running some tests so possible minor changes are possible, but 
nothing major expected.

-- 
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12016): https://lists.fd.io/g/vpp-dev/message/12016
Mute This Topic: https://lists.fd.io/mt/29539221/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] How do I get the "dpdk-shared" in VPP ?

[Ray Kinsella]

I tried doing this recently and it barfed.
How did you get it working?

Ray K

> -Original Message-
> From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Marco
> Varlese
> Sent: Friday 25 January 2019 12:38
> To: Damjan Marion 
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] How do I get the "dpdk-shared" in VPP ?
> 
> Never mind... I did find the issue. All good ;)
> 
> Thank you Damjan!!!
> 
> On 1/25/19 1:26 PM, Marco Varlese wrote:
> >
> >
> > On 1/25/19 11:14 AM, Damjan Marion wrote:
> >>
> >>
> >>> On 25 Jan 2019, at 10:49, Marco Varlese  >>> > wrote:
> >>>
> >>> Hi Damjan,
> >>>
> >>> On 1/24/19 10:46 PM, Damjan Marion via Lists.Fd.Io wrote:
> 
>  In theory like any other cmake project:
> 
>  $ mkdir build
>  $ cd build
>  $ cmake /path/to/vpp/src  $ make $ make install
> >>> Hmmm, not sure if I explained myself in the right way.
> >>>
> >>> The problem today is that I cannot find a way to tell VPP _not_ to
> >>> download the dpdk tarball, uncompress it, build it and then link
> >>> against it...
> >>
> >> That is exactly what I described above
> >>
> >>>
> >>> How does the above help in that regard?
> >>
> >> It helps as cmake doesn't download dpdk or other libs. It is
> >> downloaded by bunch of makefiles outside of src/.
> >> If you don't need such service from those Makefiles, don''t use them..
> > OK, so I did try your suggestion but in that case the DPDK plugin
> > doesn't get built at all and the build procedure doesn't even get to
> > the point of looking for a DPDK library (whether statically or
> > dynamically linked).
> >>
> >>>
> 
>  In practice probably few lines should be modified in
>  src/plugins/dodkCMakeLists.txt to enable linking with shared libs,
>  as today we do static...
> >>> So the shared-lib support was removed when moving to cmake I suppose.
> >>> Was that intentional or just a miss?
> >>
> >> it was intentional miss :)
> >>
> >> --
> >> Damjan
> >>
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> >
> > View/Reply Online (#12008):
> > https://lists.fd.io/g/vpp-dev/message/12008
> > Mute This Topic: https://lists.fd.io/mt/29529767/675056
> > Group Owner: vpp-dev+ow...@lists.fd.io
> > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [mvarl...@suse.de]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
> 
> --
> Marco Varlese, Architect Developer Technologies, SUSE Labs SUSE LINUX
> GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton
> HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12015): https://lists.fd.io/g/vpp-dev/message/12015
Mute This Topic: https://lists.fd.io/mt/29529767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Question about crypto dev queue pairs #vpp

[manuel . alonso]

Hello all,

Just tracing a bit the code I noticed that there is a concept of "queue pair" 
and every crypto device allocates its own number of queue pairs.
Two questions (version 19.01):
1. Regarding the max_res_idx (ipsec.c) calculation:
max_res_idx = (dev->max_qp / 2) - 1;   (if dev->max_qp == 1 then max_res_idx = 
65535)
Notice that if max_qp is 1 then the loop below never ends (since j is always <= 
65535)
u16 j;
 for (j = 0; j <= max_res_idx; j++, res_idx++) {
}

Another issue that I am trying to understand is the snippet below (ipsec.h):
   enq = rte_cryptodev_enqueue_burst (res->dev_id, res->qp_id + outbound,
res->ops, res->n_ops);

outbound == 1 in the enqueue case and crypto resources under that function call 
 have only  "res->gp_id" queue pairs ids, not sure how can it work(unless there 
is a contiguous allocation from another device)

I'd welcome any help on understanding both points.

BR,
Manuel
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12014): https://lists.fd.io/g/vpp-dev/message/12014
Mute This Topic: https://lists.fd.io/mt/29538345/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Question about vlib_next_frame_change_ownership

[Damjan Marion via Lists.Fd.Io]

Just to avoid potential confusion, my recent ethernet-input change mandates that
frames are not aggregated when they are coming from input nodes. Reason for 
that is much faster
processing of the frame when we know in advance that all packets in frame share 
same sw_if_index.

In other words, input node calls vlib_get_new_next_frame() instead of 
vlib_get_next_frame()
and it stores flags and sw_if_index inside frame scalar data

At the moment this is only case in current codebase which doesn't aggregate...


> On 25 Jan 2019, at 14:03, Dave Barach via Lists.Fd.Io 
>  wrote:
> 
> Dear Kingwei,
>  
> On a per-thread basis, only one input node is active at a time. In the 2x 
> active input node case you sketched, the second input node will take frame 
> ownership of the ethernet input frame - which should be pending but not yet 
> dispatched - and add more buffer indices to it. It’s possible that the first 
> frame will fill and that a second frame will be allocated and [partially] 
> filled; that’s not the typical case.
>  
> One lap around the track later, the first input node will take back ownership.
>  
> Let’s say that input node 1 adds 50 packets to the ethernet-input frame, and 
> input node 2 adds another 50 packets. The frame ownership change dance yields 
> a vector of size 100 in ethernet-input. My guess is that the increase in 
> efficiency from ethernet-input forward in the graph more than compensates for 
> the fixed cost of the frame change ownership dance. This is to confirm what 
> you wrote.
>  
> I usually call this effect “input aggregation.” Also holds true for the 
> handoff node, especially when handing off frames from multiple threads to a 
> single thread.
>  
> The alternative: dispatch two smaller frames instead of one big one. Doing 
> that might not be awful if all input nodes produced a fair number of packets. 
> The situation becomes awful when the 2..N input nodes produce very different 
> numbers of packets, e.g. 99 and 1. Anyhow, the code doesn’t work that way and 
> isn’t going to work that way, so I digress...
>  
> Thanks... Dave  
>  
> From: Kingwel Xie  > 
> Sent: Friday, January 25, 2019 5:13 AM
> To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>; 
> vpp-dev mailto:vpp-dev@lists.fd.io>>
> Subject: RE: [vpp-dev] Question about vlib_next_frame_change_ownership
>  
> Hi Dave,
>  
> After checking the code and some debug sessions, I realized where the bug is 
> – crypto-input, which is calling vlib_trace_buffer() earlier than 
> get_next_frame/put_next_frame. Therefore, the next_frame->flag is overwritten 
> by get_next_frame/change_owenrship. I’ve made a patch for it: 
> https://gerrit.fd.io/r/17079   appreciate your 
> comments.
>  
> Again, about vlib_next_frame_change_ownership, as I understand, this 
> mechanism will always try to enqueue buffers to the owner’s frame_index, so 
> please consider this scenario:
>  
> We have 2 input nodes running at the same time – dpdk-input and avf-input, 
> they will try to compete with each other to enqueue to ethernet-input. 
> Consequently, bad side, memcpy per frame as we discussed, and, good side, 
> assume they both have 10 buffers per frame, then ethernet-input will have 
> chance to get 20 buffers for batch processing, good for performance. Please 
> confirm if this is expected behavior of frame ownership.
>  
> Regards,
> Kingwel
>  
>  
> From: vpp-dev@lists.fd.io   > On Behalf Of Dave Barach via Lists.Fd.Io
> Sent: Friday, January 25, 2019 4:52 AM
> To: Kingwel Xie mailto:kingwel@ericsson.com>>; 
> vpp-dev mailto:vpp-dev@lists.fd.io>>
> Cc: vpp-dev@lists.fd.io 
> Subject: Re: [vpp-dev] Question about vlib_next_frame_change_ownership
>  
> The vpp packet trace which I extracted from your dispatch trace seems exactly 
> as I would have expected. See below. In a pg test like this one using a 
> loopback interface, anything past loopN-tx is irrelevant. The ipsec packet 
> turns into an ARP request for 18.1.0.241.
>  
> In non-cyclic graph cases, we don’t end up changing frame ownership at all. 
> In this case, you’re doing a double lookup. One small memcpy per frame is a 
> trivial cost, especially when one remembers that the cost is amortized over 
> all the packets in the frame. 
>  
> Until you produce a repeatable demonstration of the claimed issue, there’s 
> nothing that we can do.
>  
> Thanks... Dave
>  
> VPP Buffer Trace
> Trace: 
> Trace: 00:00:53:959410: pg-input
> Trace:   stream ipsec0, 100 bytes, 0 sw_if_index
> Trace:   current data 0, length 100, buffer-pool 0, clone-count 0, trace 
> 0x0
> Trace:   UDP: 192.168.2.255 -> 1.2.3.4
> Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
> Trace: fragment id 0x
> Trace:   UDP: 4321 -> 1234
> Trace: length 80, checksum 0x30d9
> Trace: 00:00:53:959426: 

Re: [vpp-dev] Question about vlib_next_frame_change_ownership

[Kingwel Xie]

Thanks, Dave. Crystally clear! It is a deliberate design, indeed.

Regards,
Kingwel


 原始邮件 
主题： RE: [vpp-dev] Question about vlib_next_frame_change_ownership
来自： "Dave Barach (dbarach)" 
发至： 2019年1月25日 下午9:03
抄送： Kingwel Xie ,vpp-dev 
Dear Kingwei,

On a per-thread basis, only one input node is active at a time. In the 2x 
active input node case you sketched, the second input node will take frame 
ownership of the ethernet input frame - which should be pending but not yet 
dispatched - and add more buffer indices to it. It’s possible that the first 
frame will fill and that a second frame will be allocated and [partially] 
filled; that’s not the typical case.

One lap around the track later, the first input node will take back ownership.

Let’s say that input node 1 adds 50 packets to the ethernet-input frame, and 
input node 2 adds another 50 packets. The frame ownership change dance yields a 
vector of size 100 in ethernet-input. My guess is that the increase in 
efficiency from ethernet-input forward in the graph more than compensates for 
the fixed cost of the frame change ownership dance. This is to confirm what you 
wrote.

I usually call this effect “input aggregation.” Also holds true for the handoff 
node, especially when handing off frames from multiple threads to a single 
thread.

The alternative: dispatch two smaller frames instead of one big one. Doing that 
might not be awful if all input nodes produced a fair number of packets. The 
situation becomes awful when the 2..N input nodes produce very different 
numbers of packets, e.g. 99 and 1. Anyhow, the code doesn’t work that way and 
isn’t going to work that way, so I digress...

Thanks... Dave

From: Kingwel Xie 
Sent: Friday, January 25, 2019 5:13 AM
To: Dave Barach (dbarach) ; vpp-dev 
Subject: RE: [vpp-dev] Question about vlib_next_frame_change_ownership

Hi Dave,

After checking the code and some debug sessions, I realized where the bug is – 
crypto-input, which is calling vlib_trace_buffer() earlier than 
get_next_frame/put_next_frame. Therefore, the next_frame->flag is overwritten 
by get_next_frame/change_owenrship. I’ve made a patch for it: 
https://gerrit.fd.io/r/17079  appreciate your comments.

Again, about vlib_next_frame_change_ownership, as I understand, this mechanism 
will always try to enqueue buffers to the owner’s frame_index, so please 
consider this scenario:

We have 2 input nodes running at the same time – dpdk-input and avf-input, they 
will try to compete with each other to enqueue to ethernet-input. Consequently, 
bad side, memcpy per frame as we discussed, and, good side, assume they both 
have 10 buffers per frame, then ethernet-input will have chance to get 20 
buffers for batch processing, good for performance. Please confirm if this is 
expected behavior of frame ownership.

Regards,
Kingwel


From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Dave Barach via 
Lists.Fd.Io
Sent: Friday, January 25, 2019 4:52 AM
To: Kingwel Xie mailto:kingwel@ericsson.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Question about vlib_next_frame_change_ownership

The vpp packet trace which I extracted from your dispatch trace seems exactly 
as I would have expected. See below. In a pg test like this one using a 
loopback interface, anything past loopN-tx is irrelevant. The ipsec packet 
turns into an ARP request for 18.1.0.241.

In non-cyclic graph cases, we don’t end up changing frame ownership at all. In 
this case, you’re doing a double lookup. One small memcpy per frame is a 
trivial cost, especially when one remembers that the cost is amortized over all 
the packets in the frame.

Until you produce a repeatable demonstration of the claimed issue, there’s 
nothing that we can do.

Thanks... Dave

VPP Buffer Trace
Trace:
Trace: 00:00:53:959410: pg-input
Trace:   stream ipsec0, 100 bytes, 0 sw_if_index
Trace:   current data 0, length 100, buffer-pool 0, clone-count 0, trace 0x0
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959426: ip4-input
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959519: ip4-lookup
Trace:   fib 0 dpo-idx 2 flow hash: 0x
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959598: ip4-rewrite
Trace:   tx_sw_if_index 2 dpo-idx 2 : ipv4 via 0.0.0.0 ipsec0: mtu:9000 
flow hash: 0x
Trace:   

Re: [vpp-dev] Question about vlib_next_frame_change_ownership

[Dave Barach via Lists.Fd.Io]

Dear Kingwei,

On a per-thread basis, only one input node is active at a time. In the 2x 
active input node case you sketched, the second input node will take frame 
ownership of the ethernet input frame - which should be pending but not yet 
dispatched - and add more buffer indices to it. It’s possible that the first 
frame will fill and that a second frame will be allocated and [partially] 
filled; that’s not the typical case.

One lap around the track later, the first input node will take back ownership.

Let’s say that input node 1 adds 50 packets to the ethernet-input frame, and 
input node 2 adds another 50 packets. The frame ownership change dance yields a 
vector of size 100 in ethernet-input. My guess is that the increase in 
efficiency from ethernet-input forward in the graph more than compensates for 
the fixed cost of the frame change ownership dance. This is to confirm what you 
wrote.

I usually call this effect “input aggregation.” Also holds true for the handoff 
node, especially when handing off frames from multiple threads to a single 
thread.

The alternative: dispatch two smaller frames instead of one big one. Doing that 
might not be awful if all input nodes produced a fair number of packets. The 
situation becomes awful when the 2..N input nodes produce very different 
numbers of packets, e.g. 99 and 1. Anyhow, the code doesn’t work that way and 
isn’t going to work that way, so I digress...

Thanks... Dave

From: Kingwel Xie 
Sent: Friday, January 25, 2019 5:13 AM
To: Dave Barach (dbarach) ; vpp-dev 
Subject: RE: [vpp-dev] Question about vlib_next_frame_change_ownership

Hi Dave,

After checking the code and some debug sessions, I realized where the bug is – 
crypto-input, which is calling vlib_trace_buffer() earlier than 
get_next_frame/put_next_frame. Therefore, the next_frame->flag is overwritten 
by get_next_frame/change_owenrship. I’ve made a patch for it: 
https://gerrit.fd.io/r/17079  appreciate your comments.

Again, about vlib_next_frame_change_ownership, as I understand, this mechanism 
will always try to enqueue buffers to the owner’s frame_index, so please 
consider this scenario:

We have 2 input nodes running at the same time – dpdk-input and avf-input, they 
will try to compete with each other to enqueue to ethernet-input. Consequently, 
bad side, memcpy per frame as we discussed, and, good side, assume they both 
have 10 buffers per frame, then ethernet-input will have chance to get 20 
buffers for batch processing, good for performance. Please confirm if this is 
expected behavior of frame ownership.

Regards,
Kingwel


From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Dave Barach via 
Lists.Fd.Io
Sent: Friday, January 25, 2019 4:52 AM
To: Kingwel Xie mailto:kingwel@ericsson.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Question about vlib_next_frame_change_ownership

The vpp packet trace which I extracted from your dispatch trace seems exactly 
as I would have expected. See below. In a pg test like this one using a 
loopback interface, anything past loopN-tx is irrelevant. The ipsec packet 
turns into an ARP request for 18.1.0.241.

In non-cyclic graph cases, we don’t end up changing frame ownership at all. In 
this case, you’re doing a double lookup. One small memcpy per frame is a 
trivial cost, especially when one remembers that the cost is amortized over all 
the packets in the frame.

Until you produce a repeatable demonstration of the claimed issue, there’s 
nothing that we can do.

Thanks... Dave

VPP Buffer Trace
Trace:
Trace: 00:00:53:959410: pg-input
Trace:   stream ipsec0, 100 bytes, 0 sw_if_index
Trace:   current data 0, length 100, buffer-pool 0, clone-count 0, trace 0x0
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959426: ip4-input
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959519: ip4-lookup
Trace:   fib 0 dpo-idx 2 flow hash: 0x
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959598: ip4-rewrite
Trace:   tx_sw_if_index 2 dpo-idx 2 : ipv4 via 0.0.0.0 ipsec0: mtu:9000 
flow hash: 0x
Trace:   : 
451c3f11b424c0a802ff0102030410e104d2005030d900010203
Trace:   0020: 0405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f
Trace: 00:00:53:959687: ipsec0-output
Trace:   ipsec0
Trace:   : 

Re: [vpp-dev] How do I get the "dpdk-shared" in VPP ?

[Marco Varlese]

Never mind... I did find the issue. All good ;)

Thank you Damjan!!!

On 1/25/19 1:26 PM, Marco Varlese wrote:
> 
> 
> On 1/25/19 11:14 AM, Damjan Marion wrote:
>>
>>
>>> On 25 Jan 2019, at 10:49, Marco Varlese >> > wrote:
>>>
>>> Hi Damjan,
>>>
>>> On 1/24/19 10:46 PM, Damjan Marion via Lists.Fd.Io wrote:

 In theory like any other cmake project:

 $ mkdir build
 $ cd build
 $ cmake /path/to/vpp/src 
 $ make
 $ make install
>>> Hmmm, not sure if I explained myself in the right way.
>>>
>>> The problem today is that I cannot find a way to tell VPP _not_ to
>>> download the dpdk tarball, uncompress it, build it and then link against
>>> it...
>>
>> That is exactly what I described above
>>
>>>
>>> How does the above help in that regard?
>>
>> It helps as cmake doesn't download dpdk or other libs. It is downloaded
>> by bunch of makefiles outside of src/.
>> If you don't need such service from those Makefiles, don''t use them..
> OK, so I did try your suggestion but in that case the DPDK plugin
> doesn't get built at all and the build procedure doesn't even get to the
> point of looking for a DPDK library (whether statically or dynamically
> linked).
>>
>>>

 In practice probably few lines should be modified in
 src/plugins/dodkCMakeLists.txt to enable linking with shared libs, as
 today we do static...
>>> So the shared-lib support was removed when moving to cmake I suppose.
>>> Was that intentional or just a miss?
>>
>> it was intentional miss :)
>>
>> -- 
>> Damjan
>>
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#12008): https://lists.fd.io/g/vpp-dev/message/12008
> Mute This Topic: https://lists.fd.io/mt/29529767/675056
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [mvarl...@suse.de]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

-- 
Marco Varlese, Architect Developer Technologies, SUSE Labs
SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12009): https://lists.fd.io/g/vpp-dev/message/12009
Mute This Topic: https://lists.fd.io/mt/29529767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] How do I get the "dpdk-shared" in VPP ?

[Marco Varlese]

On 1/25/19 11:14 AM, Damjan Marion wrote:
> 
> 
>> On 25 Jan 2019, at 10:49, Marco Varlese > > wrote:
>>
>> Hi Damjan,
>>
>> On 1/24/19 10:46 PM, Damjan Marion via Lists.Fd.Io wrote:
>>>
>>> In theory like any other cmake project:
>>>
>>> $ mkdir build
>>> $ cd build
>>> $ cmake /path/to/vpp/src 
>>> $ make
>>> $ make install
>> Hmmm, not sure if I explained myself in the right way.
>>
>> The problem today is that I cannot find a way to tell VPP _not_ to
>> download the dpdk tarball, uncompress it, build it and then link against
>> it...
> 
> That is exactly what I described above
> 
>>
>> How does the above help in that regard?
> 
> It helps as cmake doesn't download dpdk or other libs. It is downloaded
> by bunch of makefiles outside of src/.
> If you don't need such service from those Makefiles, don''t use them..
OK, so I did try your suggestion but in that case the DPDK plugin
doesn't get built at all and the build procedure doesn't even get to the
point of looking for a DPDK library (whether statically or dynamically
linked).
> 
>>
>>>
>>> In practice probably few lines should be modified in
>>> src/plugins/dodkCMakeLists.txt to enable linking with shared libs, as
>>> today we do static...
>> So the shared-lib support was removed when moving to cmake I suppose.
>> Was that intentional or just a miss?
> 
> it was intentional miss :)
> 
> -- 
> Damjan
> 

-- 
Marco Varlese, Architect Developer Technologies, SUSE Labs
SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12008): https://lists.fd.io/g/vpp-dev/message/12008
Mute This Topic: https://lists.fd.io/mt/29529767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] one question about VPP IPsec implementaions

[Kingwel Xie]

First, performance. Second, still performance, you can use QAT with dpdk IPSEC.

Also note that dpdk ipsec doesn’t support AH. And, dpdk ipsec support GCM, but 
native one doesn’t.

There is the 3rd one, ipsecmb ipsec, which provides even better performance if 
using SW ciphering/hashing only. It currently supports AES-CBC only.

Hope this helps.


From: vpp-dev@lists.fd.io  On Behalf Of Zhiyong Yang
Sent: Friday, January 25, 2019 7:15 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] one question about VPP IPsec implementaions

Hi VPP expert,
When I look at the IPsec code，It looks that two implementations 
coexists in VPP. One is dpdk IPsec, and another is native IPsec.
I wonder what’s difference between them from functionality perspective?
Any input is appreciated.

Regards
Zhiyong

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12007): https://lists.fd.io/g/vpp-dev/message/12007
Mute This Topic: https://lists.fd.io/mt/29536073/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] one question about VPP IPsec implementaions

[Zhiyong Yang]

Hi VPP expert,
When I look at the IPsec code，It looks that two implementations 
coexists in VPP. One is dpdk IPsec, and another is native IPsec.
I wonder what’s difference between them from functionality perspective?
Any input is appreciated.

Regards
Zhiyong

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12006): https://lists.fd.io/g/vpp-dev/message/12006
Mute This Topic: https://lists.fd.io/mt/29536073/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] How do I get the "dpdk-shared" in VPP ?

[Damjan Marion via Lists.Fd.Io]

> On 25 Jan 2019, at 10:49, Marco Varlese  wrote:
> 
> Hi Damjan,
> 
> On 1/24/19 10:46 PM, Damjan Marion via Lists.Fd.Io wrote:
>> 
>> In theory like any other cmake project:
>> 
>> $ mkdir build
>> $ cd build
>> $ cmake /path/to/vpp/src 
>> $ make
>> $ make install
> Hmmm, not sure if I explained myself in the right way.
> 
> The problem today is that I cannot find a way to tell VPP _not_ to
> download the dpdk tarball, uncompress it, build it and then link against
> it...

That is exactly what I described above

> 
> How does the above help in that regard?

It helps as cmake doesn't download dpdk or other libs. It is downloaded by 
bunch of makefiles outside of src/.
If you don't need such service from those Makefiles, don''t use them..

> 
>> 
>> In practice probably few lines should be modified in 
>> src/plugins/dodkCMakeLists.txt to enable linking with shared libs, as today 
>> we do static...
> So the shared-lib support was removed when moving to cmake I suppose.
> Was that intentional or just a miss?

it was intentional miss :)

-- 
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12005): https://lists.fd.io/g/vpp-dev/message/12005
Mute This Topic: https://lists.fd.io/mt/29529767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Question about vlib_next_frame_change_ownership

[Kingwel Xie]

Hi Dave,

After checking the code and some debug sessions, I realized where the bug is – 
crypto-input, which is calling vlib_trace_buffer() earlier than 
get_next_frame/put_next_frame. Therefore, the next_frame->flag is overwritten 
by get_next_frame/change_owenrship. I’ve made a patch for it: 
https://gerrit.fd.io/r/17079  appreciate your comments.

Again, about vlib_next_frame_change_ownership, as I understand, this mechanism 
will always try to enqueue buffers to the owner’s frame_index, so please 
consider this scenario:

We have 2 input nodes running at the same time – dpdk-input and avf-input, they 
will try to compete with each other to enqueue to ethernet-input. Consequently, 
bad side, memcpy per frame as we discussed, and, good side, assume they both 
have 10 buffers per frame, then ethernet-input will have chance to get 20 
buffers for batch processing, good for performance. Please confirm if this is 
expected behavior of frame ownership.

Regards,
Kingwel


From: vpp-dev@lists.fd.io  On Behalf Of Dave Barach via 
Lists.Fd.Io
Sent: Friday, January 25, 2019 4:52 AM
To: Kingwel Xie ; vpp-dev 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Question about vlib_next_frame_change_ownership

The vpp packet trace which I extracted from your dispatch trace seems exactly 
as I would have expected. See below. In a pg test like this one using a 
loopback interface, anything past loopN-tx is irrelevant. The ipsec packet 
turns into an ARP request for 18.1.0.241.

In non-cyclic graph cases, we don’t end up changing frame ownership at all. In 
this case, you’re doing a double lookup. One small memcpy per frame is a 
trivial cost, especially when one remembers that the cost is amortized over all 
the packets in the frame.

Until you produce a repeatable demonstration of the claimed issue, there’s 
nothing that we can do.

Thanks... Dave

VPP Buffer Trace
Trace:
Trace: 00:00:53:959410: pg-input
Trace:   stream ipsec0, 100 bytes, 0 sw_if_index
Trace:   current data 0, length 100, buffer-pool 0, clone-count 0, trace 0x0
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959426: ip4-input
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959519: ip4-lookup
Trace:   fib 0 dpo-idx 2 flow hash: 0x
Trace:   UDP: 192.168.2.255 -> 1.2.3.4
Trace: tos 0x00, ttl 64, length 28, checksum 0xb324
Trace: fragment id 0x
Trace:   UDP: 4321 -> 1234
Trace: length 80, checksum 0x30d9
Trace: 00:00:53:959598: ip4-rewrite
Trace:   tx_sw_if_index 2 dpo-idx 2 : ipv4 via 0.0.0.0 ipsec0: mtu:9000 
flow hash: 0x
Trace:   : 
451c3f11b424c0a802ff0102030410e104d2005030d900010203
Trace:   0020: 0405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f
Trace: 00:00:53:959687: ipsec0-output
Trace:   ipsec0
Trace:   : 
451c3f11b424c0a802ff0102030410e104d2005030d900010203
Trace:   0020: 
0405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f20212223
Trace:   0040: 
2425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f40414243
Trace:   0060: 44454647
Trace: 00:00:53:959802: ipsec0-tx
Trace:   IPSec: spi 1 seq 1
Trace: 00:00:53:959934: esp4-encrypt
Trace:   esp: spi 1 seq 1 crypto aes-cbc-128 integrity sha1-96
Trace: 00:00:53:960084: ip4-lookup
Trace:   fib 0 dpo-idx 0 flow hash: 0x
Trace:   IPSEC_ESP: 18.1.0.71 -> 18.1.0.241
Trace: tos 0x00, ttl 254, length 168, checksum 0x96ea
Trace: fragment id 0x
Trace: 00:00:53:960209: ip4-glean
Trace: IPSEC_ESP: 18.1.0.71 -> 18.1.0.241
Trace:   tos 0x00, ttl 254, length 168, checksum 0x96ea
Trace:   fragment id 0x
Trace: 00:00:53:960336: loop0-output
Trace:   loop0
Trace:   ARP: de:ad:00:00:00:00 -> ff:ff:ff:ff:ff:ff
Trace:   request, type ethernet/IP4, address size 6/4
Trace:   de:ad:00:00:00:00/18.1.0.71 -> 00:00:00:00:00:00/18.1.0.241
Trace: 00:00:53:960491: error-drop
Trace:   ip4-glean: ARP requests sent


From: Kingwel Xie mailto:kingwel@ericsson.com>>
Sent: Thursday, January 24, 2019 2:43 AM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: RE: [vpp-dev] Question about vlib_next_frame_change_ownership

Ok. As requested, pcap trace & test script attached. Actually I made some 
simplification to indicate the problem – using native IPSEC instead of DPDK.

You can see in the buffer trace that ip-lookup is referred by ip-input in the 
beginning then by esp-encrypt later. 

Re: [vpp-dev] How do I get the "dpdk-shared" in VPP ?

[Marco Varlese]

Hi Damjan,

On 1/24/19 10:46 PM, Damjan Marion via Lists.Fd.Io wrote:
> 
> In theory like any other cmake project:
> 
> $ mkdir build
> $ cd build
> $ cmake /path/to/vpp/src 
> $ make
> $ make install
Hmmm, not sure if I explained myself in the right way.

The problem today is that I cannot find a way to tell VPP _not_ to
download the dpdk tarball, uncompress it, build it and then link against
it...

How does the above help in that regard?

> 
> In practice probably few lines should be modified in 
> src/plugins/dodkCMakeLists.txt to enable linking with shared libs, as today 
> we do static...
So the shared-lib support was removed when moving to cmake I suppose.
Was that intentional or just a miss?

> 
> — 
> Damjan
Cheers,
Marco

> 
>> On Jan 24, 2019, at 9:15 PM, Marco Varlese  wrote:
>>
>> Hi Damjan and all,
>>
>> How do I get VPP master and / or 19.01-rcX to build against a DPDK
>> already on my system?
>>
>> I am basically talking about the previously available feature driven via
>> vpp.mk as per below snippet:
>>vpp_uses_external_dpdk = yes
>>vpp_dpdk_inc_dir = /usr/include/dpdk
>>vpp_dpdk_lib_dir = /usr/lib
>>vpp_dpdk_shared_lib = yes
>>
>> I can't find how right now...
>>
>>
>> Thanks in advance,
>>
>> Marco
>>
>> -- 
>> Marco Varlese, Architect Developer Technologies, SUSE Labs
>> SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton
>> HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>>
>> View/Reply Online (#11995): https://lists.fd.io/g/vpp-dev/message/11995
>> Mute This Topic: https://lists.fd.io/mt/29529767/675642
>> Group Owner: vpp-dev+ow...@lists.fd.io
>> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [dmar...@me.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#11997): https://lists.fd.io/g/vpp-dev/message/11997
> Mute This Topic: https://lists.fd.io/mt/29529767/675056
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [mvarl...@suse.de]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

-- 
Marco Varlese, Architect Developer Technologies, SUSE Labs
SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12003): https://lists.fd.io/g/vpp-dev/message/12003
Mute This Topic: https://lists.fd.io/mt/29529767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Configuring NAT and Policing together

[Raj]

Hi,

I am checking whether policing feature is enabled or not in NAT with
the following approach
---
 Did an extern declaration for policer_classify_main_t in nat/out2in.c

  if (pcm->classify_table_index_by_sw_if_index[0][sw_if_index0] != ~0)
{
   /* feature is enabled */
   next0 = SNAT_OUT2IN_NEXT_POLICER_CLASSIFY;
   vlib_node_add_next (vm, ip4_policer_classify_node->index,
  next0);
}
---
I have a feeling that this is not the best/proper way, but I could not
figure out any thing better.  Please suggest is there any other
optimized/proper way to check an interface feature is enabled or not?

But when the packets starts flowing, VPP is getting aborted !!

DBGvpp# Aborted
Makefile:473: recipe for target 'run' failed
make: *** [run] Error 134

=> following is the gdb trace for the same <==
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x7f7b08a7542a in __GI_abort () at abort.c:89
#2  0x55d570fd6f5e in os_exit (code=1) at
/home//vpp/src/vpp/vnet/main.c:349
#3  0x7f7b0a1e7cac in unix_signal_handler (signum=11,
si=0x7f7ac8fff670, uc=0x7f7ac8fff540)
at /home//vpp/src/vlib/unix/main.c:157
#4  
#5  0x7f7ac3f2606d in svs_input_inline (vm=0x7f7b0a413240
, node=0x7f7ac95a3080,
frame=0x7f7aca271d00, fproto=FIB_PROTOCOL_IP4) at
/home//vpp/src/plugins/svs/svs.c:294
#6  0x7f7ac3f262af in svs_input_ip4 (vm=0x7f7b0a413240
, node=0x7f7ac95a3080,
frame=0x7f7aca271d00) at /home//vpp/src/plugins/svs/svs.c:339
#7  0x7f7b0a18eae4 in dispatch_node (vm=0x7f7b0a413240
, node=0x7f7ac95a3080,
type=VLIB_NODE_TYPE_INTERNAL,
dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7f7aca271d00,
last_time_stamp=12587373585573984) at /home//vpp/src/vlib/main.c:989
#8  0x7f7b0a18f09d in dispatch_pending_node (vm=0x7f7b0a413240
,
pending_frame_index=3, last_time_stamp=12587373585573984) at
/home//vpp/src/vlib/main.c:1139
#9  0x7f7b0a190c9c in vlib_main_or_worker_loop (vm=0x7f7b0a413240
, is_main=1)
at /home//vpp/src/vlib/main.c:1555
#10 0x7f7b0a191478 in vlib_main_loop (vm=0x7f7b0a413240 )
at /home//vpp/src/vlib/main.c:1629
#11 0x7f7b0a19204b in vlib_main (vm=0x7f7b0a413240
, input=0x7f7ac8b0)
at /home//vpp/src/vlib/main.c:1820
#12 0x7f7b0a1e94ba in thread0 (arg=140166429749824) at
/home//vpp/src/vlib/unix/main.c:607
#13 0x7f7b098128a4 in clib_calljmp ()
   from 
/home//vpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.18.10
#14 0x7ffcc1a6b2d0 in ?? ()
#15 0x7f7b0a1e996e in vlib_unix_main (argc=2, argv=0x7ffcc1a6c5c8)
at /home//vpp/src/vlib/unix/main.c:676
#16 0x55d570fd6927 in main (argc=2, argv=0x7ffcc1a6c5c8) at
/home//vpp/src/vpp/vnet/main.c:264

Thanks and Regards,

Raj

On Wed, Jan 23, 2019 at 8:05 PM Matus Fabian -X (matfabia - PANTHEON
TECHNOLOGIES at Cisco)  wrote:
>
> Hi,
>
> You should go from nat44-out2in to ip4-policer-classify only if it is 
> configured on given interface (check if sw_if_index0 in nat44-out2in has 
> configured/enabled policer), I think this may be reason of ASSERT.
>
> Matus
>
>
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Raj
> Sent: Wednesday, January 23, 2019 3:05 PM
> To: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Configuring NAT and Policing together
>
> Hi Matus,
>
> Thanks for the code fragment it helped me get a better understanding of what 
> I have to do, and have modified the code. But occasionally VPP hits an ASSERT 
> at:
>
> DBGvpp# 0: /vpp/src/vlib/node_funcs.h:296
> (vlib_node_runtime_get_next_frame) assertion `next_index <
> n->n_next_nodes' fails
> Aborted
>
> The approach I had followed was to get the index of policer classify node and 
> setting that as the next node of 'nat44-out2in'
> ,'nat44-out2in-reass' and 'nat44-out2in-fast'.
>
> This is the partial diff of how we got the index of ip4-policer-classify and 
> setting the next node. (full diff is attached).
>
> --- a/src/plugins/nat/out2in.c
> +++ b/src/plugins/nat/out2in.c
> @@ -1110,6 +1113,15 @@ snat_out2in_node_fn (vlib_main_t * vm,
>
>   proto0 = ip_proto_to_snat_proto (ip0->protocol);
>
> +  ip4_policer_classify_node =
> +vlib_get_node_by_name (vm, (u8 *) "ip4-policer-classify");
> +  if (ip4_policer_classify_node)
> +{
> +  next0 = SNAT_OUT2IN_NEXT_POLICER_CLASSIFY;
> +  vlib_node_add_next (vm, ip4_policer_classify_node->index,
> +  next0);
> +}
> +
>   if (PREDICT_FALSE (proto0 == ~0))
> {
>   if (nat_out2in_sm_unknown_proto (sm, b0, ip0, rx_fib_index0))
>
> I hope the approach followed is the correct one, but I could not figure out 
> why the ASSERT is happening.
>
> Thanks 

Re: [vpp-dev] l2 input/output feature arcs

[Benoit Ganne (bganne) via Lists.Fd.Io]

> With gigabytes of memory used for pools and other datastructs, node
> memory structure overhead is negligible imho.

Fair enough :)

> Check the acl plugin. I have 8 nodes
> and only one instance of actual “code”. Not to say that it works perfect but
> maintenance wise it is quite reasonable.

Agreed, that's what I did too in GBP, and moving helpers to be more generic 
will help that a little bit further.

>> Anyway, I do not have a strong opinion (apart that I do not like (5)),
>> so I'll probably stop here and wait for your decision :)

> #4. :)
> The rationale behind the way i did it the way I did it was to harmonize the
> way the l2 and l3 nodes can work, as much as possible.

Got it, I'll propose a patch this way then.

Best
Ben
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12001): https://lists.fd.io/g/vpp-dev/message/12001
Mute This Topic: https://lists.fd.io/mt/29523811/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-