Re: [vpp-dev] ip4-not-enabled in IP-in-IP tunnel

[Nitin Saxena]

Thanks Ole. Makes sense. Let me try by assigning IP to ipip0

Thanks,
Nitin

On Monday 14 May 2018 06:24 PM, Ole Troan wrote:

Nitin,

A tunnel interface is just like any other interface and you need to have an IP 
address configured on it to make it IP enabled.
(Or point to another interface with IP unnumbered).

Note that the IPIP interface supports {IPvX over IPvY} where X and Y are 4 and 
6. So your patch would blindly enable IPv4, which isn't quite what you want.

Cheers,
Ole


On 14 May 2018, at 14:27, Nitin Saxena <nitin.sax...@cavium.com> wrote:

Hi,

Using VPP v1804 I created IP-in-IP tunnel and ran into IP4-not-enabled issue. 
Following is the trace

===
-- Start of thread 1 vpp_wk_0 ---
Packet 1

00:04:16:407330: dpdk-input
  VirtualFunctionEthernet1/0/1 rx queue 0
  buffer 0x291b: current data 14, length 48, free-list 0, clone-count 0, 
totlen-nifb 0, trace 0x0
 ext-hdr-valid
 l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 
l3-hdr-offset 14
  PKT MBUF: port 0, nb_segs 1, pkt_len 62
buf_len 2176, data_len 62, ol_flags 0x0, data_off 128, phys_addr 0x80148e80
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
  IP4: 90:e2:ba:91:22:04 -> 00:0f:b7:11:8d:da
  IP_IN_IP: 50.0.0.1 -> 60.0.0.1
tos 0x00, ttl 64, length 48, checksum 0x0cc8
fragment id 0x0001
00:04:16:407343: ip4-input
  IP_IN_IP: 50.0.0.1 -> 60.0.0.1
tos 0x00, ttl 64, length 48, checksum 0x0cc8
fragment id 0x0001
00:04:16:407350: ip4-lookup
  fib 0 dpo-idx 7 flow hash: 0x
  IP_IN_IP: 50.0.0.1 -> 60.0.0.1
tos 0x00, ttl 64, length 48, checksum 0x0cc8
fragment id 0x0001
00:04:16:407354: ip4-local
IP_IN_IP: 50.0.0.1 -> 60.0.0.1
  tos 0x00, ttl 64, length 48, checksum 0x0cc8
  fragment id 0x0001
00:04:16:407356: ipip4-input
  IPIP: tunnel 3 len 48 src 50.0.0.1 dst 60.0.0.1
00:04:16:407358: ip4-input
  UDP: 20.0.0.3 -> 30.0.0.2
tos 0x00, ttl 64, length 28, checksum 0x48cc
fragment id 0x0001
  UDP: 53 -> 53
length 8, checksum 0xcd6f
00:04:16:407359: ip4-not-enabled
UDP: 20.0.0.3 -> 30.0.0.2
  tos 0x00, ttl 64, length 28, checksum 0x48cc
  fragment id 0x0001
UDP: 53 -> 53
  length 8, checksum 0xcd6f
00:04:16:407365: error-drop
===

However we are able to fix above issue by following patch:

diff --git a/src/vnet/ipip/ipip.c b/src/vnet/ipip/ipip.c
index 82c961c..d3bf9d9 100644
--- a/src/vnet/ipip/ipip.c
+++ b/src/vnet/ipip/ipip.c
@@ -468,6 +468,8 @@ ipip_add_tunnel (ipip_transport_t transport,
   t->fib_index = fib_index;
   t->sw_if_index = sw_if_index;

+  ip4_sw_interface_enable_disable (sw_if_index, 1);
+
   t->transport = transport;
   vec_validate_init_empty (gm->tunnel_index_by_sw_if_index, sw_if_index, ~0);
   gm->tunnel_index_by_sw_if_index[sw_if_index] = t_idx;
@@ -529,6 +531,7 @@ ipip_del_tunnel (u32 sw_if_index)
   if (t == NULL)
 return VNET_API_ERROR_NO_SUCH_ENTRY;

+  ip4_sw_interface_enable_disable (sw_if_index, 0);
   vnet_sw_interface_set_flags (vnm, sw_if_index, 0 /* down */ );
   gm->tunnel_index_by_sw_if_index[sw_if_index] = ~0;
   vnet_delete_hw_interface (vnm, t->hw_if_index);

Are we missing anything? Any comments will be appreciated.

Thanks,
Nitin










-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9273): https://lists.fd.io/g/vpp-dev/message/9273
View All Messages In Topic (3): https://lists.fd.io/g/vpp-dev/topic/19199319
Mute This Topic: https://lists.fd.io/mt/19199319/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] ip4-not-enabled in IP-in-IP tunnel

[Nitin Saxena]

Hi,

Using VPP v1804 I created IP-in-IP tunnel and ran into IP4-not-enabled 
issue. Following is the trace


===
-- Start of thread 1 vpp_wk_0 ---
Packet 1

00:04:16:407330: dpdk-input
  VirtualFunctionEthernet1/0/1 rx queue 0
  buffer 0x291b: current data 14, length 48, free-list 0, clone-count 
0, totlen-nifb 0, trace 0x0

 ext-hdr-valid
 l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 
l3-hdr-offset 14

  PKT MBUF: port 0, nb_segs 1, pkt_len 62
buf_len 2176, data_len 62, ol_flags 0x0, data_off 128, phys_addr 
0x80148e80

packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
  IP4: 90:e2:ba:91:22:04 -> 00:0f:b7:11:8d:da
  IP_IN_IP: 50.0.0.1 -> 60.0.0.1
tos 0x00, ttl 64, length 48, checksum 0x0cc8
fragment id 0x0001
00:04:16:407343: ip4-input
  IP_IN_IP: 50.0.0.1 -> 60.0.0.1
tos 0x00, ttl 64, length 48, checksum 0x0cc8
fragment id 0x0001
00:04:16:407350: ip4-lookup
  fib 0 dpo-idx 7 flow hash: 0x
  IP_IN_IP: 50.0.0.1 -> 60.0.0.1
tos 0x00, ttl 64, length 48, checksum 0x0cc8
fragment id 0x0001
00:04:16:407354: ip4-local
IP_IN_IP: 50.0.0.1 -> 60.0.0.1
  tos 0x00, ttl 64, length 48, checksum 0x0cc8
  fragment id 0x0001
00:04:16:407356: ipip4-input
  IPIP: tunnel 3 len 48 src 50.0.0.1 dst 60.0.0.1
00:04:16:407358: ip4-input
  UDP: 20.0.0.3 -> 30.0.0.2
tos 0x00, ttl 64, length 28, checksum 0x48cc
fragment id 0x0001
  UDP: 53 -> 53
length 8, checksum 0xcd6f
00:04:16:407359: ip4-not-enabled
UDP: 20.0.0.3 -> 30.0.0.2
  tos 0x00, ttl 64, length 28, checksum 0x48cc
  fragment id 0x0001
UDP: 53 -> 53
  length 8, checksum 0xcd6f
00:04:16:407365: error-drop
===

However we are able to fix above issue by following patch:

diff --git a/src/vnet/ipip/ipip.c b/src/vnet/ipip/ipip.c
index 82c961c..d3bf9d9 100644
--- a/src/vnet/ipip/ipip.c
+++ b/src/vnet/ipip/ipip.c
@@ -468,6 +468,8 @@ ipip_add_tunnel (ipip_transport_t transport,
   t->fib_index = fib_index;
   t->sw_if_index = sw_if_index;

+  ip4_sw_interface_enable_disable (sw_if_index, 1);
+
   t->transport = transport;
   vec_validate_init_empty (gm->tunnel_index_by_sw_if_index, 
sw_if_index, ~0);

   gm->tunnel_index_by_sw_if_index[sw_if_index] = t_idx;
@@ -529,6 +531,7 @@ ipip_del_tunnel (u32 sw_if_index)
   if (t == NULL)
 return VNET_API_ERROR_NO_SUCH_ENTRY;

+  ip4_sw_interface_enable_disable (sw_if_index, 0);
   vnet_sw_interface_set_flags (vnm, sw_if_index, 0 /* down */ );
   gm->tunnel_index_by_sw_if_index[sw_if_index] = ~0;
   vnet_delete_hw_interface (vnm, t->hw_if_index);

Are we missing anything? Any comments will be appreciated.

Thanks,
Nitin






-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9271): https://lists.fd.io/g/vpp-dev/message/9271
View All Messages In Topic (1): https://lists.fd.io/g/vpp-dev/topic/19199319
Mute This Topic: https://lists.fd.io/mt/19199319/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] MTU in vpp

[Nitin Saxena]

Hi,

If dpdk plugin used, which is the default case in VPP, MTU to each 
interface is set by retrieving MTU information from underlying DPDK PMD 
API: rte_eth_dev_info_get(). So if the device supports max MTU say 9216 
then that will be configured.


I don't think setting MTU via vpp cli or startup.config is supported. 
Although it should be easy to implement by adding dpdk plugin cli by 
calling rte_eth_dev_set_mtu().


Thanks,
Nitin

On Tuesday 08 May 2018 05:54 PM, Rubina Bianchi wrote:

Hi

I set mtu to all of my interfaces using "vppctl set interface mtu 1500 
".
But when I generate packets larger than mtu size, all packet pass 
through vpp. Is there any parameter that I must set?


I generate large packet using "ping -M do -s 9472 200.200.200.40".

My vpp version: master v18.07-rc0~147-g7220f42c

Thanks,
Sincerely

Sent from Outlook 



-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9215): https://lists.fd.io/g/vpp-dev/message/9215
View All Messages In Topic (3): https://lists.fd.io/g/vpp-dev/topic/18847572
Mute This Topic: https://lists.fd.io/mt/18847572/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Packet tx functions via DPDK

[Nitin Saxena]

Hi Prashant,

Hope you are doing fine.

Regarding your question, I am not able to see macswap plugin in current 
master branch but I will try to explain wrt dpdk_plugin:


With respect to low level device each VPP device driver registers for

1) INPUT_NODE (For Rx) VLIB_REGISTER_NODE (This you already figured out)
2) And Tx function via VNET_DEVICE_CLASS (), Device class like "dpdk"
There are couple of more function pointers registered but let stick to 
Rx/Tx part.


As part of startup low level plugin/driver calls 
ethernet_register_interface() which in turn calls 
vnet_register_interface().


vnet_register_interface:
For a particular interface like Intel 40G, init time interface node is 
created and the tx function of that node is copied from 
VNET_DEVICE_CLASS{.tx_function="}. node->tx and node->output 
functions are properly initialized and node is registered.


VPP stack sends packet to this low level Tx node via sw_if_index. I am 
guessing sw_if_index is determined by IPv4 routing or L2 switching.


I think vnet_set_interface_output_node() is called for those interface 
(Tx path) whose DEVICE_CLASS do not provide tx_functions but I am not sure.


"show vlib graph" will tell you how nodes are arranged in vpp graph.

To be specific for your question

  next0 = hi0->output_node_next_index;

output_node_next_index is the index of next node at which the current 
vector has to be copied. (Transition from one node to another along the 
graph)


Note: All this I got through browsing code and if this information is 
not correct, I request VPP experts to correct it.


Thanks,
Nitin

On Thursday 10 May 2018 02:19 PM, Prashant Upadhyaya wrote:

Hi,

I am trying to walk throught the code to see how the packet arrives
into the system at dpdk rx side and finally leaves it at the dpdk tx
side. I am using the context of the macswap sample plugin for this.

It is clear to me that dpdk-input is a graph node and it is an 'input'
type graph node so it polls for the packets using dpdk functions. The
frame is then eventually passed to the sample plugin because the
sample plugin inserts itself at the right place. The sample plugin
queues the packets to the interface-output graph node.

So now I check the interface-output graph node function.
I locate that in vpp/src/vnet/interface_output.c
So the dispatch function for the graph node is vnet_per_buffer_interface_output
Here the interface-output node is queueing the packets to a next node
based on the following code --

  hi0 =
 vnet_get_sup_hw_interface (vnm,
vnet_buffer (b0)->sw_if_index
[VLIB_TX]);

   next0 = hi0->output_node_next_index;

Now I am a little lost, what is this output_node_next_index ? Which
graph node function is really called for really emitting the packet ?
Where exactly is this setup ?

I do see that the actual dpdk tx burst function is called from
tx_burst_vector_internal, which itself is called from
dpdk_interface_tx (vpp/src/plugins/dpdk/device/device.c). But how the
code reaches the dpdk_interface_tx after the packets are queued from
interface-output graph node is not clear to me. If somebody could help
me connect the dots, that would be great.

Regards
-Prashant





-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9260): https://lists.fd.io/g/vpp-dev/message/9260
View All Messages In Topic (2): https://lists.fd.io/g/vpp-dev/topic/19023164
Mute This Topic: https://lists.fd.io/mt/19023164/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] Marvell's AI from last call

[Nitin Saxena]

Hi Damjan,

This request was discussed in FD.io ARM call in order to optimize 
dpdk-input node for ARM platforms. Macchiatobin is one common platform 
where almost all ARM stakeholders are working on and profiling 
data/details can be easily shared among them.


Thanks,
Nitin

On Wednesday 16 May 2018 11:37 PM, Damjan Marion wrote:


Hi Natalie,

Who requested that?

Thanks,

--
Damjan

On 16 May 2018, at 15:33, Natalie Samsonov > wrote:


Dear Damjan,
As I understood, it was a request to use DPDK plugin with all 
platforms and I was asked to upstream Marvell DPDK plugin.

Best Regards,
Natalie
*From:*Damjan Marion [mailto:dmarion.li...@gmail.com]
*Sent:*Wednesday, May 16, 2018 16:03
*To:*Natalie Samsonov >
*Cc:*Tina Tsou >;vpp-dev@lists.fd.io 
; Maen Suleiman >

*Subject:*[EXT] Re: [vpp-dev] Marvell's AI from last call
External Email

Dear Natalie,
What is the point of using DPDK marvel driver (which is actually just 
a wrapper around musdk) if we already have native musdk support in vpp?

--
Damjan


On 15 May 2018, at 08:48, Natalie Samsonov > wrote:
Hi,
1.We are working now on upstreaming Marvell’s PMD support in VPP
DPDK plugin.
I’m attaching the patch as well in case you want play with it.
2.Regarding the L3FWD performance difference between MUSDK and
DPDK, currently we don’t have numbers with VPP L3FW.
We only have  the numbers with two different applications. One is
our own L3FW application running on top of MUSDK and other is a
standard DPDL L3FWD application, so it’s not a fair comparison.
Best Regards,
Natalie Samsonov
<0001-plugins-dpdk-Add-support-for-net_mrvl-dpdk-driver.patch>





-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9313): https://lists.fd.io/g/vpp-dev/message/9313
View All Messages In Topic (2): https://lists.fd.io/g/vpp-dev/topic/19249841
Mute This Topic: https://lists.fd.io/mt/19249841/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Enabling DPDK OCTEONTx mempool ops in VPP

[Nitin Saxena]

Hi Damjan,

Thanks for the response. We are able to get pass few hurdles in enabling 
OCTEONTx mempool ops, I may have follow up questions on this after further 
testing. Appreciate your help

Regards,
Nitin

> On 07-Jun-2018, at 20:03, Damjan Marion  wrote:
> 
> 
> Dear Nitin,
> 
> VPP expect that buffer memory is in contiguous shareable VA space represented 
> by single mmap()-able file descriptor
> and VPP needs to know that file descriptor.
> 
> Regards,
> 
> Damjan
> 
>> On 6 Jun 2018, at 11:04, Nitin Saxena  wrote:
>> 
>> Hi Damjan,
>> 
>> I want separate email-thread for the OCTEONTx mempool ops.
>> 
>>> Regarding the octeon tx mempool, no idea what it is,
>> OCTEONTx mempool ops uses hardware mempool allocator instead of DPDK 
>> software mempool. This is not enabled by default in DPDK but once enabled 
>> all buffers are managed by OCTEONTx hardware allocator.
>> 
>>> VPP expects that buffer memory is allocated by VPP and then given to
>>> DPDK via rte_mempool_create_empty() and rte_mempool_populate_iova_tab().
>> This I understood that VPP calls rte_mempool_populate_iova_tab() but I want 
>> to know why VPP allocates buffer on its own and not allow DPDK to create its 
>> own buffers? Why VPP cannot call rte_pktmbuf_pool_create()? Is there a 
>> limitation?
>> 
>> Thanks,
>> Nitin
>> 
>>> On Wednesday 06 June 2018 02:20 PM, Damjan Marion wrote:
>>> Regarding the octeon tx mempool, no idea what it is, but will not be
>>> surprised that it is not compatible with the way how we use buffer
>>> memory in vpp.
>>> VPP expects that buffer memory is allocated by VPP and then given to
>>> DPDK via rte_mempool_create_empty() and rte_mempool_populate_iova_tab().
>> 
>> 
>> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9561): https://lists.fd.io/g/vpp-dev/message/9561
View All Messages In Topic (3): https://lists.fd.io/g/vpp-dev/topic/21380227
Mute This Topic: https://lists.fd.io/mt/21380227/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
Email sent to: arch...@mail-archive.com
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Multiarch/target select for dpdk_device_input

[Nitin Saxena]

Hi Damjan,

Answers inline.

Thanks,
Nitin

> On 01-Jun-2018, at 12:15 AM, Damjan Marion  wrote:
> 
> 
> Dear Nitin,
> 
> See inline….
> 
> 
>> On 31 May 2018, at 19:59, Nitin Saxena  wrote:
>> 
>> Hi,
>> 
>> I am working on optimising dpdk-input node (based on vpp v1804) for our 
>> target. I am able to get performance improvements on our target but the 
>> problem I am finding now are:
>> 
>> 1) The dpdk-input code is completely changed on master branch from v1804.
> 
> Why is this a problem? It was done with reason and for tangible benefit.
This is a problem for me as I can not apply my v1804 changes directly to the 
master branch. I have to again rework on master branch and that’s why I am not 
able to move to master branch or v1807 in future. 
> 
>> Not to mention the dpdk-input master branch code do not give better numbers 
>> on our target as compared to v1804
> 
> Sad to hear that, good thing is, it gives better numbers on x86.
As I understand one dpdk_device_input function cannot be same for all 
architectures because if the underlying micro-architecture is different, the 
hot spots changes. I have seen dpdk-input master branch changes and on a 
positive notes those changes make sense however some codes are tuned for x86 
specially Skylake. I was looking for some kind of  way to have mutiarch select 
function for the Rx path, like the way it’s done for tx path.
> 
>> 2) I don’t know the modular approach I should follow to merge my changes as 
>> I have completely changed the quad loop handling and the prefetches order in 
>> dpdk-input.
> 
> I carefully tuned that code. It was multi day exercise and losing single 
> clock/packet on x86 with additional modifications are not acceptable. Still 
> I’m open for discussion how to address this problem.
> 
>> 
>> Note: I am far away from upstreaming the code currently as my optimisation 
>> is still in progress. It will be better if I know the proper way of doing it.
> 
> I suggest that you don’t even start on working on upstreaming before we have 
> deep understanding of what and why needs to be done and we are all in 
> agreement.
> 
>> 
>> Thanks,
>> Nitin
>> 
>> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9484): https://lists.fd.io/g/vpp-dev/message/9484
View All Messages In Topic (3): https://lists.fd.io/g/vpp-dev/topic/20748102
Mute This Topic: https://lists.fd.io/mt/20748102/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
Email sent to: arch...@mail-archive.com
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Multiarch/target select for dpdk_device_input

[Nitin Saxena]

Hi,

I am working on optimising dpdk-input node (based on vpp v1804) for our target. 
I am able to get performance improvements on our target but the problem I am 
finding now are:

1) The dpdk-input code is completely changed on master branch from v1804. Not 
to mention the dpdk-input master branch code do not give better numbers on our 
target as compared to v1804
2) I don’t know the modular approach I should follow to merge my changes as I 
have completely changed the quad loop handling and the prefetches order in 
dpdk-input.

Note: I am far away from upstreaming the code currently as my optimisation is 
still in progress. It will be better if I know the proper way of doing it.

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9480): https://lists.fd.io/g/vpp-dev/message/9480
View All Messages In Topic (1): https://lists.fd.io/g/vpp-dev/topic/20748102
Mute This Topic: https://lists.fd.io/mt/20748102/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
Email sent to: arch...@mail-archive.com
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Multiarch/target select for dpdk_device_input

[Nitin Saxena]

Hi Damjan,

If VPP is an open-source project that supports multiple architectures, then 
there should be a review of every commit which provides others using the open 
source project an opportunity to raise their concerns. So my request is to post 
changes for review before they are committed to ensure VPP stays true to 
open-source philosophy. Please let me know if this is possible. If not, i'd 
like to understand the reasons for it.

Regards,
Nitin

On 02-Jun-2018, at 00:17, Damjan Marion mailto:dmar...@me.com>> 
wrote:


Dear Nitin,

That doesn't work that way.

Regards,

Damjan

On 1 Jun 2018, at 19:41, Saxena, Nitin 
mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

 Now that you are aware that Cavium is working on optimisations for ARM, can I 
request that you check with us on implications for ARM(at least Cavium), before 
bringing changes in dpdk-input?

Regards,
Nitin

On 01-Jun-2018, at 21:39, Damjan Marion mailto:dmar...@me.com>> 
wrote:


Dear Nitin,

I really don't have anything else to add. It your call how do you want to 
proceed

Regards,

Damjan

On 1 Jun 2018, at 18:02, Nitin Saxena 
mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

Answers Inline.

Thanks,
Nitin

On Friday 01 June 2018 08:49 PM, Damjan Marion wrote:
Hi Nitin,
inline...
On 1 Jun 2018, at 15:23, Nitin Saxena 
mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

It was hard to know that you have subset of patches hidden somewhere.
I wouldn't say patches are hidden. We are trying to fine tune dpdk-input 
initially from our end first and later we will seek your expertise while 
upstreaming.
for me they were hidden.
Typically it makes sense to discuss such kind of changes with person >who 
"maintains" the code before starting writing the code.
Agreed. However we prefer to do internal analysis/POC first before reaching out 
to MAINTAINERS. That way we can better understand code review comments.
Perfectly fine, but then don't put blame on us for not knowing that you are 
doing something internally...
The intention was not to blame anybody but to understand modular approach in 
vpp to accommodate multi-arch(s).

Maybe, but sounds to me like we are still in guessing phase.
I wouldn't do any guess work with MAINTAINERS.

Maybe we even need different function for each ARM CPU core as they
maybe have different memory subsystem and pipeline
This is what I am looking for. Is it ok to detect our hardware natively from 
autoconf and append target specific macro to CFLAGS? And then separate function 
for our target in dpdk/device/node.c? Sorry my multi-arch select example was 
incorrect and that's not what I am looking at.
Here I will be able to help when I get reasonable understanding what is the 
"big" plan.
The "Big" plan is to optimize each vpp node for Aarch64. For now focus is 
dpdk-input.
I don't want that we end up in 6 months with cavium patches, nxp patches, 
marvell patches, and so on.
Is it a problem? If yes than I am not able to visualize it as the same problem 
would exist for any architecture and not just for Aarch64.

Is there an agreement between ARM vendors what is the targeted core
you want to have code tuned for or you are simply tuning to whatever
core Cavium uses?
I am trying to optimize Cavium's SOC. This question is in this regard only. 
However efforts are going on optimizing Cortex cores as well by ARM community.
What about agreeing on plan for optimising on all ARM cores, and then starting 
doing optimisation?
This is cross-company question so hard to answer but Cavium has the "big" plan 
described above.

Thanks,
Nitin

On Friday 01 June 2018 01:55 AM, Damjan Marion wrote:
inline...
--
Damjan
On 31 May 2018, at 21:10, Saxena, Nitin 
mailto:nitin.sax...@cavium.com> 
<mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

Answers inline.

Thanks,
Nitin

On 01-Jun-2018, at 12:15 AM, Damjan Marion 
mailto:dmarion.li...@gmail.com> 
<mailto:dmarion.li...@gmail.com>> wrote:


Dear Nitin,

See inline….


On 31 May 2018, at 19:59, Nitin Saxena 
mailto:nitin.sax...@cavium.com> 
<mailto:nitin.sax...@cavium.com>> wrote:

Hi,

I am working on optimising dpdk-input node (based on vpp v1804) for our target. 
I am able to get performance improvements on our target but the problem I am 
finding now are:

1) The dpdk-input code is completely changed on master branch from v1804.

Why is this a problem? It was done with reason and for tangible benefit.
This is a problem for me as I can not apply my v1804 changes directly to the 
master branch. I have to again rework on master branch and that’s why I am not 
able to move to master branch or v1807 in future.
It was hard to know that you have subset of patches hidden somewhere. Typically 
it makes sense to discuss such kind of changes with person who "maintains" the 
code before starting writing the code.

Not to mention the dpdk-input master branc

Re: [vpp-dev] Multiarch/target select for dpdk_device_input

[Nitin Saxena]

Hi Damjan,

> It was hard to know that you have subset of patches hidden somewhere.
I wouldn't say patches are hidden. We are trying to fine tune dpdk-input 
initially from our end first and later we will seek your expertise while 
upstreaming.


> Typically it makes sense to discuss such kind of changes with person 
>who "maintains" the code before starting writing the code.
Agreed. However we prefer to do internal analysis/POC first before 
reaching out to MAINTAINERS. That way we can better understand code 
review comments.


> Maybe, but sounds to me like we are still in guessing phase.
I wouldn't do any guess work with MAINTAINERS.

> Maybe we even need different function for each ARM CPU core as they
> maybe have different memory subsystem and pipeline
This is what I am looking for. Is it ok to detect our hardware natively 
from autoconf and append target specific macro to CFLAGS? And then 
separate function for our target in dpdk/device/node.c? Sorry my 
multi-arch select example was incorrect and that's not what I am looking at.


> Is there an agreement between ARM vendors what is the targeted core
> you want to have code tuned for or you are simply tuning to whatever
> core Cavium uses?
I am trying to optimize Cavium's SOC. This question is in this regard 
only. However efforts are going on optimizing Cortex cores as well by 
ARM community.


Thanks,
Nitin

On Friday 01 June 2018 01:55 AM, Damjan Marion wrote:

inline...
--
Damjan

On 31 May 2018, at 21:10, Saxena, Nitin <mailto:nitin.sax...@cavium.com>> wrote:


Hi Damjan,

Answers inline.

Thanks,
Nitin

On 01-Jun-2018, at 12:15 AM, Damjan Marion <mailto:dmarion.li...@gmail.com>> wrote:



Dear Nitin,

See inline….


On 31 May 2018, at 19:59, Nitin Saxena <mailto:nitin.sax...@cavium.com>> wrote:


Hi,

I am working on optimising dpdk-input node (based on vpp v1804) for 
our target. I am able to get performance improvements on our target 
but the problem I am finding now are:


1) The dpdk-input code is completely changed on master branch from 
v1804.


Why is this a problem? It was done with reason and for tangible benefit.
This is a problem for me as I can not apply my v1804 changes directly 
to the master branch. I have to again rework on master branch and 
that’s why I am not able to move to master branch or v1807 in future.


It was hard to know that you have subset of patches hidden somewhere. 
Typically it makes sense to discuss such kind of changes with person who 
"maintains" the code before starting writing the code.




Not to mention the dpdk-input master branch code do not give better 
numbers on our target as compared to v1804


Sad to hear that, good thing is, it gives better numbers on x86.
As I understand one dpdk_device_input function cannot be same for all 
architectures because if the underlying micro-architecture is 
different, the hot spots changes.


Maybe, but sounds to me like we are still in guessing phase.
Maybe we even need different function for each ARM CPU core as they 
maybe have different memory subsystem and pipeline


Is there an agreement between ARM vendors what is the targeted core you 
want to have code tuned for or you are simply tuning to whatever core 
Cavium uses?



I have seen dpdk-input master branch changes and on a positive notes 
those changes make sense however some codes are tuned for x86 
specially Skylake. I was looking for some kind of  way to have 
mutiarch select function for the Rx path, like the way it’s done for 
tx path.


Not sure why do you need that, unless you are going to have code 
optimised for different CPU variants (i.e. Cortex-A53 and Cortex-A72) in 
the same binary.




2) I don’t know the modular approach I should follow to merge my 
changes as I have completely changed the quad loop handling and the 
prefetches order in dpdk-input.


I carefully tuned that code. It was multi day exercise and losing 
single clock/packet on x86 with additional modifications are not 
acceptable. Still I’m open for discussion how to address this problem.




Note: I am far away from upstreaming the code currently as my 
optimisation is still in progress. It will be better if I know the 
proper way of doing it.


I suggest that you don’t even start on working on upstreaming before 
we have deep understanding of what and why needs to be done and we 
are all in agreement.




Thanks,
Nitin





-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9492): https://lists.fd.io/g/vpp-dev/message/9492
View All Messages In Topic (5): https://lists.fd.io/g/vpp-dev/topic/20748102
Mute This Topic: https://lists.fd.io/mt/20748102/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/stati

[vpp-dev] Enabling DPDK OCTEONTx mempool ops in VPP

[Nitin Saxena]

Hi Damjan,

I want separate email-thread for the OCTEONTx mempool ops.

> Regarding the octeon tx mempool, no idea what it is,
OCTEONTx mempool ops uses hardware mempool allocator instead of DPDK 
software mempool. This is not enabled by default in DPDK but once 
enabled all buffers are managed by OCTEONTx hardware allocator.


> VPP expects that buffer memory is allocated by VPP and then given to
> DPDK via rte_mempool_create_empty() and rte_mempool_populate_iova_tab().
This I understood that VPP calls rte_mempool_populate_iova_tab() but I 
want to know why VPP allocates buffer on its own and not allow DPDK to 
create its own buffers? Why VPP cannot call rte_pktmbuf_pool_create()? 
Is there a limitation?


Thanks,
Nitin

On Wednesday 06 June 2018 02:20 PM, Damjan Marion wrote:
> Regarding the octeon tx mempool, no idea what it is, but will not be
> surprised that it is not compatible with the way how we use buffer
> memory in vpp.
> VPP expects that buffer memory is allocated by VPP and then given to
> DPDK via rte_mempool_create_empty() and rte_mempool_populate_iova_tab().

-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9542): https://lists.fd.io/g/vpp-dev/message/9542
View All Messages In Topic (1): https://lists.fd.io/g/vpp-dev/topic/21380227
Mute This Topic: https://lists.fd.io/mt/21380227/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
Email sent to: arch...@mail-archive.com
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP Vnet crash with vhost-user interface

[Nitin Saxena]

Hi Ravi,

Sorry for diluting your topic. From your stack trace and show interface output 
I thought you are using OCTEONTx. 

Regards,
Nitin

> On 06-Jun-2018, at 22:10, Ravi Kerur  wrote:
> 
> Steven, Damjan, Nitin,
> 
> Let me clarify so there is no confusion, since you are assisting me to
> get this working I will make sure we are all on same page. I believe
> OcteonTx is related to Cavium/ARM and I am not using it.
> 
> DPDK/testpmd (vhost-virtio) works with both 2MB and 1GB hugepages. For
> 2MB I had to use '--single-file-segments' option.
> 
> There used to be a way in DPDK to influence compiler to compile for
> certain architecture f.e. 'nehalem'. I will try that option but I want
> to make sure steps I am executing is fine first.
> 
> (1) I compile VPP (18.04) code on x86_64 system with following
> CPUFLAGS. My system has 'avx, avx2, sse3, see4_2' for SIMD.
> 
> fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
> pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
> pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
> xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor
> ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1
> sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c
> rdrand lahf_lm abm epb invpcid_single retpoline kaiser tpr_shadow vnmi
> flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms
> invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts
> 
> (2) I run VPP on the same system.
> 
> (3) VPP on host has following startup.conf
> unix {
>  nodaemon
>  log /var/log/vpp/vpp.log
>  full-coredump
>  cli-listen /run/vpp/cli.sock
>  gid vpp
> }
> 
> api-trace {
>  on
> }
> 
> api-segment {
>  gid vpp
> }
> 
> dpdk {
>  no-pci
> 
>  vdev net_vhost0,iface=/var/run/vpp/sock1.sock
>  vdev net_vhost1,iface=/var/run/vpp/sock2.sock
> 
>  huge-dir /dev/hugepages_1G
>  socket-mem 2,0
> }
> 
> (4) VPP vhost-user config (on host)
> create vhost socket /var/run/vpp/sock3.sock
> set interface state VirtualEthernet0/0/0 up
> set interface ip address VirtualEthernet0/0/0 10.1.1.1/24
> 
> (5) show dpdk version (Version is the same on host and container, EAL
> params are different)
> DPDK Version: DPDK 18.02.1
> DPDK EAL init args:   -c 1 -n 4 --no-pci --vdev
> net_vhost0,iface=/var/run/vpp/sock1.sock --vdev
> net_vhost1,iface=/var/run/vpp/sock2.sock --huge-dir /dev/hugepages_1G
> --master-lcore 0 --socket-mem 2,0
> 
> (6) Container is instantiated as follows
> docker run -it --privileged -v
> /var/run/vpp/sock3.sock:/var/run/usvhost1 -v
> /dev/hugepages_1G:/dev/hugepages_1G dpdk-app-vpp:latest
> 
> (6) VPP startup.conf inside container is as follows
> unix {
>  nodaemon
>  log /var/log/vpp/vpp.log
>  full-coredump
>  cli-listen /run/vpp/cli.sock
>  gid vpp
> }
> 
> api-trace {
>  on
> }
> 
> api-segment {
>  gid vpp
> }
> 
> dpdk {
>  no-pci
>  huge-dir /dev/hugepages_1G
>  socket-mem 1,0
>  vdev virtio_user0,path=/var/run/usvhost1
> }
> 
> (7) VPP virtio-user config (on container)
> set interface state VirtioUser0/0/0  up
> set interface ip address VirtioUser0/0/0 10.1.1.2/24
> 
> (8) Ping... VP on host crashes. I sent one backtrace yesterday. Today
> morning tried again, no backtrace but following messages
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x7fd6f2ba3070 in dpdk_input_avx2 () from
> target:/usr/lib/vpp_plugins/dpdk_plugin.so
> (gdb)
> Continuing.
> 
> Program received signal SIGABRT, Aborted.
> 0x7fd734860428 in raise () from target:/lib/x86_64-linux-gnu/libc.so.6
> (gdb)
> Continuing.
> 
> Program terminated with signal SIGABRT, Aborted.
> The program no longer exists.
> (gdb) bt
> No stack.
> (gdb)
> 
> Thanks.
> 
>> On Wed, Jun 6, 2018 at 1:50 AM, Damjan Marion  wrote:
>> 
>> Now i'm completely confused, is this on x86 or octeon tx?
>> 
>> Regarding the octeon tx mempool, no idea what it is, but will not be
>> surprised that it is not compatible with the way how we use buffer memory in
>> vpp.
>> VPP expects that buffer memory is allocated by VPP and then given to DPDK
>> via rte_mempool_create_empty() and rte_mempool_populate_iova_tab().
>> 
>> On 6 Jun 2018, at 06:51, Saxena, Nitin  wrote:
>> 
>> Hi Ravi,
>> 
>> Two things to get vhost-user running on OCTEONTx
>> 
>> 1) use either 1 GB hugepages or 512 MB. This you did.
>> 
>> 2) You need one dpdk patch that I merged in dpdk-18.05 related to OcteonTx
>> MTU. You can get patch from dpdk git (search for nsaxena)
>> 
>> Hi damjan,
>> 
>> Currently we don't support Octeon TX mempool. Are you intentionally using
>> it?
>> 
>> I was about to send email regarding OCTEONTX mempool, as we enabled it and
>> running into issuea. Any pointers will be helpful as I didn't reach to the
>> root cause of the issue
>> 
>> Thanks,
>> Nitin
>> 
>> On 06-Jun-2018, at 01:40, Damjan Marion  wrote:
>> 
>> Dear Ravi,
>> 
>> Currently we don't support Octeon TX mempool. Are you intentionally using
>> it?
>> 

Re: [vpp-dev] Multiarch/target select for dpdk_device_input

[Nitin Saxena]

Hi Damjan,

Answers Inline.

Thanks,
Nitin

On Friday 01 June 2018 08:49 PM, Damjan Marion wrote:

Hi Nitin,

inline...



On 1 Jun 2018, at 15:23, Nitin Saxena  wrote:

Hi Damjan,


It was hard to know that you have subset of patches hidden somewhere.

I wouldn't say patches are hidden. We are trying to fine tune dpdk-input 
initially from our end first and later we will seek your expertise while 
upstreaming.


for me they were hidden.


Typically it makes sense to discuss such kind of changes with person >who 
"maintains" the code before starting writing the code.

Agreed. However we prefer to do internal analysis/POC first before reaching out 
to MAINTAINERS. That way we can better understand code review comments.


Perfectly fine, but then don't put blame on us for not knowing that you are 
doing something internally...
The intention was not to blame anybody but to understand modular 
approach in vpp to accommodate multi-arch(s).





Maybe, but sounds to me like we are still in guessing phase.

I wouldn't do any guess work with MAINTAINERS.


Maybe we even need different function for each ARM CPU core as they
maybe have different memory subsystem and pipeline

This is what I am looking for. Is it ok to detect our hardware natively from 
autoconf and append target specific macro to CFLAGS? And then separate function 
for our target in dpdk/device/node.c? Sorry my multi-arch select example was 
incorrect and that's not what I am looking at.


Here I will be able to help when I get reasonable understanding what is the 
"big" plan.
The "Big" plan is to optimize each vpp node for Aarch64. For now focus 
is dpdk-input.

I don't want that we end up in 6 months with cavium patches, nxp patches, 
marvell patches, and so on.
Is it a problem? If yes than I am not able to visualize it as the same 
problem would exist for any architecture and not just for Aarch64.





Is there an agreement between ARM vendors what is the targeted core
you want to have code tuned for or you are simply tuning to whatever
core Cavium uses?

I am trying to optimize Cavium's SOC. This question is in this regard only. 
However efforts are going on optimizing Cortex cores as well by ARM community.


What about agreeing on plan for optimising on all ARM cores, and then starting 
doing optimisation?
This is cross-company question so hard to answer but Cavium has the 
"big" plan described above.




Thanks,
Nitin

On Friday 01 June 2018 01:55 AM, Damjan Marion wrote:

inline...
--
Damjan

On 31 May 2018, at 21:10, Saxena, Nitin mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

Answers inline.

Thanks,
Nitin


On 01-Jun-2018, at 12:15 AM, Damjan Marion mailto:dmarion.li...@gmail.com>> wrote:


Dear Nitin,

See inline….



On 31 May 2018, at 19:59, Nitin Saxena mailto:nitin.sax...@cavium.com>> wrote:

Hi,

I am working on optimising dpdk-input node (based on vpp v1804) for our target. 
I am able to get performance improvements on our target but the problem I am 
finding now are:

1) The dpdk-input code is completely changed on master branch from v1804.


Why is this a problem? It was done with reason and for tangible benefit.

This is a problem for me as I can not apply my v1804 changes directly to the 
master branch. I have to again rework on master branch and that’s why I am not 
able to move to master branch or v1807 in future.

It was hard to know that you have subset of patches hidden somewhere. Typically it makes 
sense to discuss such kind of changes with person who "maintains" the code 
before starting writing the code.



Not to mention the dpdk-input master branch code do not give better numbers on 
our target as compared to v1804


Sad to hear that, good thing is, it gives better numbers on x86.

As I understand one dpdk_device_input function cannot be same for all 
architectures because if the underlying micro-architecture is different, the 
hot spots changes.

Maybe, but sounds to me like we are still in guessing phase.
Maybe we even need different function for each ARM CPU core as they maybe have 
different memory subsystem and pipeline
Is there an agreement between ARM vendors what is the targeted core you want to 
have code tuned for or you are simply tuning to whatever core Cavium uses?

I have seen dpdk-input master branch changes and on a positive notes those 
changes make sense however some codes are tuned for x86 specially Skylake. I 
was looking for some kind of  way to have mutiarch select function for the Rx 
path, like the way it’s done for tx path.

Not sure why do you need that, unless you are going to have code optimised for 
different CPU variants (i.e. Cortex-A53 and Cortex-A72) in the same binary.



2) I don’t know the modular approach I should follow to merge my changes as I 
have completely changed the quad loop handling and the prefetches order in 
dpdk-input.


I carefully tuned that code. It was multi day exerci

Re: [vpp-dev] Multiarch/target select for dpdk_device_input

[Nitin Saxena]

Hi Damjan,

 Now that you are aware that Cavium is working on optimisations for ARM, can I 
request that you check with us on implications for ARM(at least Cavium), before 
bringing changes in dpdk-input?

Regards,
Nitin

On 01-Jun-2018, at 21:39, Damjan Marion mailto:dmar...@me.com>> 
wrote:


Dear Nitin,

I really don't have anything else to add. It your call how do you want to 
proceed

Regards,

Damjan

On 1 Jun 2018, at 18:02, Nitin Saxena 
mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

Answers Inline.

Thanks,
Nitin

On Friday 01 June 2018 08:49 PM, Damjan Marion wrote:
Hi Nitin,
inline...
On 1 Jun 2018, at 15:23, Nitin Saxena 
mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

It was hard to know that you have subset of patches hidden somewhere.
I wouldn't say patches are hidden. We are trying to fine tune dpdk-input 
initially from our end first and later we will seek your expertise while 
upstreaming.
for me they were hidden.
Typically it makes sense to discuss such kind of changes with person >who 
"maintains" the code before starting writing the code.
Agreed. However we prefer to do internal analysis/POC first before reaching out 
to MAINTAINERS. That way we can better understand code review comments.
Perfectly fine, but then don't put blame on us for not knowing that you are 
doing something internally...
The intention was not to blame anybody but to understand modular approach in 
vpp to accommodate multi-arch(s).

Maybe, but sounds to me like we are still in guessing phase.
I wouldn't do any guess work with MAINTAINERS.

Maybe we even need different function for each ARM CPU core as they
maybe have different memory subsystem and pipeline
This is what I am looking for. Is it ok to detect our hardware natively from 
autoconf and append target specific macro to CFLAGS? And then separate function 
for our target in dpdk/device/node.c? Sorry my multi-arch select example was 
incorrect and that's not what I am looking at.
Here I will be able to help when I get reasonable understanding what is the 
"big" plan.
The "Big" plan is to optimize each vpp node for Aarch64. For now focus is 
dpdk-input.
I don't want that we end up in 6 months with cavium patches, nxp patches, 
marvell patches, and so on.
Is it a problem? If yes than I am not able to visualize it as the same problem 
would exist for any architecture and not just for Aarch64.

Is there an agreement between ARM vendors what is the targeted core
you want to have code tuned for or you are simply tuning to whatever
core Cavium uses?
I am trying to optimize Cavium's SOC. This question is in this regard only. 
However efforts are going on optimizing Cortex cores as well by ARM community.
What about agreeing on plan for optimising on all ARM cores, and then starting 
doing optimisation?
This is cross-company question so hard to answer but Cavium has the "big" plan 
described above.

Thanks,
Nitin

On Friday 01 June 2018 01:55 AM, Damjan Marion wrote:
inline...
--
Damjan
On 31 May 2018, at 21:10, Saxena, Nitin 
mailto:nitin.sax...@cavium.com> 
<mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

Answers inline.

Thanks,
Nitin

On 01-Jun-2018, at 12:15 AM, Damjan Marion 
mailto:dmarion.li...@gmail.com> 
<mailto:dmarion.li...@gmail.com>> wrote:


Dear Nitin,

See inline….


On 31 May 2018, at 19:59, Nitin Saxena 
mailto:nitin.sax...@cavium.com> 
<mailto:nitin.sax...@cavium.com>> wrote:

Hi,

I am working on optimising dpdk-input node (based on vpp v1804) for our target. 
I am able to get performance improvements on our target but the problem I am 
finding now are:

1) The dpdk-input code is completely changed on master branch from v1804.

Why is this a problem? It was done with reason and for tangible benefit.
This is a problem for me as I can not apply my v1804 changes directly to the 
master branch. I have to again rework on master branch and that’s why I am not 
able to move to master branch or v1807 in future.
It was hard to know that you have subset of patches hidden somewhere. Typically 
it makes sense to discuss such kind of changes with person who "maintains" the 
code before starting writing the code.

Not to mention the dpdk-input master branch code do not give better numbers on 
our target as compared to v1804

Sad to hear that, good thing is, it gives better numbers on x86.
As I understand one dpdk_device_input function cannot be same for all 
architectures because if the underlying micro-architecture is different, the 
hot spots changes.
Maybe, but sounds to me like we are still in guessing phase.
Maybe we even need different function for each ARM CPU core as they maybe have 
different memory subsystem and pipeline
Is there an agreement between ARM vendors what is the targeted core you want to 
have code tuned for or you are simply tuning to whatever core Cavium uses?
I have seen dpdk-input master branch changes 

[vpp-dev] gerrit job failing on ARM due to pre-installed dpdk package

[Nitin Saxena]

Hi,

Gerrit patch: https://gerrit.fd.io/r/#/c/12813/ failing because dpdk 
package is pre-installed on ThunderX and not compiled during this patch.


Logs from: 
https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1604/617/console

=
12:22:26 make -C dpdk install-deb
12:22:26 make[1]: Entering directory 
'/w/workspace/vpp-arm-verify-master-ubuntu1604/dpdk'

12:22:26 ==
12:22:26  Up-to-date DPDK package already installed
12:22:26 ==
12:22:26 make[1]: Leaving directory 
'/w/workspace/vpp-arm-verify-master-ubuntu1604/dpdk

=

This patch require dpdk package to re-compile and install again. How to 
trigger it on ThunderX?


Thanks,
Nitin


-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#9518): https://lists.fd.io/g/vpp-dev/message/9518
View All Messages In Topic (1): https://lists.fd.io/g/vpp-dev/topic/21233595
Mute This Topic: https://lists.fd.io/mt/21233595/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
Email sent to: arch...@mail-archive.com
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] #vpp Cavium ThunderX (ARM64) Build error: config.status: error: cannot find input file: `vlib/config.h.in'

[Nitin Saxena]

Ideally this static assert shouldn't have happened on master branch 
unless you are using external dpdk.


can you please check

1) what is the value of CONFIG_RTE_CACHE_LINE_SIZE in file

/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk/custom-config 



2) can you paste content of file

/home/stanislav/work/vpp/build-root/build-vpp_debug-native/vpp/vppinfra/config.h

Regards,
Nitin

On Wednesday 18 July 2018 07:14 PM, Stanislav Chlebec wrote:


External Email

stanislav@contivvpp:~/work/vpp$ gcc --version

gcc (Ubuntu/Linaro 5.5.0-12ubuntu1~16.04) 5.5.0 20171010

Copyright (C) 2015 Free Software Foundation, Inc.

This is free software; see the source for copying conditions. There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR 
PURPOSE.


stanislav@contivvpp:~/work/vpp$

stanislav@contivvpp:~/work/vpp$ make build

make[1]: Entering directory '/home/stanislav/work/vpp/build-root'

 Arch for platform 'vpp' is native 

 Finding source for dpdk 

 Makefile fragment found in 
/home/stanislav/work/vpp/build-data/packages/dpdk.mk 


 Source found in /home/stanislav/work/vpp/dpdk 

 Arch for platform 'vpp' is native 

 Finding source for vpp 

 Makefile fragment found in 
/home/stanislav/work/vpp/build-data/packages/vpp.mk 


 Source found in /home/stanislav/work/vpp/src 

 Configuring dpdk in 
/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk 


 Building dpdk in 
/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk 


make[2]: Entering directory '/home/stanislav/work/vpp/dpdk'

make check-deb







/home/stanislav/work/vpp/build-data/../src/plugins/dpdk/device/init.c: 
In function ‘dpdk_init’:


/home/stanislav/work/vpp/build-data/../src/plugins/dpdk/device/init.c:1743:3: 
error: static assertion failed: "DPDK RTE CACHE LINE SIZE does not 
match with 1<

STATIC_ASSERT (RTE_CACHE_LINE_SIZE == 1 << CLIB_LOG2_CACHE_LINE_BYTES,

^

/home/stanislav/work/vpp/build-data/../src/plugins/dpdk/device/init.c: 
At top level:


cc1: error: unrecognized command line option 
‘-Wno-address-of-packed-member’ [-Werror]


cc1: all warnings being treated as errors

Makefile:3694: recipe for target 'dpdk/device/dpdk_plugin_la-init.lo' 
failed


make[5]: *** [dpdk/device/dpdk_plugin_la-init.lo] Error 1

make[5]: *** Waiting for unfinished jobs

make[5]: Leaving directory 
'/home/stanislav/work/vpp/build-root/build-vpp_debug-native/vpp/plugins'


Makefile:2395: recipe for target 'all' failed

make[4]: *** [all] Error 2

make[4]: Leaving directory 
'/home/stanislav/work/vpp/build-root/build-vpp_debug-native/vpp/plugins'


Makefile:8363: recipe for target 'all-recursive' failed

make[3]: *** [all-recursive] Error 1

make[3]: Leaving directory 
'/home/stanislav/work/vpp/build-root/build-vpp_debug-native/vpp'


Makefile:4243: recipe for target 'all' failed

make[2]: *** [all] Error 2

make[2]: Leaving directory 
'/home/stanislav/work/vpp/build-root/build-vpp_debug-native/vpp'


Makefile:691: recipe for target 'vpp-build' failed

make[1]: *** [vpp-build] Error 2

make[1]: Leaving directory '/home/stanislav/work/vpp/build-root'

Makefile:333: recipe for target 'build' failed

make: *** [build] Error 2

*From:*Nitin Saxena [mailto:nitin.sax...@cavium.com]
*Sent:* Wednesday, July 18, 2018 12:58 PM
*To:* vpp-dev@lists.fd.io
*Subject:* Re: [vpp-dev] #vpp Cavium ThunderX (ARM64) Build error: 
config.status: error: cannot find input file: `vlib/config.h.in'


Hi Stanislav,

Unfortunately I do not have access to ThunderX but my ThunderX2 
compilation works on master branch.


Also I observed that following gerrit job running on Cavium's ThunderX 
compiles src/vnet/ethernet/interface.c successfully.


https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1604/1404/console

So I am guessing you are compiling VPP master branch natively on 
Cavium's ThunderX. So I suggest


1) Either remove -Werror from vpp_TAG_CFLAGS/vpp_debug/TAG_CFLAGS in 
build-data/platforms/vpp.mk and ignore warning


2) I am guessing problem with your GCC version? Which gcc version you 
are using? Can you try using gcc 5.4.0/5.5.0


Thanks,

Nitin

On Wednesday 18 July 2018 03:47 PM, Stanislav Chlebec wrote:

External Email

I got

/home/stanislav/work/vpp/build-data/../src/vnet/ethernet/interface.c:529:785:
error: self-comparison always evaluates to true
[-Werror=tautological-compare]
    vlib_validate_buffer_enqueue_x4 (vm, node, next_index,
^
/home/stanislav/work/vpp/build-data/../src/vnet/ethernet/interface.c:
At top level:
cc1: error: unrecognized command line option
‘-Wno-address-of-packed-member’ [-Werror]
cc1: all warnings being treated as errors
Makefile:7307: recipe for target 'vnet/ethernet/interface.lo' failed




-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.

View/Reply Online (#9866):https://lis

Re: [vpp-dev] #vpp Cavium ThunderX (ARM64) Build error: config.status: error: cannot find input file: `vlib/config.h.in'

[Nitin Saxena]

Hi Stanislav,

My best guess that dpdk package is already installed in the system and 
that's why VPP is not compiling internal dpdk


Please uninstall dpdk installed package and re-run compilation. The 
dpdk/custom-config file is must


In case you want to use external dpdk please set 
CONFIG_RTE_CACHE_LINE_SIZE=128 in dpdk/config file


Regards,

Nitin


On Wednesday 18 July 2018 07:52 PM, Stanislav Chlebec wrote:


External Email

Ad 1

No such file exists

Ad 2

stanislav@contivvpp:~/work/vpp/build-root/build-vpp_debug-native/dpdk$ 
cat 
/home/stanislav/work/vpp/build-root/build-vpp_debug-native/vpp/vppinfra/config.h


/** Autogenerated by Autotools **/

#ifndef included_clib_config_h

#define included_clib_config_h

#ifndef CLIB_LOG2_CACHE_LINE_BYTES

#define CLIB_LOG2_CACHE_LINE_BYTES  7

#endif

#endif

stanislav@contivvpp:~/work/vpp/build-root/build-vpp_debug-native/dpdk$

Thanks

Stanislav

*From:*Nitin Saxena [mailto:nitin.sax...@cavium.com]
*Sent:* Wednesday, July 18, 2018 4:02 PM
*To:* Stanislav Chlebec ; 
vpp-dev@lists.fd.io
*Subject:* Re: [vpp-dev] #vpp Cavium ThunderX (ARM64) Build error: 
config.status: error: cannot find input file: `vlib/config.h.in'


Ideally this static assert shouldn't have happened on master branch 
unless you are using external dpdk.


can you please check

1) what is the value of CONFIG_RTE_CACHE_LINE_SIZE in file

/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk/custom-config 



2) can you paste content of file

/home/stanislav/work/vpp/build-root/build-vpp_debug-native/vpp/vppinfra/config.h

Regards,
Nitin

On Wednesday 18 July 2018 07:14 PM, Stanislav Chlebec wrote:

External Email

stanislav@contivvpp:~/work/vpp$ gcc --version

gcc (Ubuntu/Linaro 5.5.0-12ubuntu1~16.04) 5.5.0 20171010

Copyright (C) 2015 Free Software Foundation, Inc.

This is free software; see the source for copying conditions. 
There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.

stanislav@contivvpp:~/work/vpp$

stanislav@contivvpp:~/work/vpp$ make build

make[1]: Entering directory '/home/stanislav/work/vpp/build-root'

 Arch for platform 'vpp' is native 

 Finding source for dpdk 

 Makefile fragment found in
/home/stanislav/work/vpp/build-data/packages/dpdk.mk 

 Source found in /home/stanislav/work/vpp/dpdk 

 Arch for platform 'vpp' is native 

 Finding source for vpp 

 Makefile fragment found in
/home/stanislav/work/vpp/build-data/packages/vpp.mk 

 Source found in /home/stanislav/work/vpp/src 

 Configuring dpdk in
/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk 

 Building dpdk in
/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk 

make[2]: Entering directory '/home/stanislav/work/vpp/dpdk'

make check-deb







/home/stanislav/work/vpp/build-data/../src/plugins/dpdk/device/init.c:
In function ‘dpdk_init’:


/home/stanislav/work/vpp/build-data/../src/plugins/dpdk/device/init.c:1743:3:
error: static assertion failed: "DPDK RTE CACHE LINE SIZE does not
match with 1<mailto:nitin.sax...@cavium.com]
*Sent:* Wednesday, July 18, 2018 12:58 PM
*To:* vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>
*Subject:* Re: [vpp-dev] #vpp Cavium ThunderX (ARM64) Build error:
config.status: error: cannot find input file: `vlib/config.h.in'

Hi Stanislav,

Unfortunately I do not have access to ThunderX but my ThunderX2
compilation works on master branch.

Also I observed that following gerrit job running on Cavium's
ThunderX compiles src/vnet/ethernet/interface.c successfully.

https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1604/1404/console

So I am guessing you are compiling VPP master branch natively on
Cavium's ThunderX. So I suggest

1) Either remove -Werror from vpp_TAG_CFLAGS/vpp_debug/TAG_CFLAGS
in build-data/platforms/vpp.mk and ignore warning

2) I am guessing problem with your GCC version? Which gcc version
you are using? Can you try using gcc 5.4.0/5.5.0

Thanks,

Nitin

On Wednesday 18 July 2018 03:47 PM, Stanislav Chlebec wrote:

External Email

I got

/home/stanislav/work/vpp/build-data/../src/vnet/ethernet/interface.c:529:785:
error: self-comparison always evaluates to true
[-Werror=tautological-compare]
    vlib_validate_buffer_enqueue_x4 (vm, node, next_index,
^
/home/stanislav/work/vpp/build-data/../src/vnet/ethernet/interface.c:
At top level:
cc1: error: unrecognized command line option
‘-Wno-address-of-packed-member’ [-Werror]
cc1: all warnings being treated as errors
Makefile:7307: recipe for target 'vnet/ethernet/interface.lo'

Re: [vpp-dev] #vpp Cavium ThunderX (ARM64) Build error: config.status: error: cannot find input file: `vlib/config.h.in'

[Nitin Saxena]

Hi Stan,

I prefer you should use master branch and not v1804 for ThunderX.

Also "make bootstrap" is not required anymore. From the logs look like 
problem with Autoconf tools.


Thanks,

Nitin



On Wednesday 18 July 2018 02:24 PM, Stanislav Chlebec wrote:


External Email

Hello

I am trying to build VPP on Cavium ThunderX
git clone https://gerrit.fd.io/r/vpp
cd vpp/
git checkout v18.04

stanislav@contivvpp:~/work/vpp$ git status
HEAD detached at v18.04
nothing to commit, working directory clean

make install-dep
make bootstrap
make build

then I got this error message:
config.status: error: cannot find input file: `vlib/config.h.in'
Makefile:640: recipe for target 'vpp-configure' failed
make[1]: *** [vpp-configure] Error 1
make[1]: Leaving directory '/home/stanislav/work/vpp/build-root'
Makefile:317: recipe for target 'build' failed
make: *** [build] Error 2

Plese could you help me to solve that?
Thanks
Stanislav




stanislav@contivvpp:~/work/vpp$ make build
make[1]: Entering directory '/home/stanislav/work/vpp/build-root'
 Arch for platform 'vpp' is native 
 Finding source for dpdk 
 Makefile fragment found in 
/home/stanislav/work/vpp/build-data/packages/dpdk.mk 

 Source found in /home/stanislav/work/vpp/dpdk 
 Arch for platform 'vpp' is native 
 Finding source for vpp 
 Makefile fragment found in 
/home/stanislav/work/vpp/build-data/packages/vpp.mk 

 Source found in /home/stanislav/work/vpp/src 
 Configuring dpdk: nothing to do 
 Building dpdk in 
/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk 

make[2]: Entering directory '/home/stanislav/work/vpp/dpdk'
==
Building DPDK from source. Consider installing development
package by invoking 'make dpdk-install-dev' from the
top level directory
==
make config
make[3]: Entering directory '/home/stanislav/work/vpp/dpdk'
make[3]: warning: jobserver unavailable: using -j1.  Add '+' to parent 
make rule.
  % Total    % Received % Xferd  Average Speed   Time Time Time  
Current
 Dload  Upload   Total Spent    Left  
Speed
100  9.9M  100  9.9M    0 0  2335k  0  0:00:04  0:00:04 
--:--:-- 2335k

--- extracting dpdk-18.02.1.tar.xz ---
--- generating custom config from 
/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1/config/defconfig_arm64-thunderx-linuxapp-gcc 
---
make[4]: Entering directory 
'/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1'

Configuration done using custom-config
make[4]: Leaving directory 
'/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1'

make[3]: Leaving directory '/home/stanislav/work/vpp/dpdk'
make[2]: Leaving directory '/home/stanislav/work/vpp/dpdk'
 Installing dpdk 
make[2]: Entering directory '/home/stanislav/work/vpp/dpdk'
make install
make[3]: Entering directory '/home/stanislav/work/vpp/dpdk'
make[4]: Entering directory 
'/home/stanislav/work/vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1'

Configuration done using custom-config
== Build lib
== Build lib/librte_compat
== Build lib/librte_eal
  SYMLINK-FILE include/rte_compat.h
== Build lib/librte_eal/common
  SYMLINK-FILE include/rte_branch_prediction.h
  SYMLINK-FILE include/rte_common.h
  SYMLINK-FILE include/rte_debug.h
  SYMLINK-FILE include/rte_eal.h
  SYMLINK-FILE include/rte_eal_interrupts.h
  SYMLINK-FILE include/rte_errno.h
  SYMLINK-FILE include/rte_launch.h
  SYMLINK-FILE include/rte_lcore.h
  SYMLINK-FILE include/rte_log.h
  SYMLINK-FILE include/rte_memory.h
  SYMLINK-FILE include/rte_memzone.h
  SYMLINK-FILE include/rte_per_lcore.h
  SYMLINK-FILE include/rte_random.h
  SYMLINK-FILE include/rte_tailq.h
  SYMLINK-FILE include/rte_interrupts.h
  SYMLINK-FILE include/rte_alarm.h
  SYMLINK-FILE include/rte_string_fns.h
  SYMLINK-FILE include/rte_version.h
  SYMLINK-FILE include/rte_eal_memconfig.h
  SYMLINK-FILE include/rte_malloc_heap.h
  SYMLINK-FILE include/rte_hexdump.h
  SYMLINK-FILE include/rte_devargs.h
  SYMLINK-FILE include/rte_bus.h
  SYMLINK-FILE include/rte_dev.h
  SYMLINK-FILE include/rte_pci_dev_feature_defs.h
  SYMLINK-FILE include/rte_pci_dev_features.h
  SYMLINK-FILE include/rte_malloc.h
  SYMLINK-FILE include/rte_keepalive.h
  SYMLINK-FILE include/rte_time.h
  SYMLINK-FILE include/rte_service.h
  SYMLINK-FILE include/rte_service_component.h
  SYMLINK-FILE include/rte_bitmap.h
  SYMLINK-FILE include/rte_vfio.h
  SYMLINK-FILE include/rte_hypervisor.h
  SYMLINK-FILE include/rte_test.h
  SYMLINK-FILE include/rte_reciprocal.h
  SYMLINK-FILE include/rte_atomic.h
  SYMLINK-FILE include/rte_atomic_32.h
  SYMLINK-FILE include/rte_atomic_64.h
  SYMLINK-FILE include/rte_byteorder.h
  SYMLINK-FILE include/rte_cpuflags.h
  SYMLINK-FILE include/rte_cpuflags_32.h
  SYMLINK-FILE 

Re: [vpp-dev] #vpp Cavium ThunderX (ARM64)

[Nitin Saxena]

Hi Stanislav,

Are you seeing this issue with dpdk-18.05? You will get this error with 
dpdk-18.02.1.


In vpp master branch dpdk-18.05 is enabled by default.

Thanks,

Nitin


On Friday 20 July 2018 05:49 PM, Stanislav Chlebec wrote:


External Email

Hello all

I was trying to achieve cooperation of VPP with the NIC present in the 
ThunderX (c1.large.arm)

It was not successful:
https://gist.github.com/stanislav-chlebec/a36c43c2eee8c16d3d297e67bac1d711

Conclusion:
VPP was successfully built on the system.
VPP is not able to work with the NIC + vfio_pci neither with NIC + 
uio_pci_generic neither with NIC + igb_uio


The output of the cpmmand in vpp is like this

DBGvpp# sh int


Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count


VirtualFunctionEthernet1/0/2 1 error 9000/0/0/0


local0 0 down 0/0/0/0


DBGvpp# show hardware-interfaces


Name Idx Link Hardware


VirtualFunctionEthernet1/0/2 1 down VirtualFunctionEthernet1/0/2


Ethernet address 30:0e:d5:ff:0d:9e


Cavium ThunderX


carrier down


flags: pmd pmd-init-fail maybe-multiseg


rx queues 1, rx desc 1024, tx queues 1, tx desc 1024


cpu socket 0


Errors:


rte_eth_rx_queue_setup[port:0, errno:-22]: Unknown error -22




local0 0 down local0


local


DBGvpp#




DBGvpp# set interface state VirtualFunctionEthernet1/0/2 up


set interface state: Interface in the error state


DBGvpp# set interface state VirtualFunctionEthernet1/0/2 down


set interface state: Interface not initialized


DBGvpp#



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9897): https://lists.fd.io/g/vpp-dev/message/9897
Mute This Topic: https://lists.fd.io/mt/23753373/675748
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480717
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [nsax...@caviumnetworks.com]
-=-=-=-=-=-=-=-=-=-=-=-


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9907): https://lists.fd.io/g/vpp-dev/message/9907
Mute This Topic: https://lists.fd.io/mt/23753373/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] #vpp Cavium ThunderX (ARM64) Build error: config.status: error: cannot find input file: `vlib/config.h.in'

[Nitin Saxena]

Hi Stanislav,

Unfortunately I do not have access to ThunderX but my ThunderX2 
compilation works on master branch.


Also I observed that following gerrit job running on Cavium's ThunderX 
compiles src/vnet/ethernet/interface.c successfully.


https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1604/1404/console

So I am guessing you are compiling VPP master branch natively on 
Cavium's ThunderX. So I suggest


1) Either remove -Werror from vpp_TAG_CFLAGS/vpp_debug/TAG_CFLAGS in 
build-data/platforms/vpp.mk and ignore warning


2) I am guessing problem with your GCC version? Which gcc version you 
are using? Can you try using gcc 5.4.0/5.5.0



Thanks,

Nitin

On Wednesday 18 July 2018 03:47 PM, Stanislav Chlebec wrote:


External Email

I got
/home/stanislav/work/vpp/build-data/../src/vnet/ethernet/interface.c:529:785: 
error: self-comparison always evaluates to true 
[-Werror=tautological-compare]

    vlib_validate_buffer_enqueue_x4 (vm, node, next_index,
^
/home/stanislav/work/vpp/build-data/../src/vnet/ethernet/interface.c: 
At top level:
cc1: error: unrecognized command line option 
‘-Wno-address-of-packed-member’ [-Werror]

cc1: all warnings being treated as errors
Makefile:7307: recipe for target 'vnet/ethernet/interface.lo' failed


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9866): https://lists.fd.io/g/vpp-dev/message/9866
Mute This Topic: https://lists.fd.io/mt/23669223/675191
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=2214648
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [nitin.sax...@cavium.com]
-=-=-=-=-=-=-=-=-=-=-=-


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9868): https://lists.fd.io/g/vpp-dev/message/9868
Mute This Topic: https://lists.fd.io/mt/23669223/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] cmake is on

[Nitin Saxena]

Hi Damjan,

cmake is making Cavium's OCTEONTx build failure. Following is the 
failure log




In function ‘dpdk_init’:
/home/nsaxena/vpp/src/plugins/dpdk/device/init.c:1802:3: error: static 
assertion failed: "DPDK RTE CACHE LINE SIZE does not match with 
1<
   STATIC_ASSERT (RTE_CACHE_LINE_SIZE == 1 << CLIB_LOG2_CACHE_LINE_BYTES,

===

I am failing to see translation of Autoconf changes from commit id: 
473f46135c3fd77dad5614215cc279b1164e9a74 to CMake.


Thanks,

Nitin

On Sunday 02 September 2018 06:18 PM, Damjan Marion via Lists.Fd.Io wrote:


External Email


Dear all,

We just switched from autotools to cmake and retired all autotools 
related files in src/.


All verify jobs are ok, and we also tried it on 3 different x86 and 2 
different ARM Aarch64 machines.


Due to blast radius, i will not be surprised that some small issues 
pop out, but i don't expect anything hard to fix.


Let us know if you hit something...

PS As a part of this change, CentOS 7 build are now using 
devtoolset-7, so they are compiled with gcc-7, which also means images 
have support for Skylake Servers (AVX512).


--
Damjan



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10337): https://lists.fd.io/g/vpp-dev/message/10337
Mute This Topic: https://lists.fd.io/mt/25155374/675748
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [nsax...@caviumnetworks.com]
-=-=-=-=-=-=-=-=-=-=-=-


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10407): https://lists.fd.io/g/vpp-dev/message/10407
Mute This Topic: https://lists.fd.io/mt/25155374/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] cmake is on

[Nitin Saxena]

Thanks Marco, Damjan. Adding cpu id fixes the compilation

Pushed the patch: https://gerrit.fd.io/r/#/c/14701/

Thanks,

Nitin


On Thursday 06 September 2018 07:06 PM, Damjan Marion wrote:


External Email


Dear Nitin,

Yes, it is missing.  I took care only for platforms I have access to...

—
Damjan

On 6 Sep 2018, at 14:04, Nitin Saxena <mailto:nitin.sax...@cavium.com>> wrote:



Hi Damjan,

cmake is making Cavium's OCTEONTx build failure. Following is the 
failure log




In function ‘dpdk_init’:
/home/nsaxena/vpp/src/plugins/dpdk/device/init.c:1802:3: error: 
static assertion failed: "DPDK RTE CACHE LINE SIZE does not match 
with 1<
   STATIC_ASSERT (RTE_CACHE_LINE_SIZE == 1 << CLIB_LOG2_CACHE_LINE_BYTES,

===

I am failing to see translation of Autoconf changes from commit id: 
473f46135c3fd77dad5614215cc279b1164e9a74 to CMake.


Thanks,

Nitin

On Sunday 02 September 2018 06:18 PM, Damjan Marion via Lists.Fd.Io 
wrote:


External Email


Dear all,

We just switched from autotools to cmake and retired all autotools 
related files in src/.


All verify jobs are ok, and we also tried it on 3 different x86 and 
2 different ARM Aarch64 machines.


Due to blast radius, i will not be surprised that some small issues 
pop out, but i don't expect anything hard to fix.


Let us know if you hit something...

PS As a part of this change, CentOS 7 build are now using 
devtoolset-7, so they are compiled with gcc-7, which also means 
images have support for Skylake Servers (AVX512).


--
Damjan



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10337):https://lists.fd.io/g/vpp-dev/message/10337
Mute This Topic:https://lists.fd.io/mt/25155374/675748
Group Owner:vpp-dev+ow...@lists.fd.io
Unsubscribe:https://lists.fd.io/g/vpp-dev/unsub   [nsax...@caviumnetworks.com]
-=-=-=-=-=-=-=-=-=-=-=-


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10407): 
https://lists.fd.io/g/vpp-dev/message/10407 
<https://lists.fd.io/g/vpp-dev/message/10407>

Mute This Topic: https://lists.fd.io/mt/25155374/675642
Group Owner: vpp-dev+ow...@lists.fd.io <mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [dmar...@me.com 
<mailto:dmar...@me.com>]

-=-=-=-=-=-=-=-=-=-=-=-


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10413): https://lists.fd.io/g/vpp-dev/message/10413
Mute This Topic: https://lists.fd.io/mt/25155374/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] opensuse gerrit jobs failing

[Nitin Saxena]

Hi,

In gerrit patch: https://gerrit.fd.io/r/#/c/10148/ I am seeing opensuse 
job failures in master branch. I tried rechecking multiple times. Errors 
I have been seeing are following:


https://jenkins.fd.io/job/vpp-verify-master-opensuse/2627/console

10:45:30 curl: (6) Could not resolve host: ${HTTP_HOST}
10:45:30 Makefile:232: recipe for target 
'/w/workspace/vpp-verify-master-opensuse/dpdk/nasm-2.13.01.tar.xz' failed
10:45:30 make[3]: *** 
[/w/workspace/vpp-verify-master-opensuse/dpdk/nasm-2.13.01.tar.xz] Error 6
10:45:30 make[3]: Leaving directory 
'/w/workspace/vpp-verify-master-opensuse/dpdk'

10:45:30 error: Bad exit status from /var/tmp/rpm-tmp.YslJIa (%install)


https://jenkins.fd.io/job/vpp-verify-master-opensuse/2624/console

09:31:20
  0 00 00 0  0  0 --:--:-- --:--:-- 
--:--:-- 0curl: (7) Failed to connect to www.nasm.us port 80: 
Connection refused
09:31:20 Makefile:232: recipe for target 
'/w/workspace/vpp-verify-master-opensuse/dpdk/nasm-2.13.01.tar.xz' failed
09:31:20 make[3]: *** 
[/w/workspace/vpp-verify-master-opensuse/dpdk/nasm-2.13.01.tar.xz] Error 7
09:31:20 make[3]: Leaving directory 
'/w/workspace/vpp-verify-master-opensuse/dpdk'

09:31:20 error: Bad exit status from /var/tmp/rpm-tmp.JQL7uQ (%install)
09:31:20
09:31:20
09:31:20 RPM build errors:
09:31:20 Bad exit status from /var/tmp/rpm-tmp.JQL7uQ (%install)
09:31:20 Makefile:392: recipe for target 
'vpp-dpdk-devel-18.02-vpp1.x86_64.rpm' failed

09:31:20 make[2]: *** [vpp-dpdk-devel-18.02-vpp1.x86_64.rpm] Error 1


Any help how to fix this?

Thanks,
Nitin

-=-=-=-=-=-=-=-=-=-=-=-
Links:

You receive all messages sent to this group.

View/Reply Online (#8730): https://lists.fd.io/g/vpp-dev/message/8730
View All Messages In Topic (1): https://lists.fd.io/g/vpp-dev/topic/16427705
Mute This Topic: https://lists.fd.io/mt/16427705/21656
New Topic: https://lists.fd.io/g/vpp-dev/post

Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: vpp-dev+ow...@lists.fd.io
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Skipping ipv4-input processing in L3Fwd

[Nitin Saxena]

Hi,

In case of L3Fwd, there is a possibility of handing of vlib_buffer from 
device-input to ipv4-lookup and thereby skipping 
ipv4-input/ipv4-input-no-chksum completely.
All ipv4-input/ipv4-input-nochksum functionalities can be offloaded to hardware 
and hence the node can be skipped. 

Questions:
- If ipv4-input will be skipped is it fair to enable feature-arc in 
device-input node?
- Is it fair to add ipv4-lookup next node to device-input node? 

Any other approach to be think about to leverage hardware offload 
functionalities?

Thanks,
Nitin


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10606): https://lists.fd.io/g/vpp-dev/message/10606
Mute This Topic: https://lists.fd.io/mt/25907990/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Support of dpdk rte_security library ?

[Nitin Saxena]

Ok Thanks,

Nitin


On Monday 24 September 2018 05:36 PM, Damjan Marion wrote:


External Email


No AFAIK, we prefer to have native implementations.

--
Damjan

On 24 Sep 2018, at 13:16, Nitin Saxena <mailto:nitin.sax...@cavium.com>> wrote:


Hi,

Is there any road map to support DPDK rte_security library with IPSEC 
protocol?


Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10624): 
https://lists.fd.io/g/vpp-dev/message/10624 
<https://lists.fd.io/g/vpp-dev/message/10624>

Mute This Topic: https://lists.fd.io/mt/26198618/675642
Group Owner: vpp-dev+ow...@lists.fd.io <mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [dmar...@me.com 
<mailto:dmar...@me.com>]

-=-=-=-=-=-=-=-=-=-=-=-




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10627): https://lists.fd.io/g/vpp-dev/message/10627
Mute This Topic: https://lists.fd.io/mt/26198618/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Support of dpdk rte_security library ?

[Nitin Saxena]

Hi,

Is there any road map to support DPDK rte_security library with IPSEC 
protocol?


Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10624): https://lists.fd.io/g/vpp-dev/message/10624
Mute This Topic: https://lists.fd.io/mt/26198618/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Skipping ipv4-input processing in L3Fwd

[Nitin Saxena]

Thanks Dave for the detailed answers.

> Patches which skip ANY of the work described above will not be merged.
Understood. Intention is not to break functionality/security at all.

> You would have to pull a bunch of work back into the device driver: counting 
> IP packet errors, head of ip4/6 unicast/multicast input feature arcs, and 
> mandatory input checks (expired TTL, ip header length, fragment).
Is it a confirmation that starting of "ip4/ip6 unicast/multicast" feature-arc 
along with "device-input" feature arc in the “device_input” node itself is 
acceptable by VPP?

I am also assuming that ipv4-lookup can be device-input’s next node, provided 
stack functionalities/performance remains intact.

Thanks,
Nitin

> On 22-Sep-2018, at 6:12 PM, Dave Barach via Lists.Fd.Io 
>  wrote:
> 
> External Email
> 
> You would have to pull a bunch of work back into the device driver: counting 
> IP packet errors, head of ip4/6 unicast/multicast input feature arcs, and 
> mandatory input checks (expired TTL, ip header length, fragment).
> 
> Hardware offload will NOT accomplish all of these tasks. If by chance you can 
> convince it and/or the device driver to do SOME of these things - perhaps the 
> ip header length, fragment, and expired TTL checks - variant ip4/6 input 
> nodes can be constructed by working on ip4_input_inline(...) and 
> ip6_input(...).
> 
> The rest of the forwarding graph - and by implication anyone running vpp - 
> depend on these checks having been performed. These checks are important for 
> three reasons: functional correctness, security, and observability.
> 
> Patches which skip ANY of the work described above will not be merged.
> 
> Thanks... Dave
> 
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Nitin Saxena
> Sent: Saturday, September 22, 2018 12:25 AM
> To: vpp-dev 
> Subject: [vpp-dev] Skipping ipv4-input processing in L3Fwd
> 
> Hi,
> 
> In case of L3Fwd, there is a possibility of handing of vlib_buffer from 
> device-input to ipv4-lookup and thereby skipping 
> ipv4-input/ipv4-input-no-chksum completely.
> All ipv4-input/ipv4-input-nochksum functionalities can be offloaded to 
> hardware and hence the node can be skipped.
> 
> Questions:
> - If ipv4-input will be skipped is it fair to enable feature-arc in 
> device-input node?
> - Is it fair to add ipv4-lookup next node to device-input node?
> 
> Any other approach to be think about to leverage hardware offload 
> functionalities?
> 
> Thanks,
> Nitin
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#10608): https://lists.fd.io/g/vpp-dev/message/10608
> Mute This Topic: https://lists.fd.io/mt/25907990/675748
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [nsax...@caviumnetworks.com]
> -=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10609): https://lists.fd.io/g/vpp-dev/message/10609
Mute This Topic: https://lists.fd.io/mt/25907990/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Skipping ipv4-input processing in L3Fwd

[Nitin Saxena]

Thanks Damjan. I will reanalyze my approach.

Thanks,
Nitin

> On 23-Sep-2018, at 16:29, Damjan Marion  wrote:
> 
> External Email
> 
> Dear Nitin,
> 
>> On 22 Sep 2018, at 20:30, Nitin Saxena  wrote:
>> 
>> Hi Damjan/Dave.
>> 
>>> You will end up with layering violation and bloatig driver code. Please 
>>> dont go there.
>> Ok. My motivation is the error packet should be dropped at the earliest. i.e
>> - In device-input node, if interface is not in L2 brdge or in L2XC config 
>> then drop all error packets in “device-input” node itself. Except TTL 
>> expiry, all ipv4-input checks drops the packet.
> 
> Device input node should not drop packets. However it can provide hint (like 
> we do with checksum today) so next node can deal with it faster.
> Few reasons why doing that is bad idea:
> - if packet have errors, it should still be subject of port mirroring (SPAN)
> - in some cases such packet should be sent to punt path
> 
>> - If interface is in L2 bridge or L2XC config, packet can take L2 path and 
>> ipv4-input stays in that path
> 
> Actually right now I'm working on completely decoupling device driver code 
> from l2/l3 path selection/ethertype lookup.
> So that logic is likely be removed from device driver.
> 
>> 
>>> No, it is not, ip4 packet may also go to l2 path.
>> Do you mean to say that in some cases L2 nodes are called after ip4-input? 
>> Could you please elaborate? “show vlib graph” output do not give any hint.
> 
> no, see above
> 
> 
>> 
>>> Ip4-input is also punt node, responsible for sending specific packets to 
>>> the control plane. Iz should stay in the path…
>> I fail to find the punt code that call ip4-input. Could you please point me 
>> to the code?
> 
> see src/vnet/ip/ip4_input.[ch]
> 
> --
> Damjan
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10614): https://lists.fd.io/g/vpp-dev/message/10614
Mute This Topic: https://lists.fd.io/mt/25907990/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Skipping ipv4-input processing in L3Fwd

[Nitin Saxena]

Hi Damjan/Dave.

> You will end up with layering violation and bloatig driver code. Please dont 
> go there.
Ok. My motivation is the error packet should be dropped at the earliest. i.e 
- In device-input node, if interface is not in L2 brdge or in L2XC config then 
drop all error packets in “device-input” node itself. Except TTL expiry, all 
ipv4-input checks drops the packet.
- If interface is in L2 bridge or L2XC config, packet can take L2 path and 
ipv4-input stays in that path

> No, it is not, ip4 packet may also go to l2 path.
Do you mean to say that in some cases L2 nodes are called after ip4-input? 
Could you please elaborate? “show vlib graph” output do not give any hint.

> Ip4-input is also punt node, responsible for sending specific packets to the 
> control plane. Iz should stay in the path…
I fail to find the punt code that call ip4-input. Could you please point me to 
the code?

Thanks,
Nitin

> On 22-Sep-2018, at 8:01 PM, Damjan Marion  wrote:
> 
> 
> —
> Damjan
> 
>> On 22 Sep 2018, at 16:10, Nitin Saxena  wrote:
>> 
>> Thanks Dave for the detailed answers.
>> 
>>> Patches which skip ANY of the work described above will not be merged.
>> Understood. Intention is not to break functionality/security at all.
>> 
>>> You would have to pull a bunch of work back into the device driver: 
>>> counting IP packet errors, head of ip4/6 unicast/multicast input feature 
>>> arcs, and mandatory input checks (expired TTL, ip header length, fragment).
>> Is it a confirmation that starting of "ip4/ip6 unicast/multicast" 
>> feature-arc along with "device-input" feature arc in the “device_input” node 
>> itself is acceptable by VPP?
> 
> No, it is not, ip4 packet may also go to l2 path.  You will end up with 
> layering violation and bloatig driver code. Please dont go there.
>> 
>> I am also assuming that ipv4-lookup can be device-input’s next node, 
>> provided stack functionalities/performance remains intact.
> 
> Ip4-input is also punt node, responsible for sending specific packets to the 
> control plane. Iz should stay in the path...
> 
>> 
>> Thanks,
>> Nitin
>> 
>>> On 22-Sep-2018, at 6:12 PM, Dave Barach via Lists.Fd.Io 
>>>  wrote:
>>> 
>>> External Email
>>> 
>>> You would have to pull a bunch of work back into the device driver: 
>>> counting IP packet errors, head of ip4/6 unicast/multicast input feature 
>>> arcs, and mandatory input checks (expired TTL, ip header length, fragment).
>>> 
>>> Hardware offload will NOT accomplish all of these tasks. If by chance you 
>>> can convince it and/or the device driver to do SOME of these things - 
>>> perhaps the ip header length, fragment, and expired TTL checks - variant 
>>> ip4/6 input nodes can be constructed by working on ip4_input_inline(...) 
>>> and ip6_input(...).
>>> 
>>> The rest of the forwarding graph - and by implication anyone running vpp - 
>>> depend on these checks having been performed. These checks are important 
>>> for three reasons: functional correctness, security, and observability.
>>> 
>>> Patches which skip ANY of the work described above will not be merged.
>>> 
>>> Thanks... Dave
>>> 
>>> -Original Message-
>>> From: vpp-dev@lists.fd.io  On Behalf Of Nitin Saxena
>>> Sent: Saturday, September 22, 2018 12:25 AM
>>> To: vpp-dev 
>>> Subject: [vpp-dev] Skipping ipv4-input processing in L3Fwd
>>> 
>>> Hi,
>>> 
>>> In case of L3Fwd, there is a possibility of handing of vlib_buffer from 
>>> device-input to ipv4-lookup and thereby skipping 
>>> ipv4-input/ipv4-input-no-chksum completely.
>>> All ipv4-input/ipv4-input-nochksum functionalities can be offloaded to 
>>> hardware and hence the node can be skipped.
>>> 
>>> Questions:
>>> - If ipv4-input will be skipped is it fair to enable feature-arc in 
>>> device-input node?
>>> - Is it fair to add ipv4-lookup next node to device-input node?
>>> 
>>> Any other approach to be think about to leverage hardware offload 
>>> functionalities?
>>> 
>>> Thanks,
>>> Nitin
>>> 
>>> 
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> 
>>> View/Reply Online (#10608): https://lists.fd.io/g/vpp-dev/message/10608
>>> Mute This Topic: https://lists.fd.io/mt/25907990/675748
>>> Group Owner: vpp-dev+ow...@lists.fd.io
>>> Unsubscribe: https://lists.

Re: [vpp-dev] regarding cross compiling

[Nitin Saxena]

Hi Damjan,

>>Somebody needs to spend a bit of time to teach CMake how to properly 
>>cross-compile

Correct me if I am wrong but I think cross-compilation support was there before 
CMake transition. Also I am not finding capability to link VPP with externally 
compiled dpdk? Am I correct or missing anything

Thanks,

Nitin

On 19/12/18 4:38 PM, Damjan Marion via Lists.Fd.Io wrote:

External Email

Somebody needs to spend a bit of time to teach CMake how to properly 
cross-compile

--
Damjan

On 19 Dec 2018, at 11:19, saint_sun 孙 via Lists.Fd.Io 
mailto:saint_sun=aliyun@lists.fd.io>> 
wrote:

Hi all,
I was confused with cross compiling, when I built vpp with the new 
version(18.10) using arm build tools.

The compiling environment is listed below:
-
vpp 18.10
host : ubuntu 16.04 , x86-64 cpu
target : armv71
toolchain : arm-linux-gcc alredy exist in my home directory
--

   The compiling process is listed below:

--
   user@vpp-compiler:~/vpp1810$ make build V=1 PLATFORM=arm32 or 
~/vpp1810/build-root$ make V=1 PLATFORM=arm32 TAG=vpp install-deb
make[1]: Entering directory '/home/user/vpp1810/build-root'
 Arch for platform 'arm32' is arm 
 Finding source for external 
 Makefile fragment found in 
/home/user/vpp1810/build-data/packages/external.mk 
 Source found in /home/user/vpp1810/build 
 Arch for platform 'arm32' is arm 
 Finding source for vpp 
 Makefile fragment found in /home/user/vpp1810/build-data/packages/vpp.mk 

 Source found in /home/user/vpp1810/src 
 Configuring external in 
/home/user/vpp1810/build-root/build-arm32_debug-arm/external 

 Building external in 
/home/user/vpp1810/build-root/build-arm32_debug-arm/external 
make[2]: Entering directory '/home/user/vpp1810/build/external'
make check-deb
make[3]: Entering directory '/home/user/vpp1810/build/external'
make[3]: warning: jobserver unavailable: using -j1.  Add '+' to parent make 
rule.
make[3]: Nothing to be done for 'check-deb'.
make[3]: Leaving directory '/home/user/vpp1810/build/external'
make[2]: Leaving directory '/home/user/vpp1810/build/external'
 Installing external 
make[2]: Entering directory '/home/user/vpp1810/build/external'
make[2]: Nothing to be done for 'ebuild-install'.
make[2]: Leaving directory '/home/user/vpp1810/build/external'
 Configuring vpp in /home/user/vpp1810/build-root/build-arm32_debug-arm/vpp 

-- The C compiler identification is unknown
-- Check for working C compiler using: Ninja
-- Check for working C compiler using: Ninja -- broken
CMake Error at /usr/share/cmake-3.5/Modules/CMakeTestCCompiler.cmake:61 
(message):
  The C compiler "/usr/lib/ccache/cc" is not able to compile a simple test
  program.

  It fails with the following output:

   Change Dir: 
/home/user/vpp1810/build-root/build-arm32l_debug-arm/vpp/CMakeFiles/CMakeTmp



  Run Build Command:"/usr/bin/ninja" "cmTC_d3272"

  [1/2] Building C object CMakeFiles/cmTC_d3272.dir/testCCompiler.c.o

  FAILED: /usr/lib/ccache/cc -g -O0 -DCLIB_DEBUG -DFORTIFY_SOURCE=2
  -DVLIB_MAX_CPUS=4 -march=armv7-a -fstack-protector-all -fPIC -Werror -o
  CMakeFiles/cmTC_d3272.dir/testCCompiler.c.o -c testCCompiler.c

  testCCompiler.c:1:0: error: bad value (armv7-a) for -march= switch

   #ifdef __cplusplus
   ^

  ninja: build stopped: subcommand failed.





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:16 (project)


-- Configuring incomplete, errors occurred!
See also 
"/home/user/vpp1810/build-root/build-arm32_debug-arm/vpp/CMakeFiles/CMakeOutput.log".
See also 
"/home/user/vpp1810/build-root/build-arm32_debug-arm/vpp/CMakeFiles/CMakeError.log".
Makefile:640: recipe for target 'vpp-configure' failed
make[1]: *** [vpp-configure] Error 1
make[1]: Leaving directory '/home/user/vpp1810/build-root'
Makefile:356: recipe for target 'build' failed
make: *** [build] Error 2
---

Could anyone give me some ticks or some examples how can I build sucess using 
vpp 18.10?

Any reply will be appreciated!
thanks,


saint_...@aliyun.com
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11691): https://lists.fd.io/g/vpp-dev/message/11691
Mute This Topic: https://lists.fd.io/mt/28800506/675642
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[dmar...@me.com]
-=-=-=-=-=-=-=-=-=-=-=-




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this 

Re: [EXT] Re: [vpp-dev] 128 byte cache line support

[Nitin Saxena]

Hi,


First all sorry for responding late to this mail chain. Please see my answers 
inline in blue


Thanks,

Nitin



From: Damjan Marion 
Sent: Monday, March 18, 2019 4:48 PM
To: Honnappa Nagarahalli
Cc: vpp-dev; Nitin Saxena
Subject: [EXT] Re: [vpp-dev] 128 byte cache line support

External Email



On 15 Mar 2019, at 04:52, Honnappa Nagarahalli 
mailto:honnappa.nagaraha...@arm.com>> wrote:



Related to change 18278[1], I was wondering if there is really a benefit of 
dealing with 128-byte cachelines like we do today.
Compiling VPP with cacheline size set to 128 will basically just add 64 bytes 
of unused space at the end of each cacheline so
vlib_buffer_t for example will grow from 128 bytes to 256 bytes, but we will 
still need to prefetch 2 cachelines like we do by default.

[Nitin]: This is the existing model. In case of forwarding mainly first vlib 
cache line size is being used. We are utilising existing hole (in first vlib 
cache line) by putting packet parsing info (Size ==64B). This has many 
benefits, one of them is to avoid ipv4-input-no-chksum() software checks. It 
gives us ~20 cycles benefits on our platform. So I do not want to lose that 
gain.

Whta will happen if we just leave that to be 64?

[Nitin]: This will create L1D holes on 128B targets right? Unutilized holes are 
not acceptable as it will waste L1D space and thereby affecting performance. On 
the contrary we want to pack structures from  2x64B to 1x128B cache line size 
to reduce number of pending prefetches in core pipeline. VPP heavily prefetches 
LOAD/STORE version of 64B and our effort is to reduce them for our target.

[Honnappa] Currently, ThunderX1 and Octeon TX have 128B cache line. What I have 
heard from Marvel folks is 64B cache line setting in DPDK does not work. I have 
not gone into details on what does not work exactly. May be Nitin can elaborate.

I’m curious to hear details…

1. sometimes (and not very frequently) we will issue 2 prefetch instructions 
for same cacheline, but I hope hardware is smart enough to just ignore 2nd one

2. we may face false sharing issues if first 64 bytes is touched by one thread 
and another 64 bytes are touched by another one

Second one sounds to me like a real problem, but it can be solved by aligning 
all per-thread data structures to 2 x cacheline size.

[Honnappa] Sorry, I don’t understand you here. Even if the data structure is 
aligned on 128B (2 X 64B), 2 contiguous blocks of 64B data would be on a single 
cache line.

I wanted to say that we can align all per-thread data structures to 128 bytes, 
even on systems which have 64 byte cacheline size.

Actually If i remember correctly, even on x86 some of hardware prefetchers are 
dealing with blocks of 2 cachelines.

So unless I missed something, my proposal here is, instead of maintaining 
special 128 byte images for some ARM64 machines,
let’s just align all per-thread data structures to 128 and have just one ARM 
image.

[Honnappa] When we run VPP compiled with 128B cache line size on platforms with 
64B cache line size, there is a performance degradation.

Yeah, sure, what I’m suggesting here is how to address that perf degradation.
[Nitin]: Is this proposal for Intel as well? If yes then I am fine with the 
proposal but I think it will decrease performance on 64B architecture with 
existing code.

Hence the proposal is to make sure the distro packages run on all platforms. 
But one can get the best performance when compiled for a particular target.

Thoughts?

--
Damjan


[1] https://gerrit.fd.io/r/#/c/18278/

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12532): https://lists.fd.io/g/vpp-dev/message/12532
Mute This Topic: https://lists.fd.io/mt/30426937/675477
Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[honnappa.nagaraha...@arm.com<mailto:honnappa.nagaraha...@arm.com>]
-=-=-=-=-=-=-=-=-=-=-=-
IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12595): https://lists.fd.io/g/vpp-dev/message/12595
Mute This Topic: https://lists.fd.io/mt/30633927/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] Re: [vpp-dev] 128 byte cache line support

[Nitin Saxena]

Hi Damjan,



From: vpp-dev@lists.fd.io  on behalf of Damjan Marion via 
Lists.Fd.Io 
Sent: Thursday, March 21, 2019 6:03 PM
To: Nitin Saxena
Cc: vpp-dev@lists.fd.io
Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support



> On 21 Mar 2019, at 06:51, Nitin Saxena  wrote:
>
> Hi,
>
> First all sorry for responding late to this mail chain. Please see my answers 
> inline in blue
>
> Thanks,
> Nitin
>
>
> From: Damjan Marion 
> Sent: Monday, March 18, 2019 4:48 PM
> To: Honnappa Nagarahalli
> Cc: vpp-dev; Nitin Saxena
> Subject: [EXT] Re: [vpp-dev] 128 byte cache line support
>
> External Email
>
>
>> On 15 Mar 2019, at 04:52, Honnappa Nagarahalli 
>>  wrote:
>>
>>
>>
>> Related to change 18278[1], I was wondering if there is really a benefit of 
>> dealing with 128-byte cachelines like we do today.
>> Compiling VPP with cacheline size set to 128 will basically just add 64 
>> bytes of unused space at the end of each cacheline so
>> vlib_buffer_t for example will grow from 128 bytes to 256 bytes, but we will 
>> still need to prefetch 2 cachelines like we do by default.
>>
>> [Nitin]: This is the existing model. In case of forwarding mainly first vlib 
>> cache line size is being used. We are utilising existing hole (in first vlib 
>> cache line) by putting packet parsing info (Size ==64B). This has many 
>> benefits, one of them is to avoid ipv4-input-no-chksum() software checks. It 
>> gives us ~20 cycles benefits on our platform. So I do not want to lose that 
>> gain.

That sounds like a terribly bad idea, and it likely will never be upstreamed.
vlib_buffer_t is 128-byte data structure, and it is perfect fit for 128-byte 
cacheline size systems. I don't see a point in growing this to 256-byte.
If you need more space, you can always grow headroom space for additional 
cacheline and store whatever you want there.
[Nitin]: In current upstream code, size of VLIB_BUFFER_HDR_SIZE is 256 byte for 
128B cache line target and not 128 byte. So we haven't done any changes in the 
upstream code except introducing CLIB_MIN_CACHE_LINE_BYTES == 64 macro and 
putting it across vlib_buffer_t as MARKER. So we are utilising existing hole 
(unused) in first cache line of vlib_buffer_t.

>> Whta will happen if we just leave that to be 64?
>>
>> [Nitin]: This will create L1D holes on 128B targets right?
>>
>> Unutilized holes are not acceptable as it will waste L1D space and thereby 
>> affecting performance. On the contrary we want to pack structures from  
>> 2x64B to 1x128B cache line size to reduce number of pending prefetches in 
>> core pipeline. VPP heavily prefetches LOAD/STORE version of 64B and our 
>> effort is to reduce them for our target.
>>

Not sure what do you mean by L1D holes. My proposal is that we align all 
per-thread data structures to 128 bytes, not to grow anything.
[Nitin]: Ok I misunderstood your comment when you said "if we just leave that 
to be 64".

>> [Honnappa] Currently, ThunderX1 and Octeon TX have 128B cache line. What I 
>> have heard from Marvel folks is 64B cache line setting in DPDK does not 
>> work. I have not gone into details on what does not work exactly. May be 
>> Nitin can elaborate.
>>
> I’m curious to hear details…
>>
>> 1. sometimes (and not very frequently) we will issue 2 prefetch instructions 
>> for same cacheline, but I hope hardware is smart enough to just ignore 2nd 
>> one
>>
>> 2. we may face false sharing issues if first 64 bytes is touched by one 
>> thread and another 64 bytes are touched by another one
>>
>> Second one sounds to me like a real problem, but it can be solved by 
>> aligning all per-thread data structures to 2 x cacheline size.
>>
>> [Honnappa] Sorry, I don’t understand you here. Even if the data structure is 
>> aligned on 128B (2 X 64B), 2 contiguous blocks of 64B data would be on a 
>> single cache line.
>>
> I wanted to say that we can align all per-thread data structures to 128 
> bytes, even on systems which have 64 byte cacheline size.
>>
>> Actually If i remember correctly, even on x86 some of hardware prefetchers 
>> are dealing with blocks of 2 cachelines.
>>
>> So unless I missed something, my proposal here is, instead of maintaining 
>> special 128 byte images for some ARM64 machines,
>> let’s just align all per-thread data structures to 128 and have just one ARM 
>> image.
>>
>> [Honnappa] When we run VPP compiled with 128B cache line size on platforms 
>> with 64B cache line size, there is a performance degradation.
>>
> Yeah, sure, what I’m suggestin

Re: [EXT] Re: [vpp-dev] 128 byte cache line support

[Nitin Saxena]

Hi Damjan,


From: Damjan Marion 
Sent: Thursday, March 21, 2019 8:47 PM
To: Nitin Saxena
Cc: vpp-dev@lists.fd.io; Narayana Prasad Raju Athreya
Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support



On 21 Mar 2019, at 16:04, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Damjan,


From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> on behalf of Damjan Marion 
via Lists.Fd.Io mailto:dmarion=me@lists.fd.io>>
Sent: Thursday, March 21, 2019 6:03 PM
To: Nitin Saxena
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support



> On 21 Mar 2019, at 06:51, Nitin Saxena 
> mailto:nsax...@marvell.com>> wrote:
>
> Hi,
>
> First all sorry for responding late to this mail chain. Please see my answers 
> inline in blue
>
> Thanks,
> Nitin
>
>
> From: Damjan Marion mailto:dmar...@me.com>>
> Sent: Monday, March 18, 2019 4:48 PM
> To: Honnappa Nagarahalli
> Cc: vpp-dev; Nitin Saxena
> Subject: [EXT] Re: [vpp-dev] 128 byte cache line support
>
> External Email
>
>
>> On 15 Mar 2019, at 04:52, Honnappa Nagarahalli 
>> mailto:honnappa.nagaraha...@arm.com>> wrote:
>>
>>
>>
>> Related to change 18278[1], I was wondering if there is really a benefit of 
>> dealing with 128-byte cachelines like we do today.
>> Compiling VPP with cacheline size set to 128 will basically just add 64 
>> bytes of unused space at the end of each cacheline so
>> vlib_buffer_t for example will grow from 128 bytes to 256 bytes, but we will 
>> still need to prefetch 2 cachelines like we do by default.
>>
>> [Nitin]: This is the existing model. In case of forwarding mainly first vlib 
>> cache line size is being used. We are utilising existing hole (in first vlib 
>> cache line) by putting packet parsing info (Size ==64B). This has many 
>> benefits, one of them is to avoid ipv4-input-no-chksum() software checks. It 
>> gives us ~20 cycles benefits on our platform. So I do not want to lose that 
>> gain.

That sounds like a terribly bad idea, and it likely will never be upstreamed.
vlib_buffer_t is 128-byte data structure, and it is perfect fit for 128-byte 
cacheline size systems. I don't see a point in growing this to 256-byte.
If you need more space, you can always grow headroom space for additional 
cacheline and store whatever you want there.
[Nitin]: In current upstream code, size of VLIB_BUFFER_HDR_SIZE is 256 byte for 
128B cache line target and not 128 byte. So we haven't done any changes in the 
upstream code except introducing CLIB_MIN_CACHE_LINE_BYTES == 64 macro and 
putting it across vlib_buffer_t as MARKER. So we are utilising existing hole 
(unused) in first cache line of vlib_buffer_t.

I understood that, i’m proposing that we fix that so header size is always 128, 
as it should be.
[Nitin]: At this point I am not in favour of changing existing vlib_buffer_t 
structure as I already mentioned that we gain ~20 cycles in case of L3Fwd. I 
don't have data how much it would help me in TCP or UDP handling where we would 
touch second cache line of vlib header.

>> Whta will happen if we just leave that to be 64?
>>
>> [Nitin]: This will create L1D holes on 128B targets right?
>>
>> Unutilized holes are not acceptable as it will waste L1D space and thereby 
>> affecting performance. On the contrary we want to pack structures from  
>> 2x64B to 1x128B cache line size to reduce number of pending prefetches in 
>> core pipeline. VPP heavily prefetches LOAD/STORE version of 64B and our 
>> effort is to reduce them for our target.
>>

Not sure what do you mean by L1D holes. My proposal is that we align all 
per-thread data structures to 128 bytes, not to grow anything.
[Nitin]: Ok I misunderstood your comment when you said "if we just leave that 
to be 64".

>> [Honnappa] Currently, ThunderX1 and Octeon TX have 128B cache line. What I 
>> have heard from Marvel folks is 64B cache line setting in DPDK does not 
>> work. I have not gone into details on what does not work exactly. May be 
>> Nitin can elaborate.
>>
> I’m curious to hear details…
>>
>> 1. sometimes (and not very frequently) we will issue 2 prefetch instructions 
>> for same cacheline, but I hope hardware is smart enough to just ignore 2nd 
>> one
>>
>> 2. we may face false sharing issues if first 64 bytes is touched by one 
>> thread and another 64 bytes are touched by another one
>>
>> Second one sounds to me like a real problem, but it can be solved by 
>> aligning all per-thread data structures to 2 x cacheline size.
>>
>> [

Re: [EXT] Re: [vpp-dev] 128 byte cache line support

[Nitin Saxena]

Hi Damjan,


See my comments inline.


From: Damjan Marion 
Sent: Thursday, March 21, 2019 9:34 PM
To: Nitin Saxena
Cc: vpp-dev@lists.fd.io; Narayana Prasad Raju Athreya
Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support



> On 21 Mar 2019, at 16:36, Nitin Saxena  wrote:
>
> Hi Damjan,
>
> From: Damjan Marion 
> Sent: Thursday, March 21, 2019 8:47 PM
> To: Nitin Saxena
> Cc: vpp-dev@lists.fd.io; Narayana Prasad Raju Athreya
> Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support
>
>
>
>> On 21 Mar 2019, at 16:04, Nitin Saxena  wrote:
>>
>> Hi Damjan,
>>
>> From: vpp-dev@lists.fd.io  on behalf of Damjan Marion 
>> via Lists.Fd.Io 
>> Sent: Thursday, March 21, 2019 6:03 PM
>> To: Nitin Saxena
>> Cc: vpp-dev@lists.fd.io
>> Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support
>>
>>
>>
>> > On 21 Mar 2019, at 06:51, Nitin Saxena  wrote:
>> >
>> > Hi,
>> >
>> > First all sorry for responding late to this mail chain. Please see my 
>> > answers inline in blue
>> >
>> > Thanks,
>> > Nitin
>> >
>> >
>> > From: Damjan Marion 
>> > Sent: Monday, March 18, 2019 4:48 PM
>> > To: Honnappa Nagarahalli
>> > Cc: vpp-dev; Nitin Saxena
>> > Subject: [EXT] Re: [vpp-dev] 128 byte cache line support
>> >
>> > External Email
>> >
>> >
>> >> On 15 Mar 2019, at 04:52, Honnappa Nagarahalli 
>> >>  wrote:
>> >>
>> >>
>> >>
>> >> Related to change 18278[1], I was wondering if there is really a benefit 
>> >> of dealing with 128-byte cachelines like we do today.
>> >> Compiling VPP with cacheline size set to 128 will basically just add 64 
>> >> bytes of unused space at the end of each cacheline so
>> >> vlib_buffer_t for example will grow from 128 bytes to 256 bytes, but we 
>> >> will still need to prefetch 2 cachelines like we do by default.
>> >>
>> >> [Nitin]: This is the existing model. In case of forwarding mainly first 
>> >> vlib cache line size is being used. We are utilising existing hole (in 
>> >> first vlib cache line) by putting packet parsing info (Size ==64B). This 
>> >> has many benefits, one of them is to avoid ipv4-input-no-chksum() 
>> >> software checks. It gives us ~20 cycles benefits on our platform. So I do 
>> >> not want to lose that gain.
>>
>> That sounds like a terribly bad idea, and it likely will never be upstreamed.
>> vlib_buffer_t is 128-byte data structure, and it is perfect fit for 128-byte 
>> cacheline size systems. I don't see a point in growing this to 256-byte.
>> If you need more space, you can always grow headroom space for additional 
>> cacheline and store whatever you want there.
>> [Nitin]: In current upstream code, size of VLIB_BUFFER_HDR_SIZE is 256 byte 
>> for 128B cache line target and not 128 byte. So we haven't done any changes 
>> in the upstream code except introducing CLIB_MIN_CACHE_LINE_BYTES == 64 
>> macro and putting it across vlib_buffer_t as MARKER. So we are utilising 
>> existing hole (unused) in first cache line of vlib_buffer_t.
>
> I understood that, i’m proposing that we fix that so header size is always 
> 128, as it should be.
> [Nitin]: At this point I am not in favour of changing existing vlib_buffer_t 
> structure as I already mentioned that we gain ~20 cycles in case of L3Fwd. I 
> don't have data how much it would help me in TCP or UDP handling where we 
> would touch second cache line of vlib header.

What you are doing in your closed repo cannot be argument here.
We need to do what is right for everybody, not something to support your closed 
repo hacks.
[Nitin]: We have plan to upstream that piece of code. We are not willing to 
keep it closed as its inline with the upstream code. Currently we are in 
process of adding hardware mempool in VPP and native plugin for our target. We 
have plan to upstream those changes but we cannot upstream code while they are 
failing on our target.

If you plan to keep it closed source, you can do with that code whatever you 
want anyway.
If you want to upstream it, right thing to do is to discuss such changes before 
coding them.
[Nitin]: Actually I did discuss this piece of code, we have implemented 
"ipv4-input-nochksum-nottl-nodata" node which you and Dave had suggested.

>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12605): https://lists.fd.io/g/vpp-dev/message/12605
Mute This Topic: https://lists.fd.io/mt/30633927/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] 128 byte cache line support

[Nitin Saxena]

From: Damjan Marion 
Sent: Friday, March 22, 2019 3:59 PM
To: Nitin Saxena 
Cc: vpp-dev@lists.fd.io; Narayana Prasad Raju Athreya 
Subject: Re: [EXT] [vpp-dev] 128 byte cache line support




On 22 Mar 2019, at 09:35, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Damjan,

>> Can we conclude that there is no technical arguments?

On technical aspects:

  1.  Changing VLIB structure to 128 byte from current 256 byte (for our target)

 *   In case of forwarding we only use single Dcache line till ipv4-input 
(because of packet parsing info in second 64B of vlib). The second cache line, 
which is data, being accessed only in ipv4-lookup and ipv4-rewrite node. With 
the proposal we have to put our hardware parsing information to HEADROOM, 
thereby increasing one cache line access to every node. This will impact 
performance for us.

I don't think this is technical argument, it falls to me under tecnical depth 
category.
Just for my curiosity, what’s wrong with putting your data into vlib_buffer_t-> 
opaque2 ?
[Nitin]: So 56 byte of opaque2 data should be fine if nobody overwrites to it 
in stack. In that case we prefer to have opaque2 data  at 64 byte aligned 
position (just after cacheline1 marker). Then I am fine with the proposal.


 *

  1.  Changing remaining per-thread structures to 128B

 *   This proposal looks fine for other data structures. However:
   i.  You 
already mentioned that in worst case there will be two prefetches of 64B but in 
our processor second prefetch should be ignored. I checked internally and yes 
the second prefetch will be ignored if the two prefetches are very close. 
However the second prefetch will not be ignored on the very first instruction 
pipeline. So I feel its wastage of few pipelining stages per packet. Otherwise 
it looks fine to me.

So here we just have your feeling, no real arguments, so i guess it is fine...

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12619): https://lists.fd.io/g/vpp-dev/message/12619
Mute This Topic: https://lists.fd.io/mt/30699557/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] 128 byte cache line support

[Nitin Saxena]

From: Damjan Marion 
Sent: Friday, March 22, 2019 12:35 AM
To: Nitin Saxena
Cc: vpp-dev@lists.fd.io; Narayana Prasad Raju Athreya
Subject: Re: [EXT] [vpp-dev] 128 byte cache line support



On 21 Mar 2019, at 17:11, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Damjan,

See my comments inline.


From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, March 21, 2019 9:34 PM
To: Nitin Saxena
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; Narayana Prasad Raju 
Athreya
Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support



> On 21 Mar 2019, at 16:36, Nitin Saxena 
> mailto:nsax...@marvell.com>> wrote:
>
> Hi Damjan,
>
> From: Damjan Marion mailto:dmar...@me.com>>
> Sent: Thursday, March 21, 2019 8:47 PM
> To: Nitin Saxena
> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; Narayana Prasad Raju 
> Athreya
> Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support
>
>
>
>> On 21 Mar 2019, at 16:04, Nitin Saxena 
>> mailto:nsax...@marvell.com>> wrote:
>>
>> Hi Damjan,
>>
>> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
>> mailto:vpp-dev@lists.fd.io>> on behalf of Damjan Marion 
>> via Lists.Fd.Io 
>> mailto:dmarion=me@lists.fd.io>>
>> Sent: Thursday, March 21, 2019 6:03 PM
>> To: Nitin Saxena
>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
>> Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support
>>
>>
>>
>> > On 21 Mar 2019, at 06:51, Nitin Saxena 
>> > mailto:nsax...@marvell.com>> wrote:
>> >
>> > Hi,
>> >
>> > First all sorry for responding late to this mail chain. Please see my 
>> > answers inline in blue
>> >
>> > Thanks,
>> > Nitin
>> >
>> >
>> > From: Damjan Marion mailto:dmar...@me.com>>
>> > Sent: Monday, March 18, 2019 4:48 PM
>> > To: Honnappa Nagarahalli
>> > Cc: vpp-dev; Nitin Saxena
>> > Subject: [EXT] Re: [vpp-dev] 128 byte cache line support
>> >
>> > External Email
>> >
>> >
>> >> On 15 Mar 2019, at 04:52, Honnappa Nagarahalli 
>> >> mailto:honnappa.nagaraha...@arm.com>> wrote:
>> >>
>> >>
>> >>
>> >> Related to change 18278[1], I was wondering if there is really a benefit 
>> >> of dealing with 128-byte cachelines like we do today.
>> >> Compiling VPP with cacheline size set to 128 will basically just add 64 
>> >> bytes of unused space at the end of each cacheline so
>> >> vlib_buffer_t for example will grow from 128 bytes to 256 bytes, but we 
>> >> will still need to prefetch 2 cachelines like we do by default.
>> >>
>> >> [Nitin]: This is the existing model. In case of forwarding mainly first 
>> >> vlib cache line size is being used. We are utilising existing hole (in 
>> >> first vlib cache line) by putting packet parsing info (Size ==64B). This 
>> >> has many benefits, one of them is to avoid ipv4-input-no-chksum() 
>> >> software checks. It gives us ~20 cycles benefits on our platform. So I do 
>> >> not want to lose that gain.
>>
>> That sounds like a terribly bad idea, and it likely will never be upstreamed.
>> vlib_buffer_t is 128-byte data structure, and it is perfect fit for 128-byte 
>> cacheline size systems. I don't see a point in growing this to 256-byte.
>> If you need more space, you can always grow headroom space for additional 
>> cacheline and store whatever you want there.
>> [Nitin]: In current upstream code, size of VLIB_BUFFER_HDR_SIZE is 256 byte 
>> for 128B cache line target and not 128 byte. So we haven't done any changes 
>> in the upstream code except introducing CLIB_MIN_CACHE_LINE_BYTES == 64 
>> macro and putting it across vlib_buffer_t as MARKER. So we are utilising 
>> existing hole (unused) in first cache line of vlib_buffer_t.
>
> I understood that, i’m proposing that we fix that so header size is always 
> 128, as it should be.
> [Nitin]: At this point I am not in favour of changing existing vlib_buffer_t 
> structure as I already mentioned that we gain ~20 cycles in case of L3Fwd. I 
> don't have data how much it would help me in TCP or UDP handling where we 
> would touch second cache line of vlib header.

What you are doing in your closed repo cannot be argument here.
We need to do what is right for everybody, not something to support your closed 
repo hacks.
[Nitin]: We have plan to upstream that piece of code. We are n

Re: [EXT] [vpp-dev] 128 byte cache line support

[Nitin Saxena]

Hi Damjan,

>> Can we conclude that there is no technical arguments?

On technical aspects:

  1.  Changing VLIB structure to 128 byte from current 256 byte (for our target)
 *   In case of forwarding we only use single Dcache line till ipv4-input 
(because of packet parsing info in second 64B of vlib). The second cache line, 
which is data, being accessed only in ipv4-lookup and ipv4-rewrite node. With 
the proposal we have to put our hardware parsing information to HEADROOM, 
thereby increasing one cache line access to every node. This will impact 
performance for us.
  2.  Changing remaining per-thread structures to 128B
 *   This proposal looks fine for other data structures. However:

   i.  You 
already mentioned that in worst case there will be two prefetches of 64B but in 
our processor second prefetch should be ignored. I checked internally and yes 
the second prefetch will be ignored if the two prefetches are very close. 
However the second prefetch will not be ignored on the very first instruction 
pipeline. So I feel its wastage of few pipelining stages per packet. Otherwise 
it looks fine to me.
Thanks,
Nitin


From: vpp-dev@lists.fd.io  On Behalf Of Damjan Marion via 
Lists.Fd.Io
Sent: Friday, March 22, 2019 1:32 PM
To: Nitin Saxena 
Cc: vpp-dev@lists.fd.io
Subject: Re: [EXT] [vpp-dev] 128 byte cache line support


—
Damjan

On 22 Mar 2019, at 01:37, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:




From: Damjan Marion mailto:dmar...@me.com>>
Sent: Friday, March 22, 2019 12:35 AM
To: Nitin Saxena
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; Narayana Prasad Raju 
Athreya
Subject: Re: [EXT] [vpp-dev] 128 byte cache line support




On 21 Mar 2019, at 17:11, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Damjan,

See my comments inline.


From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, March 21, 2019 9:34 PM
To: Nitin Saxena
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; Narayana Prasad Raju 
Athreya
Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support



> On 21 Mar 2019, at 16:36, Nitin Saxena 
> mailto:nsax...@marvell.com>> wrote:
>
> Hi Damjan,
>
> From: Damjan Marion mailto:dmar...@me.com>>
> Sent: Thursday, March 21, 2019 8:47 PM
> To: Nitin Saxena
> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; Narayana Prasad Raju 
> Athreya
> Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support
>
>
>
>> On 21 Mar 2019, at 16:04, Nitin Saxena 
>> mailto:nsax...@marvell.com>> wrote:
>>
>> Hi Damjan,
>>
>> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
>> mailto:vpp-dev@lists.fd.io>> on behalf of Damjan Marion 
>> via Lists.Fd.Io 
>> mailto:dmarion=me@lists.fd.io>>
>> Sent: Thursday, March 21, 2019 6:03 PM
>> To: Nitin Saxena
>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
>> Subject: Re: [EXT] Re: [vpp-dev] 128 byte cache line support
>>
>>
>>
>> > On 21 Mar 2019, at 06:51, Nitin Saxena 
>> > mailto:nsax...@marvell.com>> wrote:
>> >
>> > Hi,
>> >
>> > First all sorry for responding late to this mail chain. Please see my 
>> > answers inline in blue
>> >
>> > Thanks,
>> > Nitin
>> >
>> >
>> > From: Damjan Marion mailto:dmar...@me.com>>
>> > Sent: Monday, March 18, 2019 4:48 PM
>> > To: Honnappa Nagarahalli
>> > Cc: vpp-dev; Nitin Saxena
>> > Subject: [EXT] Re: [vpp-dev] 128 byte cache line support
>> >
>> > External Email
>> >
>> >
>> >> On 15 Mar 2019, at 04:52, Honnappa Nagarahalli 
>> >> mailto:honnappa.nagaraha...@arm.com>> wrote:
>> >>
>> >>
>> >>
>> >> Related to change 18278[1], I was wondering if there is really a benefit 
>> >> of dealing with 128-byte cachelines like we do today.
>> >> Compiling VPP with cacheline size set to 128 will basically just add 64 
>> >> bytes of unused space at the end of each cacheline so
>> >> vlib_buffer_t for example will grow from 128 bytes to 256 bytes, but we 
>> >> will still need to prefetch 2 cachelines like we do by default.
>> >>
>> >> [Nitin]: This is the existing model. In case of forwarding mainly first 
>> >> vlib cache line size is being used. We are utilising existing hole (in 
>> >> first vlib cache line) by putting packet parsing info (Size ==64B). This 
>> >> has many benefits, one of them is to avoid ipv4-input-no-chksum

Re: [vpp-dev] Understanding vlib_buffer_enqueue()

[Nitin Saxena]

If yes, why the frame needs to traverse two directions?Should it not 
traverse only feature-arc?

The flow is in one direction only.

Every node has speculated "next_index" node. If the speculated "next_index" 
node is not same as actual "next0" node  (which is typically retrieved by 
either parsing data or vlib buffer). So if speculated "next_index" node is not 
same as actual "next0" node, release/put the frame of "next_index" node and get 
the frame corresponding to "next0" node. Also make "next0" as "next_index" node 
(considering further packets will come corresponding to this "next_index" node).


Thanks,
Nitin

From: vpp-dev@lists.fd.io  on behalf of Gudimetla, Leela 
Sankar 
Sent: Friday, February 8, 2019 9:23 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Understanding vlib_buffer_enqueue()


External Email

Hello,



I am trying to understand a piece of code that enqueues a frame to next-node.

But when a frame is traversing feature-nodes, I see the following boiler-plate 
code that seems to be sending the frames to two nodes at the same time.

Is my understanding correct?



218 #define 
vlib_validate_buffer_enqueue_x1(vm,node,next_index,to_next,n_left_to_next,bi0,next0)
 \

219 do {  \

220   if (PREDICT_FALSE (next0 != next_index))\

221 { \

222   vlib_put_next_frame (vm, node, next_index, n_left_to_next + 1);   \

223   next_index = next0; \

224   vlib_get_next_frame (vm, node, next_index, to_next, n_left_to_next); \

225\

226   to_next[0] = bi0;   \

227   to_next += 1;\

228   n_left_to_next -= 1;\

229 } \

230 } while (0)



If yes, why the frame needs to traverse two directions?Should it not 
traverse only feature-arc?



Any pointers in understanding would really help.



Thanks,

Leela sankar
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12217): https://lists.fd.io/g/vpp-dev/message/12217
Mute This Topic: https://lists.fd.io/mt/29698182/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp graph dispatch fine-grained event log capture and visualization

[Nitin Saxena]

+1. Worth trying.


-Nitin


From: vpp-dev@lists.fd.io  on behalf of Dave Barach via 
Lists.Fd.Io 
Sent: Friday, February 8, 2019 9:51 PM
To: vpp-dev@lists.fd.io
Cc: vpp-dev@lists.fd.io
Subject: [vpp-dev] vpp graph dispatch fine-grained event log capture and 
visualization

External Email

Folks,

Here is a short VoD which shows how to capture and visualize fine-grained event 
log data: https://youtu.be/KwHOgdm3_sY

HTH... Dave

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12223): https://lists.fd.io/g/vpp-dev/message/12223
Mute This Topic: https://lists.fd.io/mt/29703228/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] RFC: buffer manager rework

[Nitin Saxena]

Hi Damjan,

Which exact operation do they accelerate?
There are many…basic features are…
- they accelerate fast buffer free and alloc. Single instruction required for 
both operations.
- Free list is maintained by hardware and not software.

Further other co-processors are dependent on buffer being managed by hardware 
instead of software so it is must to add support of hardware mem-pool in VPP. 
Software mempool will not work with other packet engines.

Thanks,
Nitin

On 03-Feb-2019, at 10:34 PM, Damjan Marion via Lists.Fd.Io 
mailto:dmarion=me@lists.fd.io>> wrote:


External Email


On 3 Feb 2019, at 16:58, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Damjan,

I have few queries regarding this patch.

 - DPDK mempools are not used anymore, we register custom mempool ops, and dpdk 
is taking buffers from VPP
Some of the targets uses hardware memory allocator like OCTEONTx family and 
NXP's dpaa. Those hardware allocators are exposed as dpdk mempools.

Which exact operation do they accelerate?

Now with this change I can see rte_mempool_populate_iova() is not anymore 
called.

Yes, but new code does pretty much the same thing, it populates both elt_list 
and mem_list. Also new code puts IOVA into mempool_objhdr.

So what is your suggestion to support such hardware.

Before I can provide any suggestion I need to understand better what those 
hardware buffer managers do
and why they are better than pure software solution we have today.



 - first 64-bytes of metadata are initialised on free, so buffer alloc is very 
fast
Is it fair to say if a mempool is created per worker core per sw_index 
(interface) then buffer template copy can be avoided even during free (It can 
be done only once at init time)

The really expensive part of buffer free operation is bringing cacheline into 
L1, and we need to do that to verify reference count of the packet.
At the moment when data is in L1, simply copying template will not cost much. 
1-2 clocks on x86, not sure about arm but still i expect that it will result in 
4 128-bit stores.
That was the rationale for resetting the metadata during buffer free.

So to answer your question, having buffer per sw-interface will likely improve 
performance a bit, but it will also cause sub-optimal use of buffer memory.
Such solution will also have problem in scaling, for example if you have 
hundreds of virtual interfaces...



Thanks,
Nitin


From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> on behalf of Damjan Marion 
via Lists.Fd.Io mailto:dmarion=me@lists.fd.io>>
Sent: Friday, January 25, 2019 10:38 PM
To: vpp-dev
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [vpp-dev] RFC: buffer manager rework

External Email

I am very close to the finish line with buffer management rework patch, and 
would like to
ask people to take a look before it is merged.

https://gerrit.fd.io/r/16638

It significantly improves performance of buffer alloc free and introduces numa 
awareness.
On my skylake platinum 8180 system, with native AVF driver observed performance 
improvement is:

- single core, 2 threads, ipv4 base forwarding test, CPU running at 2.5GHz (TB 
off):

old code - dpdk buffer manager: 20.4 Mpps
old code - old native buffer manager: 19.4 Mpps
new code: 24.9 Mpps

With DPDK drivers performance stays same as DPDK is maintaining own internal 
buffer cache.
So major perf gain should be observed in native code like: vhost-user, memif, 
AVF, host stack.

user facing changes:
to change number of buffers:
  old startup.conf:
dpdk { num-mbufs  }
  new startup.conf:
buffers { buffers-per-numa }

Internal changes:
 - free lists are deprecated
 - buffer metadata is always initialised.
 - first 64-bytes of metadata are initialised on free, so buffer alloc is very 
fast
 - DPDK mempools are not used anymore, we register custom mempool ops, and dpdk 
is taking buffers from VPP
 - to support such operation plugin can request external header space - in case 
of DPDK it stores rte_mbuf + rte_mempool_objhdr

I'm still running some tests so possible minor changes are possible, but 
nothing major expected.

--
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12016): https://lists.fd.io/g/vpp-dev/message/12016
Mute This Topic: https://lists.fd.io/mt/29539221/675748
Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[nsax...@caviumnetworks.com<mailto:nsax...@caviumnetworks.com>]
-=-=-=-=-=-=-=-=-=-=-=-

--
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12144): https://lists.fd.io/g/vpp-dev/message/12144
Mute This Topic: https://lists.fd.io/mt/29539221/675748
Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lis

Re: [vpp-dev] RFC: buffer manager rework

[Nitin Saxena]

Hi Damjan,

See function octeontx_fpa_bufpool_alloc() called by octeontx_fpa_dequeue(). Its 
a single read instruction to get the pointer of data.
Similarly, octeontx_fpa_bufpool_free() is also a single write instruction.

So, If you are able to prove with numbers that current software solution is 
low-performant and that you are confident that you can do significantly better, 
I will be happy to work with you on implementing support for hardware buffer 
manager.
First of all I welcome your patch as we were also trying to remove latencies 
seen by memcpy_x4() of buffer template. As I said earlier hardware buffer 
coprocessor is being used by other packet engines hence the support has to be 
added in VPP. I am looking for suggestion for its resolution.

Thanks,
Nitin

On 03-Feb-2019, at 11:39 PM, Damjan Marion via Lists.Fd.Io 
mailto:dmarion=me@lists.fd.io>> wrote:


External Email


On 3 Feb 2019, at 18:38, Nitin Saxena 
mailto:nitin.sax...@cavium.com>> wrote:

Hi Damjan,

Which exact operation do they accelerate?
There are many…basic features are…
- they accelerate fast buffer free and alloc. Single instruction required for 
both operations.

I quickly looked into DPDK octeontx_fpavf_dequeue() and it looks to me much 
more than one instruction.

In case of DPDK, how that works with DPDK mempool cache or are you disabling 
mempool cache completely?

Does single instruction alloc/free include:
 - reference_count check and decrement?
 - user metadata initialization ?

- Free list is maintained by hardware and not software.

Sounds to me that it is slower to program hardware, than to simply add few 
buffer indices to the end of vector but I may be wrong...


Further other co-processors are dependent on buffer being managed by hardware 
instead of software so it is must to add support of hardware mem-pool in VPP. 
Software mempool will not work with other packet engines.

But that can also be handled internally by device driver...

So, If you are able to prove with numbers that current software solution is 
low-performant and that you are confident that you can do significantly better, 
I will be happy to work with you on implementing support for hardware buffer 
manager.


Thanks,
Nitin

On 03-Feb-2019, at 10:34 PM, Damjan Marion via Lists.Fd.Io 
mailto:dmarion=me@lists.fd.io>> wrote:


External Email


On 3 Feb 2019, at 16:58, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Damjan,

I have few queries regarding this patch.

 - DPDK mempools are not used anymore, we register custom mempool ops, and dpdk 
is taking buffers from VPP
Some of the targets uses hardware memory allocator like OCTEONTx family and 
NXP's dpaa. Those hardware allocators are exposed as dpdk mempools.

Which exact operation do they accelerate?

Now with this change I can see rte_mempool_populate_iova() is not anymore 
called.

Yes, but new code does pretty much the same thing, it populates both elt_list 
and mem_list. Also new code puts IOVA into mempool_objhdr.

So what is your suggestion to support such hardware.

Before I can provide any suggestion I need to understand better what those 
hardware buffer managers do
and why they are better than pure software solution we have today.



 - first 64-bytes of metadata are initialised on free, so buffer alloc is very 
fast
Is it fair to say if a mempool is created per worker core per sw_index 
(interface) then buffer template copy can be avoided even during free (It can 
be done only once at init time)

The really expensive part of buffer free operation is bringing cacheline into 
L1, and we need to do that to verify reference count of the packet.
At the moment when data is in L1, simply copying template will not cost much. 
1-2 clocks on x86, not sure about arm but still i expect that it will result in 
4 128-bit stores.
That was the rationale for resetting the metadata during buffer free.

So to answer your question, having buffer per sw-interface will likely improve 
performance a bit, but it will also cause sub-optimal use of buffer memory.
Such solution will also have problem in scaling, for example if you have 
hundreds of virtual interfaces...



Thanks,
Nitin


From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> on behalf of Damjan Marion 
via Lists.Fd.Io mailto:dmarion=me@lists.fd.io>>
Sent: Friday, January 25, 2019 10:38 PM
To: vpp-dev
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [vpp-dev] RFC: buffer manager rework

External Email

I am very close to the finish line with buffer management rework patch, and 
would like to
ask people to take a look before it is merged.

https://gerrit.fd.io/r/16638

It significantly improves performance of buffer alloc free and introduces numa 
awareness.
On my skylake platinum 8180 system, with native AVF driver observed performance 
improvement is:

- single core, 2 threads, ipv4 base forwarding 

Re: [vpp-dev] RFC: buffer manager rework

[Nitin Saxena]

Hi Damjan,


I have few queries regarding this patch.


 - DPDK mempools are not used anymore, we register custom mempool ops, and dpdk 
is taking buffers from VPP

Some of the targets uses hardware memory allocator like OCTEONTx family and 
NXP's dpaa. Those hardware allocators are exposed as dpdk mempools. Now with 
this change I can see rte_mempool_populate_iova() is not anymore called. So 
what is your suggestion to support such hardware.

 - first 64-bytes of metadata are initialised on free, so buffer alloc is very 
fast
Is it fair to say if a mempool is created per worker core per sw_index 
(interface) then buffer template copy can be avoided even during free (It can 
be done only once at init time)

Thanks,
Nitin



From: vpp-dev@lists.fd.io  on behalf of Damjan Marion via 
Lists.Fd.Io 
Sent: Friday, January 25, 2019 10:38 PM
To: vpp-dev
Cc: vpp-dev@lists.fd.io
Subject: [vpp-dev] RFC: buffer manager rework

External Email

I am very close to the finish line with buffer management rework patch, and 
would like to
ask people to take a look before it is merged.

https://gerrit.fd.io/r/16638

It significantly improves performance of buffer alloc free and introduces numa 
awareness.
On my skylake platinum 8180 system, with native AVF driver observed performance 
improvement is:

- single core, 2 threads, ipv4 base forwarding test, CPU running at 2.5GHz (TB 
off):

old code - dpdk buffer manager: 20.4 Mpps
old code - old native buffer manager: 19.4 Mpps
new code: 24.9 Mpps

With DPDK drivers performance stays same as DPDK is maintaining own internal 
buffer cache.
So major perf gain should be observed in native code like: vhost-user, memif, 
AVF, host stack.

user facing changes:
to change number of buffers:
  old startup.conf:
dpdk { num-mbufs  }
  new startup.conf:
buffers { buffers-per-numa }

Internal changes:
 - free lists are deprecated
 - buffer metadata is always initialised.
 - first 64-bytes of metadata are initialised on free, so buffer alloc is very 
fast
 - DPDK mempools are not used anymore, we register custom mempool ops, and dpdk 
is taking buffers from VPP
 - to support such operation plugin can request external header space - in case 
of DPDK it stores rte_mbuf + rte_mempool_objhdr

I'm still running some tests so possible minor changes are possible, but 
nothing major expected.

--
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12016): https://lists.fd.io/g/vpp-dev/message/12016
Mute This Topic: https://lists.fd.io/mt/29539221/675748
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [nsax...@caviumnetworks.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12145): https://lists.fd.io/g/vpp-dev/message/12145
Mute This Topic: https://lists.fd.io/mt/29539221/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Centos job failing because of python-ply package

[Nitin Saxena]

Hi,

The Centos job has been failing in https://gerrit.fd.io/r/#/c/18564/ because 
cmake is not able to find python-ply package.
I can see that the master branch has fix from Paul for this but still this is 
failing. Any help??

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12882): https://lists.fd.io/g/vpp-dev/message/12882
Mute This Topic: https://lists.fd.io/mt/31418969/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Centos job failing because of python-ply package

[Nitin Saxena]

Failure log from 
https://jenkins.fd.io/job/vpp-verify-master-centos7/18637/console

13:05:40 FAILED: vlibmemory/memclnt.api.h
13:05:40 cd 
/w/workspace/vpp-verify-master-centos7/build-root/build-vpp-native/vpp/vlibmemory
 && mkdir -p 
/w/workspace/vpp-verify-master-centos7/build-root/build-vpp-native/vpp/vlibmemory
 && /w/workspace/vpp-verify-master-centos7/src/tools/vppapigen/vppapigen 
--includedir /w/workspace/vpp-verify-master-centos7/src --input 
/w/workspace/vpp-verify-master-centos7/src/vlibmemory/memclnt.api --output 
/w/workspace/vpp-verify-master-centos7/build-root/build-vpp-native/vpp/vlibmemory/memclnt.api.h
13:05:40 Traceback (most recent call last):
13:05:40   File 
"/w/workspace/vpp-verify-master-centos7/src/tools/vppapigen/vppapigen", line 4, 
in 
13:05:40 import ply.lex as lex
13:05:40 ModuleNotFoundError: No module named 'ply'

-Nitin

From: vpp-dev@lists.fd.io  On Behalf Of Nitin Saxena
Sent: Tuesday, April 30, 2019 4:52 PM
To: vpp-dev@lists.fd.io
Subject: [EXT] [vpp-dev] Centos job failing because of python-ply package

External Email

Hi,

The Centos job has been failing in https://gerrit.fd.io/r/#/c/18564/ because 
cmake is not able to find python-ply package.
I can see that the master branch has fix from Paul for this but still this is 
failing. Any help??

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12883): https://lists.fd.io/g/vpp-dev/message/12883
Mute This Topic: https://lists.fd.io/mt/31418969/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Marvell OCTEONTx2 plugin

[Nitin Saxena]

Hi Maintainers,

I have pushed patch for Marvell OCTEON TX2 driver plugin: 
https://gerrit.fd.io/r/#/c/vpp/+/21394/

Appreciate your feedback on the patch.

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13785): https://lists.fd.io/g/vpp-dev/message/13785
Mute This Topic: https://lists.fd.io/mt/32945036/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Marvell OCTEONTx2 plugin

[Nitin Saxena]

Hi,

Following is the link for README.md
https://gerrit.fd.io/r/#/c/vpp/+/21394/1/src/plugins/octeontx2/README.md

Thanks,
Nitin
From: vpp-dev@lists.fd.io  On Behalf Of Nitin Saxena
Sent: Monday, August 19, 2019 8:21 PM
To: vpp-dev@lists.fd.io
Cc: Narayana Prasad Raju Athreya 
Subject: [EXT] [vpp-dev] Marvell OCTEONTx2 plugin

External Email

Hi Maintainers,

I have pushed patch for Marvell OCTEON TX2 driver plugin: 
https://gerrit.fd.io/r/#/c/vpp/+/21394/

Appreciate your feedback on the patch.

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13786): https://lists.fd.io/g/vpp-dev/message/13786
Mute This Topic: https://lists.fd.io/mt/32945036/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] Re: [vpp-dev] Marvell OCTEONTx2 plugin

[Nitin Saxena]

Hi Damjan,


Thanks for comment. libmarvelldpdk is not intended to be a closed source and is 
part of Marvell SDK for our customers. Would it be ok to provide patches 
against DPDK release (from dpdk.org) OR does it need to be hosted publicly?


Thanks,

Nitin


From: vpp-dev@lists.fd.io  on behalf of Damjan Marion via 
Lists.Fd.Io 
Sent: Monday, August 19, 2019 9:30 PM
To: Nitin Saxena 
Cc: vpp-dev@lists.fd.io 
Subject: [EXT] Re: [vpp-dev] Marvell OCTEONTx2 plugin

External Email


Dear Nitin,

Where we can find “libmarvelldpdk”? Google search doesn’t show single 
occurrence.

We don’t support contributions linked to closed source libraries…

—
Damjan




On 19 Aug 2019, at 16:53, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi,

Following is the link for README.md
https://gerrit.fd.io/r/#/c/vpp/+/21394/1/src/plugins/octeontx2/README.md

Thanks,
Nitin
From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Nitin Saxena
Sent: Monday, August 19, 2019 8:21 PM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Cc: Narayana Prasad Raju Athreya 
mailto:pathr...@marvell.com>>
Subject: [EXT] [vpp-dev] Marvell OCTEONTx2 plugin

External Email

Hi Maintainers,

I have pushed patch for Marvell OCTEON TX2 driver plugin: 
https://gerrit.fd.io/r/#/c/vpp/+/21394/

Appreciate your feedback on the patch.

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13786): https://lists.fd.io/g/vpp-dev/message/13786
Mute This Topic: https://lists.fd.io/mt/32945036/675642
Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[dmar...@me.com<mailto:dmar...@me.com>]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13791): https://lists.fd.io/g/vpp-dev/message/13791
Mute This Topic: https://lists.fd.io/mt/32958379/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] Marvell OCTEONTx2 plugin

[Nitin Saxena]

Hi Damjan,

See my response inline

Thanks,
Nitin

From: Damjan Marion 
Sent: Tuesday, August 20, 2019 12:24 AM
To: Nitin Saxena 
Cc: vpp-dev@lists.fd.io
Subject: Re: [EXT] [vpp-dev] Marvell OCTEONTx2 plugin

Dear Nitin,

Here is my view on this:

 - All VPP dependencies should be open source and freely available software, 
afaik access to Marvell SDK is restricted to NDA customers and that is no go
[Nitin]: Got it. By dependency do you mean compile-time dependencies or 
run-time as well? Could you please clarify following doubts (None of them 
applies to my patch submission though)

  1.  If I have firmware that is loaded onto hardware and that would be 
completely closed-source(and part of Marvell SDK), would such firmware violate 
the dependency policy?
  2.  If I have a kernel module that is part of the Marvell SDK. Can VPP issue 
system calls to such a module, or would it violate the dependency policy you 
mentioned?
  3.  If I use dlopen to operate with a proprietary library in Marvell SDK, 
would that violate the dependency policy?

- preferred way to add device driver support in VPP is native one, typically 
provided as plugin and such driver can be linked to external library if it is 
open source. We already do that with marvel MUSDK and rdma-core. Nice feature 
of both libraries is that they provide way to access descriptors directly 
without metadata conversion into intermediate data structure (e.g. dpdk 
rte_mbuf).

- we are fine to have out-of-tree patches on top of DPDK as long as they are 
limited to providing new PMD and not altering common dpdk code.
[Nitin]: Yes my intent was same. I can share DPDK patches over email so you can 
take a look and let me know your views on that. Let me know if that is ok.

- Having separate plugin with own "special" version of DPDK is something I 
don't like. You should decide either to go with dpdk pmd and extend existing 
dpdk plugin, or create native driver without dpdk code.
[Nitin]: I tried to think along these lines but I could not contemplate all 
ramifications beyond OCTEONTX2. Perhaps you can look at my patches and see 
whether what you are suggesting is possible with the existing plugin?

Thanks,

Damjan


On 19 Aug 2019, at 20:03, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Damjan,

Thanks for comment. libmarvelldpdk is not intended to be a closed source and is 
part of Marvell SDK for our customers. Would it be ok to provide patches 
against DPDK release (from dpdk.org<http://dpdk.org/>) OR does it need to be 
hosted publicly?

Thanks,
Nitin

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> on behalf of Damjan Marion 
via Lists.Fd.Io mailto:dmarion=me@lists.fd.io>>
Sent: Monday, August 19, 2019 9:30 PM
To: Nitin Saxena mailto:nsax...@marvell.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>>
Subject: [EXT] Re: [vpp-dev] Marvell OCTEONTx2 plugin

External Email


Dear Nitin,

Where we can find “libmarvelldpdk”? Google search doesn’t show single 
occurrence.

We don’t support contributions linked to closed source libraries…

—
Damjan





On 19 Aug 2019, at 16:53, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi,

Following is the link for README.md
https://gerrit.fd.io/r/#/c/vpp/+/21394/1/src/plugins/octeontx2/README.md

Thanks,
Nitin
From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Nitin Saxena
Sent: Monday, August 19, 2019 8:21 PM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Cc: Narayana Prasad Raju Athreya 
mailto:pathr...@marvell.com>>
Subject: [EXT] [vpp-dev] Marvell OCTEONTx2 plugin

External Email

Hi Maintainers,

I have pushed patch for Marvell OCTEON TX2 driver plugin: 
https://gerrit.fd.io/r/#/c/vpp/+/21394/

Appreciate your feedback on the patch.

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13786): https://lists.fd.io/g/vpp-dev/message/13786
Mute This Topic: https://lists.fd.io/mt/32945036/675642
Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[dmar...@me.com<mailto:dmar...@me.com>]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13791): https://lists.fd.io/g/vpp-dev/message/13791
Mute This Topic: https://lists.fd.io/mt/32958379/675642
Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[dmar...@me.com<mailto:dmar...@me.com>]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13794): https://lists.fd.io/g/vpp-dev/message/13794

Re: [EXT] [vpp-dev] Marvell OCTEONTx2 plugin

[Nitin Saxena]

Hi Damjan and others,

I would appreciate if we continue to have discussion about pros and cons of the 
patch we pushed for Marvell OCTEON TX2 plugin.  With our DPDK based custom 
plugin we have achieved benefits of both worlds.

  1.  Leverage nice abstraction of DPDK APIs
  2.  Specific optimizations  for VPP by removing rte_mbuf() dependency

Just to re-iterate we want to have plugin based on custom DPDK because

  1.  The custom plugin allow us to achieve what a VPP native driver plugin can 
do. Currently both INPUT node (octeontx2-input) and Tx node takes ~35 cycles 
each and there is further scope for optimization.
  2.  The plugin does not conflict with existing dpdk plugin (as cmake require 
“libmarvelldpdk” library check to enable this plugin compilation).
 *   We are open to provide patches for review
 *   If required we can host custom library publicly
 *   All custom changes are restricted to PMD and there is no change in rte 
library.

As things stand out we have two alternatives to upstream this plugin in VPP

  1.  Add native support

  1.  Adding native ethdev driver support in VPP is a gigantic effort.(See: 
https://git.dpdk.org/dpdk/tree/drivers/net/octeontx2)
  2.  This mammoth task is not going to be meaningful for us as we are able to 
achieve similar performance with our custom DPDK plugin
  3.  We also have plans to use other well defined abstractions of DPDK like 
rte_security()/rte_flow()/rte_tm() etc.



  1.  Extend existing DPDK plugin for OCTEON TX2
 *   By doing so we will loose the cycles we have achieved by removing 
rte_mbuf both in Rx/Tx path
 *   Separate plugin make sense for us as we have plans to use optimized 
version of other DPDK’s abstraction APIs like

 i.   
rte_security (https://doc.dpdk.org/guides/prog_guide/rte_security.html)


So help us in understanding what can we do to upstream this plugin.  Appreciate 
if questions in following mail can be answered.

Thanks,
Nitin

From: vpp-dev@lists.fd.io  On Behalf Of Nitin Saxena
Sent: Tuesday, August 20, 2019 3:31 PM
To: Damjan Marion 
Cc: vpp-dev@lists.fd.io; Narayana Prasad Raju Athreya 
Subject: Re: [EXT] [vpp-dev] Marvell OCTEONTx2 plugin

Hi Damjan,

See my response inline

Thanks,
Nitin

From: Damjan Marion mailto:dmar...@me.com>>
Sent: Tuesday, August 20, 2019 12:24 AM
To: Nitin Saxena mailto:nsax...@marvell.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [EXT] [vpp-dev] Marvell OCTEONTx2 plugin

Dear Nitin,

Here is my view on this:

 - All VPP dependencies should be open source and freely available software, 
afaik access to Marvell SDK is restricted to NDA customers and that is no go
[Nitin]: Got it. By dependency do you mean compile-time dependencies or 
run-time as well? Could you please clarify following doubts (None of them 
applies to my patch submission though)

  1.  If I have firmware that is loaded onto hardware and that would be 
completely closed-source(and part of Marvell SDK), would such firmware violate 
the dependency policy?
  2.  If I have a kernel module that is part of the Marvell SDK. Can VPP issue 
system calls to such a module, or would it violate the dependency policy you 
mentioned?
  3.  If I use dlopen to operate with a proprietary library in Marvell SDK, 
would that violate the dependency policy?

- preferred way to add device driver support in VPP is native one, typically 
provided as plugin and such driver can be linked to external library if it is 
open source. We already do that with marvel MUSDK and rdma-core. Nice feature 
of both libraries is that they provide way to access descriptors directly 
without metadata conversion into intermediate data structure (e.g. dpdk 
rte_mbuf).

- we are fine to have out-of-tree patches on top of DPDK as long as they are 
limited to providing new PMD and not altering common dpdk code.
[Nitin]: Yes my intent was same. I can share DPDK patches over email so you can 
take a look and let me know your views on that. Let me know if that is ok.

- Having separate plugin with own "special" version of DPDK is something I 
don't like. You should decide either to go with dpdk pmd and extend existing 
dpdk plugin, or create native driver without dpdk code.
[Nitin]: I tried to think along these lines but I could not contemplate all 
ramifications beyond OCTEONTX2. Perhaps you can look at my patches and see 
whether what you are suggesting is possible with the existing plugin?

Thanks,

Damjan

On 19 Aug 2019, at 20:03, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Damjan,

Thanks for comment. libmarvelldpdk is not intended to be a closed source and is 
part of Marvell SDK for our customers. Would it be ok to provide patches 
against DPDK release (from dpdk.org<http://dpdk.org/>) OR does it need to be 
hosted publicly?

Thanks,
Nitin

From: vpp-dev@lists.fd.io<

Re: [vpp-dev] VPP Cross compilation (x86_64 -> aarch64)

[Nitin Saxena]

Hi,

  *   I didn't see an option in the root Makefile that would build just VPP 
without DPDK, but we want to support both.
As already described by Damjan in earlier mails, one can compile VPP without 
DPDK like below

  1.  git clone https://gerrit.fd.io/r/vpp;
  2.  Optional: cross compiled DPDK library present in $PWD/dpdk_cross/
  3.  mkdir ./vpp_cross; cd vpp_cross
  4.  cmake  -G Ninja -DCMAKE_PREFIX_PATH:PATH=$PWD/../dpdk_cross/ 
$PWD/../vpp/src/
  5.  ninja

In order to upstream above support for platform one has to add 
build-data/platforms/$(PLATFORM).mk.

However patch https://gerrit.fd.io/r/#/c/vpp/+/21035/ is trying to generate 
cross compiled DPDK through VPP build system which is different way of 
compiling VPP from the one I explained above.

Thanks,
Nitin
From: vpp-dev@lists.fd.io  On Behalf Of Juraj Linkeš
Sent: Friday, August 2, 2019 8:35 PM
To: vpp-dev@lists.fd.io
Subject: [EXT] [vpp-dev] VPP Cross compilation (x86_64 -> aarch64)

External Email

Hi VPP Devs,

I'm trying to understand how to implement support for cross compilation for 
aarch64. I've put together some changes that are finally cross compiling DPDK 
(https://gerrit.fd.io/r/#/c/vpp/+/21035/), but the compilation is failing when 
it doesn't find openssl headers:
In file included from 
/home/jlinkes/vpp/build-root/build-aarch64-aarch64/external/dpdk-19.05/drivers/crypto/qat/qat_sym.c:5:0:
/usr/include/openssl/evp.h:13:11: fatal error: openssl/opensslconf.h: No such 
file or directory
# include 
   ^~~
compilation terminated.

I've added locally cross compiled numalib to cflags and ldflags just to see how 
far I'd go with compilation. It seems that I'll need to do the same for not 
only openssl, but also the other dpdk dependencies (if I understand it 
correctly, I'll need at least ipseb-mb nasm) and possibly VPP dependencies.

I've seen some e-mails suggesting that I should produce my own 
platforms/.mk file, but not much about any other steps. I also see 
the CROSS_TOOLS var in build-root/Makefile (and the corresponding 
$(PLATFORM)_cross_tools), but I wasn't able to figure out how to use those - is 
that something I should look into? What are the actual dependencies that *need* 
to be cross compiled before attempting to cross compile vpp/dpdk? I'd like to 
understand a bit more about this before I attempt to possibly fit a circle into 
a square-shaped hole.

The goal here is to build VPP and DPDK (I didn't see an option in the root 
Makefile that would build just VPP without DPDK, but we want to support both) 
for generic armv8 linux target.

I'd appreciate it if someone looked at my WIP patch and told me what I'm doing 
wrong and what's missing.

Thanks,
Juraj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13670): https://lists.fd.io/g/vpp-dev/message/13670
Mute This Topic: https://lists.fd.io/mt/32691486/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] VPP version 19.04 - Confusing with vlib_buffer and rte_mbuf (use DPDK as interface input) referent count operations #vpp

[Nitin Saxena]

Hi,

>> I have searched the mailing list and know about the DPDK take buffer from 
>> VPP,
Correct. VPP does two things to allow DPDK to use VPP buffers.


  1.  Register two mempool ops to DPDK namely “vpp” (cached/fast) and 
“vpp-no-cache” (non-cached/slow). See function dpdk_buffer_pool_create() in 
src/plugin/dpdk/buffer.c
  2.  For each VPP buffer pool (as per NUMA) create two mempools, one having 
mempool_ops set as “vpp” and other one having “vpp-no-cache”

For each outgoing packet, non-cached mempool is used whenever there is (refcnt 
>1) (See dpdk_validate_rte_mbuf() function in 
src/plugins/dpdk/device/device.c). In case where refcnt >1, DPDK will end up 
calling enqueue/dequeue function of “vpp-no-cache” mempool ops, otherwise 
function of “vpp” mempool ops are called

Bottom line, buffers will be freed to VPP pool only when mbuf.refcnt ==1.

Thanks,
Nitin
From: vpp-dev@lists.fd.io  On Behalf Of guangwei
Sent: Wednesday, July 24, 2019 3:14 PM
To: vpp-dev@lists.fd.io
Subject: [EXT] [vpp-dev] VPP version 19.04 - Confusing with vlib_buffer and 
rte_mbuf (use DPDK as interface input) referent count operations #vpp

External Email

Hi, everyone

I'am a newer with VPP architeture,  I want't to use the vlib_buffer reference 
count(such as clone) to reduce the copies,
but I'am confused with the referent count  operations between the vlib_buffer 
and rte_mbuf (use DPDK as the interface input),  I have searched the mailing 
list and know about the DPDK take buffer from VPP,  but I'am still confused 
with these memory implementations,

Is there anyone  give some key points about the relationship between the 
rte_mbuf and vlib_buffer memory implementation, such as how DPDK take buffer 
from VPP and  allocation/free between the two parts?


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13560): https://lists.fd.io/g/vpp-dev/message/13560
Mute This Topic: https://lists.fd.io/mt/32582620/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] Request HELP with the boilerplate code in Node.c of a plugin #vpp

[Nitin Saxena]

Hi,

See if following link helps

https://fdio-vpp.readthedocs.io/en/latest/gettingstarted/developers/vlib.html#graph-dispatcher-internals

Thanks,
Nitin

From: vpp-dev@lists.fd.io  On Behalf Of 
chandan.moha...@nokia.com
Sent: Wednesday, July 24, 2019 3:55 PM
To: vpp-dev@lists.fd.io
Subject: [EXT] [vpp-dev] Request HELP with the boilerplate code in Node.c of a 
plugin #vpp

External Email


Hi ,
I am new to VPP.
I request,if any kind hearted soul can help me understand,the boilerplate code 
in Node.c of a plugin. I want to understand the buffer / frame / next handling 
of frame/next handling of node/prefetch code ,that is under both whileS() as 
below.

If you can point me to any URL or any PDF/DOC,that will help me a lot in 
understanding.


while (n_left_from > 0)
{
  u32 n_left_to_next;

  vlib_get_next_frame (vm, node, next_index, to_next, n_left_to_next);

  while (n_left_from > 0 && n_left_to_next > 0)
 {
.

...
}

If you can point me to any URL or any PDF/DOC,that will help me a lot in 
understanding.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13561): https://lists.fd.io/g/vpp-dev/message/13561
Mute This Topic: https://lists.fd.io/mt/32582776/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] cross-compilation support

[Nitin Saxena]

Hi Damjan,

>> My comment bellow is that only 1) is really needed to build VPP,  people can 
>> decide to use own build system like buildroot or yocto and invoke cmake 
>> directly, and completely ignore 2) and 3
Correct me if I am wrong, purpose of build-data/platforms/*.mk  (except vpp.mk) 
can be to configure ebuild for cross compilation. make build[-release] 
PLATFORM="aarch64..." was recommended way for cross compilation in VPP? If yes, 
is it OK to use 1) and 2) for cross compilation?

We internally uses (1) and (2) for our buildroot based cross compilation for 
octeontx. Hence I asked for the capability to pass vpp_cmake_args from 
build-data/platforms/octeontx.mk because I found it as a cleaner approach. 
Comments?

Thanks,
Nitin 

> -Original Message-
> From: Damjan Marion 
> Sent: Friday, November 1, 2019 1:34 AM
> To: Damjan Marion 
> Cc: Nitin Saxena ; vpp-dev@lists.fd.io
> Subject: Re: [EXT] [vpp-dev] cross-compilation support
> 
> 
> Let me clarify a bit more, as i can see that this still may be confusing:
> 
> 1) VPP is using cmake to build binaries and packages, all cmake related stuff
> is in src/.
> 
> 2) there is build-root/Makefile and files in build-data/* which are part of 
> old
> build system called ebuild
>  - ebuild is very complex set of make scripts which have similar functionality
> like buildroot, it was able to build kernel, toolchain , userspace tools and
> libraries
>  - today we don't use much of ebuild, it is just used to run VPP cmake 
> prioject
> in the right directory with he right set of command line arguments
> 
> 3) Other Makefiles
>   - top level makefile
>   - external deps makefiles in build/external/
> 
> My comment bellow is that only 1) is really needed to build VPP,  people can
> decide to use own build system like buildroot or yocto and invoke cmake
> directly, and completely ignore 2) and 3). In such case selected build system
> also needs to take care for dependencies like DPDK.
> 
> ebuild 2) have excellent cross compilation support for building target images
> when those images contain everything from kernel and libraries to vpp but it
> is not appropriate tool for creating distribution packages, i.e. creating 
> Centos
> ARM rpms on Ubuntu x86 machine. That is the reason why i tried to see if we
> can use docker instead to be able to build cross-arch, cross-distro or cross-
> distro-version (or all 3 together) packages.
> While my patch is incomplete, it looks to me like this approach will work.
> 
> My patch does native compilation only if TARGET_QUAD is equal to
> HOST_QUAD, where QUAD is:
>  - distro name (ubuntu, centos)
>  - distro version (18.04, 7.3)
>  - arch (x86_64, aarch64, ...)
>  - platform (generic, thunderx, ….)
> 
> it also support multiple build_types (release, debug, gcov, … )
> 
> At this point it is just early draft, but it shows some basic mechanics used 
> to
> produce both native and cross packages.
> 
> 
> > On 31 Oct 2019, at 20:32, Damjan Marion via Lists.Fd.Io
>  wrote:
> >
> > I have similar scheme on my mind, where you can have platform specific
> mk files loaded….
> >
> >> On 31 Oct 2019, at 20:23, Nitin Saxena  wrote:
> >>
> >> Hi,
> >>
> >>>> cmake  /path/to/vpp/src [your favorite xcompile args] cmake —build .
> >> Instead of having a shell script to pass cmake commands, I really liked the
> idea of passing vpp_cmake_args from build-data/platforms/*.mk
> (https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.fd.io_r_-
> 23_c_vpp_-2B_21035_23_build-2Ddata_platforms_native.mk-
> 4042=DwIFaQ=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGd
> uCfk9LbZMPOAecQGDzWV0=IxbLPjpkpGrr4_eh99mqx2_EhWrxhKwu-
> 8jEs5kosq8=izhC0z_k8Jz2hBw10il9EQpbYMyaLJ-d_7UczT_Ip8I= ). Any
> comment of taking those changes?
> >>
> >> Thanks,
> >> Nitin
> >>
> >>> -Original Message-
> >>> From: vpp-dev@lists.fd.io  On Behalf Of Damjan
> >>> Marion via Lists.Fd.Io
> >>> Sent: Friday, November 1, 2019 12:15 AM
> >>> To: Christian Hopps 
> >>> Cc: vpp-dev@lists.fd.io
> >>> Subject: [EXT] Re: [vpp-dev] cross-compilation support
> >>>
> >>> External Email
> >>>
> >>> 
> >>> --
> >>>
> >>> Nobody requires you to use docker, you are free to pass right
> >>> arguments straight to the cmake.
> >>>
> >>> All the stuff in build-data, build-root, build/ is optional, and it
> >>> is there to help developers to stage workspace an

Re: [EXT] Re: [vpp-dev] cross-compilation support

[Nitin Saxena]

Hi,

>> cmake  /path/to/vpp/src [your favorite xcompile args] cmake —build .
Instead of having a shell script to pass cmake commands, I really liked the 
idea of passing vpp_cmake_args from build-data/platforms/*.mk 
(https://gerrit.fd.io/r/#/c/vpp/+/21035/23/build-data/platforms/native.mk@42). 
Any comment of taking those changes?

Thanks,
Nitin

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Damjan
> Marion via Lists.Fd.Io
> Sent: Friday, November 1, 2019 12:15 AM
> To: Christian Hopps 
> Cc: vpp-dev@lists.fd.io
> Subject: [EXT] Re: [vpp-dev] cross-compilation support
> 
> External Email
> 
> --
> 
> Nobody requires you to use docker, you are free to pass right arguments
> straight to the cmake.
> 
> All the stuff in build-data, build-root, build/ is optional, and it is there 
> to help
> developers to stage workspace and create packages but VPP can be built as
> simple as:
> 
> mkdir build
> cd build
> cmake  /path/to/vpp/src [your favorite xcompile args] cmake —build .
> 
> 
> 
> > On 31 Oct 2019, at 19:39, Christian Hopps  wrote:
> >
> > I mean to call out the use of docker/qemu. In order to allow for this the
> user has to have extra privileges.
> >
> > As I mentioned, locally we use docker for building, I like it, it's a nice 
> > tool;
> however, my personal opinion is that it seems wrong to require docker
> and/or qemu for compiling an open source project.
> >
> > Important in holding this opinion is that it's seems possible to make this
> work w/o docker/qemu/root, I think someone just has to find the time to
> make an acceptable to the project working patch.
> >
> > I asked internally and it seems that our changes to support x-compile do
> not go as far as producing debian/ubuntu packages, so they aren't ready for
> upstreaming, unfortunately.
> >
> > Thanks,
> > Chris.
> >
> >> On Oct 31, 2019, at 2:23 PM, Damjan Marion  wrote:
> >>
> >>
> >> That is just copy paste from Ben’s command line.
> >>
> >> As. i wrote, this is just about presenting the concept...
> >>
> >>> On 31 Oct 2019, at 18:39, Christian Hopps  wrote:
> >>>
> >>> Requiring root privileges for building software is pretty atypical and
> limiting, even for x-compiling.
> >>>
> >>> Thanks,
> >>> Chris.
> >>>
>  On Oct 31, 2019, at 12:45 PM, Damjan Marion via Lists.Fd.Io
>  wrote:
> 
> 
>  So what people think about following approach for cross-compilation:
> 
>  Patch:
>  https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.fd.io_r
>  _c_vpp_-
> 2B_23153=DwIFaQ=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YO
> 
> vfL3IkGduCfk9LbZMPOAecQGDzWV0=G81bDw8Lj77hbqAq7cdeDRXZev-
> ZVMwh0n_
>  oePUFrlQ=Lk5OD0CVQuGPdj-h0TfK-m8NMJj0Zci0EhjBcQxYn2E=
> 
>  At the moment this is just to show idea, so it works only for
>  x86-to-x86 and between ubuntu versions only but it can be easily
> extended to scenarios like build vpp packages for AArch64 centos7 on
> ubuntu x86 host.
> 
>  export CROSS_DISTRO_NAME=ubuntu
>  exprot CROSS_DISTRO_VER=18.04
>  export CROSS_ARCH=x86_64
>  export CROSS_TARGET=release
> 
>  $ make cross-env-init
> 
>  this is executed once, it creates docker container named "vpp-dev-env-
> ubuntu-18.04-x86_64”, install all dependencies including DPDK and other
> libs”
> 
>  $ make cross-env-update
> 
>  updates cross-env to latest packages and ext-deps
> 
>  $ make cross-shell
> 
>  starts shell inside cross-env
> 
>  $ make cross-package
> 
>  builds VPP and creates packages:
> 
>  ls -la build/ubuntu-18.04-x86_64-release total 11716
>  drwxr-xr-x  3 root root4096 Oct 31 17:30 .
>  drwxrwxr-x  4 damarion damarion4096 Oct 31 17:36 ..
>  drwxr-xr-x 22 root root4096 Oct 31 17:30 build
>  -rw-r--r--  1 root root  190124 Oct 31 17:30 libvppinfra_20.01-
> rc0~555-g7f980bcfe_amd64.deb
>  -rw-r--r--  1 root root  145056 Oct 31 17:30 
>  libvppinfra-dev_20.01-
> rc0~555-g7f980bcfe_amd64.deb
>  -rw-r--r--  1 root root   23140 Oct 31 17:30 
>  python3-vpp-api_20.01-
> rc0~555-g7f980bcfe_amd64.deb
>  -rw-r--r--  1 root root8815 Oct 31 17:30 vpp_20.01-rc0~555-
> g7f980bcfe_amd64.buildinfo
>  -rw-r--r--  1 root root4147 Oct 31 17:30 vpp_20.01-rc0~555-
> g7f980bcfe_amd64.changes
>  -rw-r--r--  1 root root 3674024 Oct 31 17:30 vpp_20.01-rc0~555-
> g7f980bcfe_amd64.deb
>  -rw-r--r--  1 root root   23104 Oct 31 17:30 
>  vpp-api-python_20.01-
> rc0~555-g7f980bcfe_amd64.deb
>  -rw-r--r--  1 root root  719712 Oct 31 17:30 
>  vpp-dbg_20.01-rc0~555-
> g7f980bcfe_amd64.deb
>  -rw-r--r--  1 root root  915744 Oct 31 17:30 
>  vpp-dev_20.01-rc0~555-
> g7f980bcfe_amd64.deb
>  -rw-r--r--  1 root root 3113820 Oct 31 17:30 
>  

Re: [EXT] [vpp-dev] cross-compilation support

[Nitin Saxena]

Hi Damjan,

>> I don’t understand why you cannot specify your preferred set of args 
>> directly in the buildroot makefile...
We can do that. I thought (1) and (2) is the recommended way of cross compiling 
VPP. So if I understand correctly all Makefiles in build-data/platforms/ are 
intended for native compilation and not for cross

Thanks,
Nitin

> -Original Message-
> From: Damjan Marion 
> Sent: Friday, November 1, 2019 4:10 PM
> To: Nitin Saxena 
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [EXT] [vpp-dev] cross-compilation support
> 
> 
> 
> > On 1 Nov 2019, at 09:57, Nitin Saxena  wrote:
> >
> > Hi Damjan,
> >
> >>> My comment bellow is that only 1) is really needed to build VPP,
> >>> people can decide to use own build system like buildroot or yocto
> >>> and invoke cmake directly, and completely ignore 2) and 3
> > Correct me if I am wrong, purpose of build-data/platforms/*.mk  (except
> vpp.mk) can be to configure ebuild for cross compilation. make build[-
> release] PLATFORM="aarch64..." was recommended way for cross
> compilation in VPP? If yes, is it OK to use 1) and 2) for cross compilation?
> 
> Personally i would not recommend to use 2) for cross-compilation because of
> reasons i explained in my previous emails.
> 
> >
> > We internally uses (1) and (2) for our buildroot based cross compilation for
> octeontx. Hence I asked for the capability to pass vpp_cmake_args from
> build-data/platforms/octeontx.mk because I found it as a cleaner approach.
> Comments?
> 
> I don't see a point of doing that. you just add another moving part to your
> build system.
> I don’t understand why you cannot specify your preferred set of args directly
> in the buildroot makefile...
> 
> 
> >
> > Thanks,
> > Nitin
> >
> >> -Original Message-
> >> From: Damjan Marion 
> >> Sent: Friday, November 1, 2019 1:34 AM
> >> To: Damjan Marion 
> >> Cc: Nitin Saxena ; vpp-dev@lists.fd.io
> >> Subject: Re: [EXT] [vpp-dev] cross-compilation support
> >>
> >>
> >> Let me clarify a bit more, as i can see that this still may be confusing:
> >>
> >> 1) VPP is using cmake to build binaries and packages, all cmake
> >> related stuff is in src/.
> >>
> >> 2) there is build-root/Makefile and files in build-data/* which are
> >> part of old build system called ebuild
> >> - ebuild is very complex set of make scripts which have similar
> >> functionality like buildroot, it was able to build kernel, toolchain
> >> , userspace tools and libraries
> >> - today we don't use much of ebuild, it is just used to run VPP cmake
> >> prioject in the right directory with he right set of command line
> >> arguments
> >>
> >> 3) Other Makefiles
> >>  - top level makefile
> >>  - external deps makefiles in build/external/
> >>
> >> My comment bellow is that only 1) is really needed to build VPP,
> >> people can decide to use own build system like buildroot or yocto and
> >> invoke cmake directly, and completely ignore 2) and 3). In such case
> >> selected build system also needs to take care for dependencies like DPDK.
> >>
> >> ebuild 2) have excellent cross compilation support for building
> >> target images when those images contain everything from kernel and
> >> libraries to vpp but it is not appropriate tool for creating
> >> distribution packages, i.e. creating Centos ARM rpms on Ubuntu x86
> >> machine. That is the reason why i tried to see if we can use docker
> >> instead to be able to build cross-arch, cross-distro or cross- 
> >> distro-version
> (or all 3 together) packages.
> >> While my patch is incomplete, it looks to me like this approach will work.
> >>
> >> My patch does native compilation only if TARGET_QUAD is equal to
> >> HOST_QUAD, where QUAD is:
> >> - distro name (ubuntu, centos)
> >> - distro version (18.04, 7.3)
> >> - arch (x86_64, aarch64, ...)
> >> - platform (generic, thunderx, ….)
> >>
> >> it also support multiple build_types (release, debug, gcov, … )
> >>
> >> At this point it is just early draft, but it shows some basic
> >> mechanics used to produce both native and cross packages.
> >>
> >>
> >>> On 31 Oct 2019, at 20:32, Damjan Marion via Lists.Fd.Io
> >>  wrote:
> >>>
> >>> I have similar scheme on my mind, where you can have platform
> >>> specific
> >> m

Re: [EXT] [vpp-dev] cross-compilation support

[Nitin Saxena]

Understood. 

Thanks
Nitin

> -Original Message-
> From: Damjan Marion 
> Sent: Friday, November 1, 2019 5:13 PM
> To: Nitin Saxena 
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [EXT] [vpp-dev] cross-compilation support
> 
> 
> 
> > On 1 Nov 2019, at 12:14, Nitin Saxena  wrote:
> >
> > Hi Damjan,
> >
> >>> I don’t understand why you cannot specify your preferred set of args
> directly in the buildroot makefile...
> > We can do that. I thought (1) and (2) is the recommended way of cross
> > compiling VPP. So if I understand correctly all Makefiles in
> > build-data/platforms/ are intended for native compilation and not for
> > cross
> 
> There ere 2 different types of cross-compilation in my view:
> 
> 1. Platform images - where you build everything from scratch, toolchain,
> kernel, glibc, busybox - that is the case for which 2) was used in the past, 
> and
> it can be still used but somebody will need to create and maintain bunch of
> build-data/packages/*.mk files and those days it is not feasible that such
> thing will happen, as there are few well maintained open source projects
> which provide same functionality like buildroot, yocto, openwrt ...
> 
> 2. Distro packages - where vpp is crosscompiled, linked against distro
> provided shared libraries and packaged for that distro. That is something 2)
> cannot do properly, and that is the reason why i was looking for alternative
> solution with docker.
> 
> So in my view 2) provides maintenance overhead but not lot of value for our
> use cases, so we should replace it with something much simpler to maintain.
> 
> >
> > Thanks,
> > Nitin
> >
> >> -Original Message-
> >> From: Damjan Marion 
> >> Sent: Friday, November 1, 2019 4:10 PM
> >> To: Nitin Saxena 
> >> Cc: vpp-dev@lists.fd.io
> >> Subject: Re: [EXT] [vpp-dev] cross-compilation support
> >>
> >>
> >>
> >>>> On 1 Nov 2019, at 09:57, Nitin Saxena  wrote:
> >>>
> >>> Hi Damjan,
> >>>
> >>>>> My comment bellow is that only 1) is really needed to build VPP,
> >>>>> people can decide to use own build system like buildroot or yocto
> >>>>> and invoke cmake directly, and completely ignore 2) and 3
> >>> Correct me if I am wrong, purpose of build-data/platforms/*.mk
> >>> (except
> >> vpp.mk) can be to configure ebuild for cross compilation. make
> >> build[- release] PLATFORM="aarch64..." was recommended way for cross
> >> compilation in VPP? If yes, is it OK to use 1) and 2) for cross 
> >> compilation?
> >>
> >> Personally i would not recommend to use 2) for cross-compilation
> >> because of reasons i explained in my previous emails.
> >>
> >>>
> >>> We internally uses (1) and (2) for our buildroot based cross
> >>> compilation for
> >> octeontx. Hence I asked for the capability to pass vpp_cmake_args
> >> from build-data/platforms/octeontx.mk because I found it as a cleaner
> approach.
> >> Comments?
> >>
> >> I don't see a point of doing that. you just add another moving part
> >> to your build system.
> >> I don’t understand why you cannot specify your preferred set of args
> >> directly in the buildroot makefile...
> >>
> >>
> >>>
> >>> Thanks,
> >>> Nitin
> >>>
> >>>> -Original Message-
> >>>> From: Damjan Marion 
> >>>> Sent: Friday, November 1, 2019 1:34 AM
> >>>> To: Damjan Marion 
> >>>> Cc: Nitin Saxena ; vpp-dev@lists.fd.io
> >>>> Subject: Re: [EXT] [vpp-dev] cross-compilation support
> >>>>
> >>>>
> >>>> Let me clarify a bit more, as i can see that this still may be confusing:
> >>>>
> >>>> 1) VPP is using cmake to build binaries and packages, all cmake
> >>>> related stuff is in src/.
> >>>>
> >>>> 2) there is build-root/Makefile and files in build-data/* which are
> >>>> part of old build system called ebuild
> >>>> - ebuild is very complex set of make scripts which have similar
> >>>> functionality like buildroot, it was able to build kernel,
> >>>> toolchain , userspace tools and libraries
> >>>> - today we don't use much of ebuild, it is just used to run VPP
> >>>> cmake prioject in the right directory with he right set of command
>

Re: [vpp-dev] efficient use of DPDK

[Nitin Saxena]

Hi Jerome,

I have query unrelated to the original thread. 

>> There are other examples (lookaside and inline)
By inline do you mean "Inline IPSEC"? Could you please elaborate what you meant 
by inline offload in VPP?

Thanks,
Nitin

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Jerome Tollet
> via Lists.Fd.Io
> Sent: Wednesday, December 4, 2019 9:00 PM
> To: Thomas Monjalon ; Damjan Marion
> 
> Cc: vpp-dev@lists.fd.io
> Subject: [EXT] Re: [vpp-dev] efficient use of DPDK
> 
> External Email
> 
> --
> Hi Thomas,
> I strongly disagree with your conclusions from this discussion:
> 1) Yes, VPP made the choice of not being DPDK dependent BUT certainly not
> at the cost of performance. (It's actually the opposite ie AVF driver)
> 2) VPP is NOT exclusively CPU centric. I gave you the example of crypto
> offload based on Intel QAT cards (lookaside). There are other examples
> (lookaside and inline)
> 3) Plugins are free to use any sort of offload (and they do).
> 
> Jerome
> 
> Le 04/12/2019 15:19, « vpp-dev@lists.fd.io au nom de Thomas Monjalon »
>  a écrit :
> 
> 03/12/2019 20:01, Damjan Marion:
> > On 3 Dec 2019, at 17:06, Thomas Monjalon wrote:
> > > 03/12/2019 13:12, Damjan Marion:
> > >> On 3 Dec 2019, at 09:28, Thomas Monjalon wrote:
> > >>> 03/12/2019 00:26, Damjan Marion:
> >  On 2 Dec 2019, at 23:35, Thomas Monjalon wrote:
> > > VPP has a buffer called vlib_buffer_t, while DPDK has rte_mbuf.
> > > Are there some benchmarks about the cost of converting, from
> one format
> > > to the other one, during Rx/Tx operations?
> > 
> >  We are benchmarking both dpdk i40e PMD performance and native
> VPP AVF driver performance and we are seeing significantly better
> performance with native AVF.
> >  If you taake a look at [1] you will see that DPDK i40e driver 
> provides
> 18.62 Mpps and exactly the same test with native AVF driver is giving us
> arounf 24.86 Mpps.
> > > [...]
> > 
> > > So why not improving DPDK integration in VPP to make it faster?
> > 
> >  Yes, if we can get freedom to use parts of DPDK we want instead of
> being forced to adopt whole DPDK ecosystem.
> >  for example, you cannot use dpdk drivers without using EAL,
> mempool, rte_mbuf... rte_eal_init is monster which I was hoping that it will
> disappear for long time...
> 
> As stated below, I take this feedback, thanks.
> However it won't change VPP choice of not using rte_mbuf natively.
> 
> [...]
> > >> At the moment we have good coverage of native drivers, and still
> there is a option for people to use dpdk. It is now mainly up to driver 
> vendors
> to decide if they are happy with performance they wil get from dpdk pmd or
> they want better...
> > >
> > > Yes it is possible to use DPDK in VPP with degraded performance.
> > > If an user wants best performance with VPP and a real NIC,
> > > a new driver must be implemented for VPP only.
> > >
> > > Anyway real performance benefits are in hardware device offloads
> > > which will be hard to implement in VPP native drivers.
> > > Support (investment) would be needed from vendors to make it
> happen.
> > > About offloads, VPP is not using crypto or compression drivers
> > > that DPDK provides (plus regex coming).
> >
> > Nice marketing pitch for your company :)
> 
> I guess you mean Mellanox has a good offloads offering.
> But my point is about the end of Moore's law,
> and the offload trending of most of device vendors.
> However I truly respect the choice of avoiding device offloads.
> 
> > > VPP is a CPU-based packet processing software.
> > > If users want to leverage hardware device offloads,
> > > a truly DPDK-based software is required.
> > > If I understand well your replies, such software cannot be VPP.
> >
> > Yes, DPDK is centre of the universe/
> 
> DPDK is where most of networking devices are supported in userspace.
> That's all.
> 
> 
> > So Dear Thomas, I can continue this discussion forever, but that is not
> something I'm going to do as it started to be trolling contest.
> 
> I agree
> 
> > I can understand that you may be passionate about you project and that
> you maybe think that it is the greatest thing after sliced bread, but please
> allow that other people have different opinion. Instead of giving the lessons
> to other people what they should do, if you are interested for dpdk to be
> better consumed, please take a feedback provided to you. I assume that you
> are interested as you showed up on this mailing list, if not there was no
> reason for starting this thread in the first place.
> 
> Thank you for the feedbacks, this discussion was required:
> 1/ it gives more motivation to improve EAL API
> 2/ it confirms the 

Re: [vpp-dev] efficient use of DPDK

[Nitin Saxena]

Hi Jerome,

Thanks for the clarification

Regards,
Nitin

> -Original Message-
> From: Jerome Tollet (jtollet) 
> Sent: Wednesday, December 4, 2019 11:30 PM
> To: Nitin Saxena ; Thomas Monjalon
> ; Damjan Marion 
> Cc: vpp-dev@lists.fd.io
> Subject: [EXT] Re: [vpp-dev] efficient use of DPDK
> 
> External Email
> 
> --
> Hi Nitin,
> 
> I am not necessarily speaking about Inline IPSec. I was just saying that VPP
> lets you the choice to do both inline and lookaside types of offload.
> 
> Here is a public example of inline acceleration:
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.intel.com_content_dam_www_programmable_us_en_pdfs_litera
> ture_wp_wp-2D01295-2Dhcl-2Dsegment-2Drouting-2Dover-2Dipv6-
> 2Dacceleration-2Dusing-2Dintel-2Dfpga-2Dprogrammable-2Dacceleration-
> 2Dcard-
> 2Dn3000.pdf=DwIGaQ=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YO
> vfL3IkGduCfk9LbZMPOAecQGDzWV0=cegWttxd0q-
> zJRoby35GO4izb4G9cJy-87BbbFqgeu8=7bej1I-
> Gyc_F0C_RWhtavWmKiyNbC1m-0tkkrG005r0=
> 
> Jerome
> 
> 
> 
> Le 04/12/2019 18:19, « Nitin Saxena »  a écrit :
> 
> 
> 
> Hi Jerome,
> 
> 
> 
> I have query unrelated to the original thread.
> 
> 
> 
> >> There are other examples (lookaside and inline)
> 
> By inline do you mean "Inline IPSEC"? Could you please elaborate what you
> meant by inline offload in VPP?
> 
> 
> 
> Thanks,
> 
> Nitin
> 
> 
> 
> > -Original Message-
> 
> > From: vpp-dev@lists.fd.io  On Behalf Of Jerome
> Tollet
> 
> > via Lists.Fd.Io
> 
> > Sent: Wednesday, December 4, 2019 9:00 PM
> 
> > To: Thomas Monjalon ; Damjan Marion
> 
> > 
> 
> > Cc: vpp-dev@lists.fd.io
> 
> > Subject: [EXT] Re: [vpp-dev] efficient use of DPDK
> 
> >
> 
> > External Email
> 
> >
> 
> > --
> 
> > Hi Thomas,
> 
> > I strongly disagree with your conclusions from this discussion:
> 
> > 1) Yes, VPP made the choice of not being DPDK dependent BUT certainly
> not
> 
> > at the cost of performance. (It's actually the opposite ie AVF driver)
> 
> > 2) VPP is NOT exclusively CPU centric. I gave you the example of crypto
> 
> > offload based on Intel QAT cards (lookaside). There are other examples
> 
> > (lookaside and inline)
> 
> > 3) Plugins are free to use any sort of offload (and they do).
> 
> >
> 
> > Jerome
> 
> >
> 
> > Le 04/12/2019 15:19, « vpp-dev@lists.fd.io au nom de Thomas Monjalon
> »
> 
> >  a écrit :
> 
> >
> 
> > 03/12/2019 20:01, Damjan Marion:
> 
> > > On 3 Dec 2019, at 17:06, Thomas Monjalon wrote:
> 
> > > > 03/12/2019 13:12, Damjan Marion:
> 
> > > >> On 3 Dec 2019, at 09:28, Thomas Monjalon wrote:
> 
> > > >>> 03/12/2019 00:26, Damjan Marion:
> 
> > > >>>> On 2 Dec 2019, at 23:35, Thomas Monjalon wrote:
> 
> > > >>>>> VPP has a buffer called vlib_buffer_t, while DPDK has
> rte_mbuf.
> 
> > > >>>>> Are there some benchmarks about the cost of converting,
> from
> 
> > one format
> 
> > > >>>>> to the other one, during Rx/Tx operations?
> 
> > > >>>>
> 
> > > >>>> We are benchmarking both dpdk i40e PMD performance and
> native
> 
> > VPP AVF driver performance and we are seeing significantly better
> 
> > performance with native AVF.
> 
> > > >>>> If you taake a look at [1] you will see that DPDK i40e driver
> provides
> 
> > 18.62 Mpps and exactly the same test with native AVF driver is giving us
> 
> > arounf 24.86 Mpps.
> 
> > > > [...]
> 
> > > >>>>
> 
> > > >>>>> So why not improving DPDK integration in VPP to make it
> faster?
> 
> > > >>>>
> 
> > > >>>> Yes, if we can get freedom to use parts of DPDK we want
> instead of
> 
> > being forced to adopt whole DPDK ecosystem.
> 
> > > >>>> for example, you cannot use dpdk drivers without using EAL,
> 
> > mempool, rte_mbuf... rte_e

Re: [vpp-dev] [EXT] [csit-dev] crypto_ia32 -> crypto_native

[Nitin Saxena]

+1

Thanks,
Nitin

> -Original Message-
> From: csit-...@lists.fd.io  On Behalf Of Damjan Marion
> via Lists.Fd.Io
> Sent: Tuesday, January 28, 2020 2:24 AM
> To: vpp-dev ; csit-...@lists.fd.io
> Cc: csit-...@lists.fd.io
> Subject: [EXT] [csit-dev] crypto_ia32 -> crypto_native
> 
> External Email
> 
> --
> 
> Folks,
> 
> To avoid code duplication i would like to rename crypto_ia32 plugin into
> crypto_native. Reason is adding ARMv8 support which seems to be very
> similar to IA32 in terms of implementing CBC and GCM.
> 
> Any objections or caveats?
> 
> --
> Damjan
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15266): https://lists.fd.io/g/vpp-dev/message/15266
Mute This Topic: https://lists.fd.io/mt/70214116/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Regarding unix_epoll_input on worker cores

[Nitin Saxena]

Hi,

I understand the purpose of unix-epoll-input PRE_INPUT node on main thread but 
I am not able to envision why it is required on worker core.
The way I see it all clib_file_xx() operations are on main core.

It will be helpful if any scenario can be explained where presence of 
unix-epoll-input() on worker core solves the purpose

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15340): https://lists.fd.io/g/vpp-dev/message/15340
Mute This Topic: https://lists.fd.io/mt/70991597/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] Re: [vpp-dev] Regarding unix_epoll_input on worker cores

[Nitin Saxena]

Thanks Damjan.

Then I think there will be plan to change default state of INPUT nodes from 
POLLING to INTERRUPTIBLE. I think it is required to enable this feature at some 
point.

Thanks,
Nitin

From: Damjan Marion 
Sent: Wednesday, February 5, 2020 2:00 PM
To: Nitin Saxena 
Cc: vpp-dev 
Subject: [EXT] Re: [vpp-dev] Regarding unix_epoll_input on worker cores

External Email

There are 2 use cases:

- delivering RX queue interrupts directpy to the worker which handles specific 
queue
- waking up worker thread over eventfd

None of them is used today, but it should be at some point...

--
DAmjan


On 5 Feb 2020, at 08:39, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi,

I understand the purpose of unix-epoll-input PRE_INPUT node on main thread but 
I am not able to envision why it is required on worker core.
The way I see it all clib_file_xx() operations are on main core.

It will be helpful if any scenario can be explained where presence of 
unix-epoll-input() on worker core solves the purpose

Thanks,
Nitin


--
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15342): https://lists.fd.io/g/vpp-dev/message/15342
Mute This Topic: https://lists.fd.io/mt/70992277/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] Re: [vpp-dev] Regarding unix_epoll_input on worker cores

[Nitin Saxena]

>> No, there is no need to change default state. State can be changed during 
>> runtime...
That part I understand. I realized that AVF supports MSIX interrupts hence 
native plugin can easily leverage this feature (provided hardware supports MSIX 
per RQ)

Thanks,
Nitin

From: vpp-dev@lists.fd.io  On Behalf Of Damjan Marion via 
Lists.Fd.Io
Sent: Wednesday, February 5, 2020 2:32 PM
To: Nitin Saxena 
Cc: vpp-dev@lists.fd.io
Subject: Re: [EXT] Re: [vpp-dev] Regarding unix_epoll_input on worker cores

No, there is no need to change default state. State can be changed during 
runtime...
—
Damjan


On 5 Feb 2020, at 09:54, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Thanks Damjan.

Then I think there will be plan to change default state of INPUT nodes from 
POLLING to INTERRUPTIBLE. I think it is required to enable this feature at some 
point.

Thanks,
Nitin

From: Damjan Marion mailto:dmar...@me.com>>
Sent: Wednesday, February 5, 2020 2:00 PM
To: Nitin Saxena mailto:nsax...@marvell.com>>
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: [EXT] Re: [vpp-dev] Regarding unix_epoll_input on worker cores

External Email

There are 2 use cases:

- delivering RX queue interrupts directpy to the worker which handles specific 
queue
- waking up worker thread over eventfd

None of them is used today, but it should be at some point...

--
DAmjan



On 5 Feb 2020, at 08:39, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi,

I understand the purpose of unix-epoll-input PRE_INPUT node on main thread but 
I am not able to envision why it is required on worker core.
The way I see it all clib_file_xx() operations are on main core.

It will be helpful if any scenario can be explained where presence of 
unix-epoll-input() on worker core solves the purpose

Thanks,
Nitin

--
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15344): https://lists.fd.io/g/vpp-dev/message/15344
Mute This Topic: https://lists.fd.io/mt/70992277/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] Regarding unix_epoll_input on worker cores

[Nitin Saxena]

Yeah I got the clear picture. Eventually MSIX can be routed to worker cores 
which can wake on eventfd, hence unix-epoll-input node required.

Thanks,
Nitin

From: Damjan Marion 
Sent: Wednesday, February 5, 2020 3:08 PM
To: Nitin Saxena 
Cc: vpp-dev@lists.fd.io
Subject: Re: [EXT] [vpp-dev] Regarding unix_epoll_input on worker cores


Yes, but you keep interrupts disabled while in polling mode. At the point when 
traffic load goes down, you can switch to interrupt mode by switching MSI-X 
interrupts on.
We already have that functionality today, only issue is that interrupts are 
routed to main thread, so it may happen that worker is sleeping and that will 
cause delay in taking packet from the rx queue handled by that worker...


On 5 Feb 2020, at 10:24, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

>> No, there is no need to change default state. State can be changed during 
>> runtime...
That part I understand. I realized that AVF supports MSIX interrupts hence 
native plugin can easily leverage this feature (provided hardware supports MSIX 
per RQ)

Thanks,
Nitin

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan Marion 
via Lists.Fd.Io
Sent: Wednesday, February 5, 2020 2:32 PM
To: Nitin Saxena mailto:nsax...@marvell.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [EXT] Re: [vpp-dev] Regarding unix_epoll_input on worker cores

No, there is no need to change default state. State can be changed during 
runtime...
—
Damjan


On 5 Feb 2020, at 09:54, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Thanks Damjan.

Then I think there will be plan to change default state of INPUT nodes from 
POLLING to INTERRUPTIBLE. I think it is required to enable this feature at some 
point.

Thanks,
Nitin

From: Damjan Marion mailto:dmar...@me.com>>
Sent: Wednesday, February 5, 2020 2:00 PM
To: Nitin Saxena mailto:nsax...@marvell.com>>
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: [EXT] Re: [vpp-dev] Regarding unix_epoll_input on worker cores

External Email

There are 2 use cases:

- delivering RX queue interrupts directpy to the worker which handles specific 
queue
- waking up worker thread over eventfd

None of them is used today, but it should be at some point...

--
DAmjan




On 5 Feb 2020, at 08:39, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi,

I understand the purpose of unix-epoll-input PRE_INPUT node on main thread but 
I am not able to envision why it is required on worker core.
The way I see it all clib_file_xx() operations are on main core.

It will be helpful if any scenario can be explained where presence of 
unix-epoll-input() on worker core solves the purpose

Thanks,
Nitin

--
Damjan



--
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15346): https://lists.fd.io/g/vpp-dev/message/15346
Mute This Topic: https://lists.fd.io/mt/70992570/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] Regarding buffers-per-numa parameter

[Nitin Saxena]

HI Chethan,

Your packet trace shows that packet data is all 0 and that’s why you are 
running into l3 mac mismatch.
I am guessing something messed with IOMMU due to which translation is not 
happening. Although packet length is correct.
You can try out AVF plugin to iron out where problem exists, in dpdk plugin or 
vlib

Thanks,
Nitin

From: chetan bhasin 
Sent: Tuesday, February 18, 2020 12:50 PM
To: me 
Cc: Nitin Saxena ; vpp-dev 
Subject: Re: [EXT] [vpp-dev] Regarding buffers-per-numa parameter

Hi,
One more finding related to intel nic and number of buffers (537600)

vpp branch
driver
card
buffers
Traffic
Err
stable/1908
uio_pci_genric
X722(10G)
537600
 Working

stable/1908
vfio-pci
XL710(40G)
537600
Not Working
l3 mac mismatch
stable/2001
uio_pci_genric
X722(10G)
537600
 Working

stable/2001
vfio-pci
XL710(40G)
537600
 Working



Thanks,
Chetan

On Mon, Feb 17, 2020 at 7:17 PM chetan bhasin via 
Lists.Fd.Io<https://urldefense.proofpoint.com/v2/url?u=http-3A__Lists.Fd.Io=DwMFaQ=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0=qxJrqbz5sNlCrzJTOZjaJ0jHeaW077bX6ZxmV308jfg=ffS1Y8GHllzjueMUVW31gwrVEIK1HVSNTKk2yA-VjG8=>
 mailto:gmail@lists.fd.io>> wrote:
Hi Nitin,

https://github.com/FDio/vpp/commits/stable/2001/src/vlib<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_FDio_vpp_commits_stable_2001_src_vlib=DwMFaQ=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0=qxJrqbz5sNlCrzJTOZjaJ0jHeaW077bX6ZxmV308jfg=LljqKCmXwjl4uzuLM_oB-jhjYV5xVGFpHPDomTZwKAU=>
As per stable/2001 branch , the given change is checked-in around Oct 28 2019.

df0191ead2cf39611714b6603cdc5bdddc445b57 is previous commit of 
b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897?
Yes (branch vpp 20.01)

Thanks,
Chetan Bhasin

On Mon, Feb 17, 2020 at 5:33 PM Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:
Hi Damjan,

>> if you read Chetan’s email bellow, you will see that this one is already 
>> excluded…
Sorry I missed that part. After seeing diffs between stable/1908 and 
stable/2001, commit: b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897 is the only 
visible git commit in dpdk plugin which is playing with mempool buffers. If it 
does not solve the problem then I suspect problem lies outside dpdk plugin. I 
am guessing DPDK-19.08 is being used here with VPP-19.08

Hi Chetan,
> > 3) I took previous commit of  "vlib: don't use vector for keeping buffer
> indices in the pool " ie "df0191ead2cf39611714b6603cdc5bdddc445b57" :
> Everything looks fine with Buffers 537600.
In which branch, Commit: df0191ead2cf39611714b6603cdc5bdddc445b57 is previous 
commit of b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897?

Thanks,
Nitin
> -Original Message-
> From: Damjan Marion mailto:dmar...@me.com>>
> Sent: Monday, February 17, 2020 3:47 PM
> To: Nitin Saxena mailto:nsax...@marvell.com>>
> Cc: chetan bhasin 
> mailto:chetan.bhasin...@gmail.com>>; 
> vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
> Subject: Re: [EXT] [vpp-dev] Regarding buffers-per-numa parameter
>
>
> Dear Nitin,
>
> if you read Chetan’s email bellow, you will see that this one is already
> excluded…
>
> Also, it will not be easy to explain how this patch blows tx function in dpdk
> mlx5 pmd…
>
> —
> Damjan
>
> > On 17 Feb 2020, at 11:12, Nitin Saxena 
> > mailto:nsax...@marvell.com>> wrote:
> >
> > Hi Prashant/Chetan,
> >
> > I would try following change first to solve the problem in 1908
> >
> > commit b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897b
> > Author: Damjan Marion mailto:damar...@cisco.com>>
> > Date:   Tue Mar 12 18:14:15 2019 +0100
> >
> > vlib: don't use vector for keeping buffer indices in
> >
> > Type: refactor
> >
> > Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4
> > Signed-off-by: Damjan Marion 
> > damar...@cisco.com<mailto:damar...@cisco.com>
> >
> > You can also try copying src/plugins/dpdk/buffer.c from stable/2001
> branch to stable/1908
> >
> > Thanks,
> > Nitin
> >
> > From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> > mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan
> Marion via 
> Lists.Fd.Io<https://urldefense.proofpoint.com/v2/url?u=http-3A__Lists.Fd.Io=DwMFaQ=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0=qxJrqbz5sNlCrzJTOZjaJ0jHeaW077bX6ZxmV308jfg=ffS1Y8GHllzjueMUVW31gwrVEIK1HVSNTKk2yA-VjG8=>
> > Sent: Monday, February 17, 2020 1:52 PM
> > To: chetan bhasin 
> > mailto:chetan.bhasin...@gmail.com>>
> > Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
> > Subject: [EXT] Re: [vpp-dev] Regarding buffers-per-numa parameter
> >
> > External Email
> >
> > On 17 Feb 2020

Re: [EXT] [vpp-dev] Regarding buffers-per-numa parameter

[Nitin Saxena]

Hi Damjan,

>> if you read Chetan’s email bellow, you will see that this one is already 
>> excluded…
Sorry I missed that part. After seeing diffs between stable/1908 and 
stable/2001, commit: b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897 is the only 
visible git commit in dpdk plugin which is playing with mempool buffers. If it 
does not solve the problem then I suspect problem lies outside dpdk plugin. I 
am guessing DPDK-19.08 is being used here with VPP-19.08

Hi Chetan,
> > 3) I took previous commit of  "vlib: don't use vector for keeping buffer
> indices in the pool " ie "df0191ead2cf39611714b6603cdc5bdddc445b57" :
> Everything looks fine with Buffers 537600.
In which branch, Commit: df0191ead2cf39611714b6603cdc5bdddc445b57 is previous 
commit of b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897?

Thanks,
Nitin
> -Original Message-
> From: Damjan Marion 
> Sent: Monday, February 17, 2020 3:47 PM
> To: Nitin Saxena 
> Cc: chetan bhasin ; vpp-dev@lists.fd.io
> Subject: Re: [EXT] [vpp-dev] Regarding buffers-per-numa parameter
> 
> 
> Dear Nitin,
> 
> if you read Chetan’s email bellow, you will see that this one is already
> excluded…
> 
> Also, it will not be easy to explain how this patch blows tx function in dpdk
> mlx5 pmd…
> 
> —
> Damjan
> 
> > On 17 Feb 2020, at 11:12, Nitin Saxena  wrote:
> >
> > Hi Prashant/Chetan,
> >
> > I would try following change first to solve the problem in 1908
> >
> > commit b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897b
> > Author: Damjan Marion 
> > Date:   Tue Mar 12 18:14:15 2019 +0100
> >
> > vlib: don't use vector for keeping buffer indices in
> >
> > Type: refactor
> >
> > Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4
> > Signed-off-by: Damjan Marion damar...@cisco.com
> >
> > You can also try copying src/plugins/dpdk/buffer.c from stable/2001
> branch to stable/1908
> >
> > Thanks,
> > Nitin
> >
> > From: vpp-dev@lists.fd.io  On Behalf Of Damjan
> Marion via Lists.Fd.Io
> > Sent: Monday, February 17, 2020 1:52 PM
> > To: chetan bhasin 
> > Cc: vpp-dev@lists.fd.io
> > Subject: [EXT] Re: [vpp-dev] Regarding buffers-per-numa parameter
> >
> > External Email
> >
> > On 17 Feb 2020, at 07:37, chetan bhasin 
> wrote:
> >
> > Bottom line is stable/vpp 908 does not work with higher number of buffers
> but stable/vpp2001 does. Could you please advise which area we can look at
> ,as it would be difficult for us to move to vpp2001 at this time.
> >
> > I really don’t have idea what caused this problem to disappear.
> > You may try to use “git bisect” to find out which commit fixed it….
> >
> > —
> > Damjan
> >
> >
> >
> > On Mon, Feb 17, 2020 at 11:01 AM chetan bhasin via Lists.Fd.Io
>  wrote:
> > Thanks Damjan for the reply!
> >
> > Following are my observations on Intel X710/XL710 pci-
> > 1) I took latest code base from stable/vpp19.08  : Seeing error as "
> ethernet-input l3 mac mismatch"
> > With Buffers 537600
> > vpp# show buffers
> |
> > Pool NameIndex NUMA  Size  Data Size  Total  Avail  Cached   
> > Used
> > default-numa-0 0 0   2496 2048   537600 510464   1319
> > 25817
> > default-numa-1 1 1   2496 2048   537600 528896390
> > 8314
> >
> > vpp# show hardware-interfaces
> >   NameIdx   Link  Hardware
> > BondEthernet0  3 up   BondEthernet0
> >   Link speed: unknown
> >   Ethernet address 3c:fd:fe:b5:5e:40
> > FortyGigabitEthernet12/0/0 1 up   FortyGigabitEthernet12/0/0
> >   Link speed: 40 Gbps
> >   Ethernet address 3c:fd:fe:b5:5e:40
> >   Intel X710/XL710 Family
> > carrier up full duplex mtu 9206
> > flags: admin-up pmd rx-ip4-cksum
> > rx: queues 16 (max 320), desc 1024 (min 64 max 4096 align 32)
> > tx: queues 16 (max 320), desc 4096 (min 64 max 4096 align 32)
> > pci: device 8086:1583 subsystem 8086:0001 address :12:00.00 numa
> 0
> > max rx packet len: 9728
> > promiscuous: unicast off all-multicast on
> > vlan offload: strip off filter off qinq off
> > rx offload avail:  vlan-strip ipv4-cksum udp-cksum tcp-cksum qinq-strip
> >outer-ipv4-cksum vlan-filter vlan-extend jumbo-frame
> >scatter keep-crc
> > rx offload active: ipv4-cksum
> > tx offload avail:  vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-ck

Re: [EXT] [vpp-dev] Regarding buffers-per-numa parameter

[Nitin Saxena]

>> I am guessing DPDK-19.08 is being used here with VPP-19.08
Typo, dpdk-19.05 and not dpdk-19.08

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Nitin Saxena
> Sent: Monday, February 17, 2020 5:34 PM
> To: Damjan Marion 
> Cc: chetan bhasin ; vpp-dev@lists.fd.io
> Subject: Re: [EXT] [vpp-dev] Regarding buffers-per-numa parameter
> 
> Hi Damjan,
> 
> >> if you read Chetan’s email bellow, you will see that this one is
> >> already excluded…
> Sorry I missed that part. After seeing diffs between stable/1908 and
> stable/2001, commit: b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897 is the
> only visible git commit in dpdk plugin which is playing with mempool buffers.
> If it does not solve the problem then I suspect problem lies outside dpdk
> plugin. I am guessing DPDK-19.08 is being used here with VPP-19.08
> 
> Hi Chetan,
> > > 3) I took previous commit of  "vlib: don't use vector for keeping
> > > buffer
> > indices in the pool " ie "df0191ead2cf39611714b6603cdc5bdddc445b57" :
> > Everything looks fine with Buffers 537600.
> In which branch, Commit: df0191ead2cf39611714b6603cdc5bdddc445b57 is
> previous commit of b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897?
> 
> Thanks,
> Nitin
> > -Original Message-
> > From: Damjan Marion 
> > Sent: Monday, February 17, 2020 3:47 PM
> > To: Nitin Saxena 
> > Cc: chetan bhasin ; vpp-dev@lists.fd.io
> > Subject: Re: [EXT] [vpp-dev] Regarding buffers-per-numa parameter
> >
> >
> > Dear Nitin,
> >
> > if you read Chetan’s email bellow, you will see that this one is
> > already excluded…
> >
> > Also, it will not be easy to explain how this patch blows tx function
> > in dpdk
> > mlx5 pmd…
> >
> > —
> > Damjan
> >
> > > On 17 Feb 2020, at 11:12, Nitin Saxena  wrote:
> > >
> > > Hi Prashant/Chetan,
> > >
> > > I would try following change first to solve the problem in 1908
> > >
> > > commit b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897b
> > > Author: Damjan Marion 
> > > Date:   Tue Mar 12 18:14:15 2019 +0100
> > >
> > > vlib: don't use vector for keeping buffer indices in
> > >
> > > Type: refactor
> > >
> > > Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4
> > > Signed-off-by: Damjan Marion damar...@cisco.com
> > >
> > > You can also try copying src/plugins/dpdk/buffer.c from stable/2001
> > branch to stable/1908
> > >
> > > Thanks,
> > > Nitin
> > >
> > > From: vpp-dev@lists.fd.io  On Behalf Of Damjan
> > Marion via Lists.Fd.Io
> > > Sent: Monday, February 17, 2020 1:52 PM
> > > To: chetan bhasin 
> > > Cc: vpp-dev@lists.fd.io
> > > Subject: [EXT] Re: [vpp-dev] Regarding buffers-per-numa parameter
> > >
> > > External Email
> > >
> > > On 17 Feb 2020, at 07:37, chetan bhasin 
> > wrote:
> > >
> > > Bottom line is stable/vpp 908 does not work with higher number of
> > > buffers
> > but stable/vpp2001 does. Could you please advise which area we can
> > look at ,as it would be difficult for us to move to vpp2001 at this time.
> > >
> > > I really don’t have idea what caused this problem to disappear.
> > > You may try to use “git bisect” to find out which commit fixed it….
> > >
> > > —
> > > Damjan
> > >
> > >
> > >
> > > On Mon, Feb 17, 2020 at 11:01 AM chetan bhasin via Lists.Fd.Io
> >  wrote:
> > > Thanks Damjan for the reply!
> > >
> > > Following are my observations on Intel X710/XL710 pci-
> > > 1) I took latest code base from stable/vpp19.08  : Seeing error as "
> > ethernet-input l3 mac mismatch"
> > > With Buffers 537600 vpp# show buffers
> > |
> > > Pool NameIndex NUMA  Size  Data Size  Total  Avail  Cached   
> > > Used
> > > default-numa-0 0 0   2496 2048   537600 510464   1319
> > > 25817
> > > default-numa-1 1 1   2496 2048   537600 528896390
> > > 8314
> > >
> > > vpp# show hardware-interfaces
> > >   NameIdx   Link  Hardware
> > > BondEthernet0  3 up   BondEthernet0
> > >   Link speed: unknown
> > >   Ethernet address 3c:fd:fe:b5:5e:40
> > > FortyGigabitEthernet12/0/0 1 up   FortyGigabitEthernet12/0/0
> >

Re: [EXT] Re: [vpp-dev] Regarding buffers-per-numa parameter

[Nitin Saxena]

Hi Prashant/Chetan,

I would try following change first to solve the problem in 1908

commit b6e8b1a7c8bf9f9fbd05cdc3c90111d9e7a6897b
Author: Damjan Marion 
Date:   Tue Mar 12 18:14:15 2019 +0100

vlib: don't use vector for keeping buffer indices in

Type: refactor

Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4
Signed-off-by: Damjan Marion damar...@cisco.com

You can also try copying src/plugins/dpdk/buffer.c from stable/2001 branch to 
stable/1908

Thanks,
Nitin

From: vpp-dev@lists.fd.io  On Behalf Of Damjan Marion via 
Lists.Fd.Io
Sent: Monday, February 17, 2020 1:52 PM
To: chetan bhasin 
Cc: vpp-dev@lists.fd.io
Subject: [EXT] Re: [vpp-dev] Regarding buffers-per-numa parameter

External Email


On 17 Feb 2020, at 07:37, chetan bhasin 
mailto:chetan.bhasin...@gmail.com>> wrote:

Bottom line is stable/vpp 908 does not work with higher number of buffers but 
stable/vpp2001 does. Could you please advise which area we can look at ,as it 
would be difficult for us to move to vpp2001 at this time.

I really don’t have idea what caused this problem to disappear.
You may try to use “git bisect” to find out which commit fixed it….

—
Damjan



On Mon, Feb 17, 2020 at 11:01 AM chetan bhasin via 
Lists.Fd.Io
 mailto:gmail@lists.fd.io>> wrote:
Thanks Damjan for the reply!

Following are my observations on Intel X710/XL710 pci-
1) I took latest code base from stable/vpp19.08  : Seeing error as " 
ethernet-input l3 mac mismatch"
With Buffers 537600
vpp# show buffers   
|
Pool NameIndex NUMA  Size  Data Size  Total  Avail  Cached   Used
default-numa-0 0 0   2496 2048   537600 510464   131925817
default-numa-1 1 1   2496 2048   537600 5288963908314

vpp# show hardware-interfaces
  NameIdx   Link  Hardware
BondEthernet0  3 up   BondEthernet0
  Link speed: unknown
  Ethernet address 3c:fd:fe:b5:5e:40
FortyGigabitEthernet12/0/0 1 up   FortyGigabitEthernet12/0/0
  Link speed: 40 Gbps
  Ethernet address 3c:fd:fe:b5:5e:40
  Intel X710/XL710 Family
carrier up full duplex mtu 9206
flags: admin-up pmd rx-ip4-cksum
rx: queues 16 (max 320), desc 1024 (min 64 max 4096 align 32)
tx: queues 16 (max 320), desc 4096 (min 64 max 4096 align 32)
pci: device 8086:1583 subsystem 8086:0001 address :12:00.00 numa 0
max rx packet len: 9728
promiscuous: unicast off all-multicast on
vlan offload: strip off filter off qinq off
rx offload avail:  vlan-strip ipv4-cksum udp-cksum tcp-cksum qinq-strip
   outer-ipv4-cksum vlan-filter vlan-extend jumbo-frame
   scatter keep-crc
rx offload active: ipv4-cksum
tx offload avail:  vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-cksum
   tcp-tso outer-ipv4-cksum qinq-insert vxlan-tnl-tso
   gre-tnl-tso ipip-tnl-tso geneve-tnl-tso multi-segs
   mbuf-fast-free
tx offload active: none
rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other 
ipv6-frag
   ipv6-tcp ipv6-udp ipv6-sctp ipv6-other l2-payload
rss active:ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv6-frag ipv6-tcp
   ipv6-udp ipv6-other
tx burst function: i40e_xmit_pkts_vec_avx2
rx burst function: i40e_recv_pkts_vec_avx2
tx errors 17
rx frames ok4585
rx bytes ok   391078
extended stats:
  rx good packets   4585
  rx good bytes   391078
  tx errors   17
  rx multicast packets  4345
  rx broadcast packets   243
  rx unknown protocol packets   4588
  rx size 65 to 127 packets 4529
  rx size 128 to 255 packets  32
  rx size 256 to 511 packets  26
  rx size 1024 to 1522 packets 1
  tx size 65 to 127 packets   33
FortyGigabitEthernet12/0/1 2 up   FortyGigabitEthernet12/0/1
  Link speed: 40 Gbps
  Ethernet address 3c:fd:fe:b5:5e:40
  Intel X710/XL710 Family
carrier up full duplex mtu 9206
flags: admin-up pmd rx-ip4-cksum
rx: queues 16 (max 320), desc 1024 (min 64 max 4096 

[vpp-dev] Check in ip4_local_inline()

[Nitin Saxena]

Hi,

I am sending UDP termination packets to VPP interface with single source IP. I 
find that fib lookup is happening for every packet, even if source IP for 
current packet is same as last packet. Is it expected behavior? Following patch 
seems to avoid lookup for every packet.

Thanks,
Nitin

diff --git a/src/vnet/ip/ip4_forward.c b/src/vnet/ip/ip4_forward.c
index aa554ea..59edaba 100644
--- a/src/vnet/ip/ip4_forward.c
+++ b/src/vnet/ip/ip4_forward.c
@@ -1542,6 +1542,7 @@ ip4_local_check_src (vlib_buffer_t * b, ip4_header_t * 
ip0,
   last_check->src.as_u32 = ip0->src_address.as_u32;
   last_check->lbi = lbi0;
   last_check->error = *error0;
+  last_check->first = 0;
 }
   else
 {
@@ -1549,7 +1550,6 @@ ip4_local_check_src (vlib_buffer_t * b, ip4_header_t * 
ip0,
vnet_buffer (b)->ip.adj_index[VLIB_TX];
   vnet_buffer (b)->ip.adj_index[VLIB_TX] = last_check->lbi;
   *error0 = last_check->error;
-  last_check->first = 0;
 }
}

@@ -1638,6 +1638,7 @@ ip4_local_check_src_x2 (vlib_buffer_t ** b, ip4_header_t 
** ip,
   last_check->src.as_u32 = ip[1]->src_address.as_u32;
   last_check->lbi = lbi[1];
   last_check->error = error[1];
+  last_check->first = 0;
 }
   else
 {
@@ -1651,7 +1652,6 @@ ip4_local_check_src_x2 (vlib_buffer_t ** b, ip4_header_t 
** ip,

   error[0] = last_check->error;
   error[1] = last_check->error;
-  last_check->first = 0;
 }
}



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15030): https://lists.fd.io/g/vpp-dev/message/15030
Mute This Topic: https://lists.fd.io/mt/69397810/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Check in ip4_local_inline()

[Nitin Saxena]

Thanks Balaji. Destination IP lookup happens in ip4-lookup node. The source IP 
getting checked in ip4-local for uRPF functionality

Thanks,
Nitin

From: Balaji Venkatraman (balajiv) 
Sent: Friday, January 3, 2020 8:39 PM
To: Nitin Saxena 
Cc: Dave Barach (dbarach) ; vpp-dev@lists.fd.io
Subject: [EXT] Re: [vpp-dev] Check in ip4_local_inline()

External Email

Hi Nitin,
Isn’t ip_lookup based of destination ip, Could you verify if the lookup happens 
on every packet when ‘destination ip’ is the same?

--
Regards,
Balaji.


From: mailto:vpp-dev@lists.fd.io>> on behalf of "Dave 
Barach via Lists.Fd.Io" 
mailto:dbarach=cisco@lists.fd.io>>
Reply-To: "Dave Barach (dbarach)" mailto:dbar...@cisco.com>>
Date: Friday, January 3, 2020 at 6:38 AM
To: Nitin Saxena mailto:nsax...@marvell.com>>, 
"vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" 
mailto:vpp-dev@lists.fd.io>>
Cc: "vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" 
mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] Check in ip4_local_inline()

Ask yourself how often there will be precisely one source (or dst) IP address 
in this path. Optimizing a specific lab/benchmark case may or may not make 
sense.

D.

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Nitin Saxena
Sent: Friday, January 3, 2020 8:02 AM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [vpp-dev] Check in ip4_local_inline()

Hi,

I am sending UDP termination packets to VPP interface with single source IP. I 
find that fib lookup is happening for every packet, even if source IP for 
current packet is same as last packet. Is it expected behavior? Following patch 
seems to avoid lookup for every packet.

Thanks,
Nitin

diff --git a/src/vnet/ip/ip4_forward.c b/src/vnet/ip/ip4_forward.c
index aa554ea..59edaba 100644
--- a/src/vnet/ip/ip4_forward.c
+++ b/src/vnet/ip/ip4_forward.c
@@ -1542,6 +1542,7 @@ ip4_local_check_src (vlib_buffer_t * b, ip4_header_t * 
ip0,
   last_check->src.as_u32 = ip0->src_address.as_u32;
   last_check->lbi = lbi0;
   last_check->error = *error0;
+  last_check->first = 0;
 }
   else
 {
@@ -1549,7 +1550,6 @@ ip4_local_check_src (vlib_buffer_t * b, ip4_header_t * 
ip0,
vnet_buffer (b)->ip.adj_index[VLIB_TX];
   vnet_buffer (b)->ip.adj_index[VLIB_TX] = last_check->lbi;
   *error0 = last_check->error;
-  last_check->first = 0;
 }
}

@@ -1638,6 +1638,7 @@ ip4_local_check_src_x2 (vlib_buffer_t ** b, ip4_header_t 
** ip,
   last_check->src.as_u32 = ip[1]->src_address.as_u32;
   last_check->lbi = lbi[1];
   last_check->error = error[1];
+  last_check->first = 0;
 }
   else
 {
@@ -1651,7 +1652,6 @@ ip4_local_check_src_x2 (vlib_buffer_t ** b, ip4_header_t 
** ip,

   error[0] = last_check->error;
   error[1] = last_check->error;
-  last_check->first = 0;
 }
}



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15037): https://lists.fd.io/g/vpp-dev/message/15037
Mute This Topic: https://lists.fd.io/mt/69397810/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Check in ip4_local_inline()

[Nitin Saxena]

Hi Dave,

Thanks.

I agree with your point that there is less chance of consecutive packets having 
same source IP. However both functions: ip4_local_check_src(), 
ip4_local_check_src_x2() already have the trick to avoid fib lookup for 
consecutive packets having same source IP. Correct me if I am wrong, currently 
else {} part in both aforementioned functions seems to be a dead code as 
PREDICT_FALSE(last_check->first) is always TRUE (as last_check->first is always 
1 throughout ip4_local_inline() function).

Also with my patch, there is no impact on cycle count of ip4_local node (both 
x86 and ARM) where source IP increments for every packet in a terminating 
frame. It does decrease cycles for ip4-local when all packets have similar 
source IP.

So is there any gap in my understanding or is it deliberate to make else {} 
case  as dead code?

Thanks,
Nitin

From: Dave Barach (dbarach) 
Sent: Friday, January 3, 2020 8:08 PM
To: Nitin Saxena ; vpp-dev@lists.fd.io
Subject: [EXT] RE: [vpp-dev] Check in ip4_local_inline()

External Email

Ask yourself how often there will be precisely one source (or dst) IP address 
in this path. Optimizing a specific lab/benchmark case may or may not make 
sense.

D.

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Nitin Saxena
Sent: Friday, January 3, 2020 8:02 AM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [vpp-dev] Check in ip4_local_inline()

Hi,

I am sending UDP termination packets to VPP interface with single source IP. I 
find that fib lookup is happening for every packet, even if source IP for 
current packet is same as last packet. Is it expected behavior? Following patch 
seems to avoid lookup for every packet.

Thanks,
Nitin

diff --git a/src/vnet/ip/ip4_forward.c b/src/vnet/ip/ip4_forward.c
index aa554ea..59edaba 100644
--- a/src/vnet/ip/ip4_forward.c
+++ b/src/vnet/ip/ip4_forward.c
@@ -1542,6 +1542,7 @@ ip4_local_check_src (vlib_buffer_t * b, ip4_header_t * 
ip0,
   last_check->src.as_u32 = ip0->src_address.as_u32;
   last_check->lbi = lbi0;
   last_check->error = *error0;
+  last_check->first = 0;
 }
   else
 {
@@ -1549,7 +1550,6 @@ ip4_local_check_src (vlib_buffer_t * b, ip4_header_t * 
ip0,
vnet_buffer (b)->ip.adj_index[VLIB_TX];
   vnet_buffer (b)->ip.adj_index[VLIB_TX] = last_check->lbi;
   *error0 = last_check->error;
-  last_check->first = 0;
 }
}

@@ -1638,6 +1638,7 @@ ip4_local_check_src_x2 (vlib_buffer_t ** b, ip4_header_t 
** ip,
   last_check->src.as_u32 = ip[1]->src_address.as_u32;
   last_check->lbi = lbi[1];
   last_check->error = error[1];
+  last_check->first = 0;
 }
   else
 {
@@ -1651,7 +1652,6 @@ ip4_local_check_src_x2 (vlib_buffer_t ** b, ip4_header_t 
** ip,

   error[0] = last_check->error;
   error[1] = last_check->error;
-  last_check->first = 0;
 }
}



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15036): https://lists.fd.io/g/vpp-dev/message/15036
Mute This Topic: https://lists.fd.io/mt/69397810/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] Re: [vpp-dev] Check in ip4_local_inline()

[Nitin Saxena]

Hi Florin,

Thanks. I have raised https://gerrit.fd.io/r/#/c/vpp/+/24192/ for formal review.
Neale, Dave and you are code reviewer.

Thanks,
Nitin

From: Florin Coras 
Sent: Saturday, January 4, 2020 12:33 AM
To: Nitin Saxena 
Cc: Dave Barach (dbarach) ; vpp-dev@lists.fd.io; Neale Ranns 
(nranns) 
Subject: [EXT] Re: [vpp-dev] Check in ip4_local_inline()

External Email

Hi Nitin,

I believe your observation is correct. Adding Neale in case we missed something.

Regards,
Florin


On Jan 3, 2020, at 10:32 AM, Nitin Saxena 
mailto:nsax...@marvell.com>> wrote:

Hi Dave,

Thanks.

I agree with your point that there is less chance of consecutive packets having 
same source IP. However both functions: ip4_local_check_src(), 
ip4_local_check_src_x2() already have the trick to avoid fib lookup for 
consecutive packets having same source IP. Correct me if I am wrong, currently 
else {} part in both aforementioned functions seems to be a dead code as 
PREDICT_FALSE(last_check->first) is always TRUE (as last_check->first is always 
1 throughout ip4_local_inline() function).

Also with my patch, there is no impact on cycle count of ip4_local node (both 
x86 and ARM) where source IP increments for every packet in a terminating 
frame. It does decrease cycles for ip4-local when all packets have similar 
source IP.

So is there any gap in my understanding or is it deliberate to make else {} 
case  as dead code?

Thanks,
Nitin

From: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Sent: Friday, January 3, 2020 8:08 PM
To: Nitin Saxena mailto:nsax...@marvell.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [EXT] RE: [vpp-dev] Check in ip4_local_inline()

External Email

Ask yourself how often there will be precisely one source (or dst) IP address 
in this path. Optimizing a specific lab/benchmark case may or may not make 
sense.

D.

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Nitin Saxena
Sent: Friday, January 3, 2020 8:02 AM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [vpp-dev] Check in ip4_local_inline()

Hi,

I am sending UDP termination packets to VPP interface with single source IP. I 
find that fib lookup is happening for every packet, even if source IP for 
current packet is same as last packet. Is it expected behavior? Following patch 
seems to avoid lookup for every packet.

Thanks,
Nitin

diff --git a/src/vnet/ip/ip4_forward.c b/src/vnet/ip/ip4_forward.c
index aa554ea..59edaba 100644
--- a/src/vnet/ip/ip4_forward.c
+++ b/src/vnet/ip/ip4_forward.c
@@ -1542,6 +1542,7 @@ ip4_local_check_src (vlib_buffer_t * b, ip4_header_t * 
ip0,
   last_check->src.as_u32 = ip0->src_address.as_u32;
   last_check->lbi = lbi0;
   last_check->error = *error0;
+  last_check->first = 0;
 }
   else
 {
@@ -1549,7 +1550,6 @@ ip4_local_check_src (vlib_buffer_t * b, ip4_header_t * 
ip0,
vnet_buffer (b)->ip.adj_index[VLIB_TX];
   vnet_buffer (b)->ip.adj_index[VLIB_TX] = last_check->lbi;
   *error0 = last_check->error;
-  last_check->first = 0;
 }
}

@@ -1638,6 +1638,7 @@ ip4_local_check_src_x2 (vlib_buffer_t ** b, ip4_header_t 
** ip,
   last_check->src.as_u32 = ip[1]->src_address.as_u32;
   last_check->lbi = lbi[1];
   last_check->error = error[1];
+  last_check->first = 0;
 }
   else
 {
@@ -1651,7 +1652,6 @@ ip4_local_check_src_x2 (vlib_buffer_t ** b, ip4_header_t 
** ip,

   error[0] = last_check->error;
   error[1] = last_check->error;
-  last_check->first = 0;
 }
}



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15036): 
https://lists.fd.io/g/vpp-dev/message/15036<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fd.io_g_vpp-2Ddev_message_15036=DwMFAg=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0=iWWj-GvN2fElm6OFyL2H1EtRfOLaIRb7i-UCLFG-300=VSzNbdec-4dJCpWuzjMhl2iRGi6UFtAsJ8VMCKq9EuQ=>
Mute This Topic: 
https://lists.fd.io/mt/69397810/675152<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fd.io_mt_69397810_675152=DwMFAg=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0=iWWj-GvN2fElm6OFyL2H1EtRfOLaIRb7i-UCLFG-300=ve0Vw9DgVR-V93kzd3um8PQeiN4zf1Q-j_81dfN4RY8=>
Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: 
https://lists.fd.io/g/vpp-dev/unsub<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fd.io_g_vpp-2Ddev_unsub=DwMFAg=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0=iWWj-GvN2fElm6OFyL2H1EtRfOLaIRb7i-UCLFG-300=_5IrITVAS_jkkNxG7rVBePlr4tiYUQ8cjWZFx5j_QLE=>
  [fcoras.li...@gmail.com<mailto:fcoras.li...@gmail.com>]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to t

Re: [EXT] Re: [vpp-dev] n_vectors...

[Nitin Saxena]

Hi,

I too agree with Chris and Elias. Although my suggestion is not to change code 
variables or any code per say because it's not difficult to understand the code.
But at least, we should change documentation/comments and CLI output which 
makes obvious that packet != vector (although it is currently). People tend to 
relate vector with vlib_frame and it is hard to make them understand that 
vector is not vlib_frame but a single packet.

Thanks,
Nitin

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Christian
> Hopps
> Sent: Tuesday, March 31, 2020 7:36 PM
> To: Elias Rudberg 
> Cc: Christian Hopps ; dbar...@cisco.com; vpp-
> d...@lists.fd.io
> Subject: [EXT] Re: [vpp-dev] n_vectors...
> 
> External Email
> 
> --
> 
> 
> > On Mar 31, 2020, at 9:20 AM, Elias Rudberg 
> wrote:
> >
> > Hi Chris and Dave,
> >
> > Thanks for bringing this up, and thanks for explaining!
> >
> > I agree with Chris that this is confusing, it makes it much more
> > difficult to understand the code.
> 
> As someone who UTSL as a first, second and third setp, I tend to agree. It
> made things harder then they probably needed to be when I was trying to
> grok the gestalt.
> 
> Lot of history though, hard to change (or even ask) for something like this.
> 
> A happy start for me would be if "show runtime" didn't call packets
> "Vectors". And, if someone repurposed vlib for non-packets they could
> change "Packets" to "Elements" or whatever they want :) Of course I did this
> in our local branch, but wasn't planning on trying to upstream.
> 
> Thanks,
> Chris.
> 
> >
> > Perhaps this is the kind of thing that doesn't matter much to those
> > who are already familiar with the code, while at the same time it
> > matters a lot for newcomers. If you want to lower the threshold for
> > new people to be able to come in and understand the code and possibly
> > contribute, then I think it would be a good idea to fix this even if
> > it means changing many lines of code. It could be argued that the fact
> > that "n_vectors" exists in so many places makes it even more important
> > to have a reasonable name for it. One way could be to start with
> > renaming things in some of the main data structures like those in
> > vlib/node.h and vlib/threads.h and such places, and the changes the
> > compiler will force as a result of that.
> >
> > Best regards,
> > Elias
> >
> >
> > On Tue, 2020-03-31 at 00:45 +, Dave Barach via Lists.Fd.Io wrote:
> >> Hmmm, yeah. Been at this for years, I can’t really remember when we
> >> settled on e.g. n_vectors vs. n_vector_elts or some such.
> >>
> >> In new code, it’s perfectly fair to use whatever names seem fit for
> >> purpose.
> >>
> >> Vlib would be happy doing image processing, or any other kind of
> >> vector processing. There’s no law which says that frames need to have
> >> 32-bit elements. Each node decides.
> >>
> >> FWIW... Dave
> >>
> >> From: vpp-dev@lists.fd.io  On Behalf Of
> >> Christian Hopps
> >> Sent: Monday, March 30, 2020 8:07 PM
> >> To: vpp-dev 
> >> Cc: Christian Hopps 
> >> Subject: [vpp-dev] n_vectors...
> >>
> >> Something has always bothered me about my understanding of VPPs use
> >> of the term "vector" and "vectors". When I think of Vector Packet
> >> Processing I think of processing a vector (array) of packets in a
> >> single call to a node. The code, though, then seems to refer to the
> >> individual packets as "vectors" when it uses field names like
> >> "n_vectors" to refer to the number of buffers in a frame, or when
> >> "show runtime" talks about "vectors per call", when I think it's
> >> really talking about "packets/buffers per call" (and my mind wants to
> >> think that it's always *1* vector/frame of packets per call by
> >> design).
> >>
> >> I find this confusing, and so I thought I'd ask if there was some
> >> meaning here I'm missing?
> >>
> >> Thanks,
> >> Chris.
> >
> >

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15973): https://lists.fd.io/g/vpp-dev/message/15973
Mute This Topic: https://lists.fd.io/mt/72724951/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Next LTS release?

[Nitin Saxena]

Hi Dave,

I know from mailing list that current VPP LTS release is 19.08. Could you 
please confirm which next release will be candidate for LTS? Is it going to be 
20.08?
I could not find any LTS release information on VPP 
homepage(https://wiki.fd.io/view/VPP). Appreciate if you share the link for 
current/next LTS release info.

Thanks,
Nitin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15928): https://lists.fd.io/g/vpp-dev/message/15928
Mute This Topic: https://lists.fd.io/mt/72646638/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] Re: [vpp-dev] Next LTS release?

[Nitin Saxena]

Hi Dave,

Thanks for sharing the information.

-Nitin

From: vpp-dev@lists.fd.io  On Behalf Of Dave Wallace
Sent: Monday, March 30, 2020 8:49 PM
To: Nitin Saxena 
Cc: vpp-dev 
Subject: [EXT] Re: [vpp-dev] Next LTS release?

External Email

Hi Nitin,

The plan of record for the next VPP LTS release is 20.09.

Note: There was a change to a 4 month release cycle for VPP last fall.  The 
release cycle for 2020 and beyond is 20.01, 20.05, 20.09LTS, 21.01 ...

Thanks,
-daw-


On 3/30/2020 2:00 AM, Nitin Saxena wrote:
Hi Dave,

I know from mailing list that current VPP LTS release is 19.08. Could you 
please confirm which next release will be candidate for LTS? Is it going to be 
20.08?
I could not find any LTS release information on VPP 
homepage(https://wiki.fd.io/view/VPP<https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.fd.io_view_VPP=DwMD-g=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0=CqvOzq_fOUOo-zhv6ETUeKqwG_M3UcV6fBzW8fFj1gA=44dETF7VmYeSKeQ7tFsPiOpDtbLA_jOQe-Ga_W7k364=>).
 Appreciate if you share the link for current/next LTS release info.

Thanks,
Nitin





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15940): https://lists.fd.io/g/vpp-dev/message/15940
Mute This Topic: https://lists.fd.io/mt/72657190/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] How to see coverity logs

[Nitin Saxena]

Hi Maintainers,

I would like to see coverity logs for latest VPP build. I tried adding 
https://scan.coverity.com/projects/fd-io-vp to my github but when I click on 
"View Defects" it says "Permission denied".
Is it possible to view coverity logs or I am missing anything?

Thanks,
Nitin


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#17312): https://lists.fd.io/g/vpp-dev/message/17312
Mute This Topic: https://lists.fd.io/mt/76577815/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] How to see coverity logs

[Nitin Saxena]

Hi Dave/Chris,

Thanks for the pointers. I am able to see Coverity defects now. My reason for 
looking for Coverity logs is following

It has recently been came to my notice that vnet APIs and their caller do not 
free clib_error_t * properly. Usage of vnet_[sw|hw]_interface_set_flags() APIs 
are widespread across VPP but often clib_error_t *error  returned by these APIs 
are not freed. Is it my correct understanding that non-zero error returned by 
these APIs must be freed by the caller? I tried fixing some of the main APIs in 
vnet (https://gerrit.fd.io/r/c/vpp/+/28643)

Thanks,
Nitin

From: vpp-dev@lists.fd.io  On Behalf Of Dave Barach via 
lists.fd.io
Sent: Wednesday, September 2, 2020 6:44 PM
To: Nitin Saxena ; vpp-dev 
Subject: [EXT] Re: [vpp-dev] How to see coverity logs

External Email

Known Coverity UI bug. Happens all the time for me.

Back up one page, and click on "show me the bugs" again. It should work.

D.

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Nitin Saxena
Sent: Wednesday, September 2, 2020 3:31 AM
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: [vpp-dev] How to see coverity logs

Hi Maintainers,

I would like to see coverity logs for latest VPP build. I tried adding 
https://scan.coverity.com/projects/fd-io-vp<https://urldefense.proofpoint.com/v2/url?u=https-3A__scan.coverity.com_projects_fd-2Dio-2Dvp=DwMFAg=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0=TX49XF4W5LMPHXROUhmceNFYYPdAjN594gNBNp4vUCQ=h79x--OWOPP2w6PLpRmbqgayjfen8be8uNjkTWqxmmU=>
 to my github but when I click on "View Defects" it says "Permission denied".
Is it possible to view coverity logs or I am missing anything?

Thanks,
Nitin


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#17322): https://lists.fd.io/g/vpp-dev/message/17322
Mute This Topic: https://lists.fd.io/mt/76577815/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] Re: [vpp-committers] [vpp-dev] VPP committers: VPP PTL vote

[Nitin Saxena]

Thanks Dave for your relentless efforts in making VPP better, not just from 
performance perspective but also from its usability point of view. Contribution 
like perfmon, pcap tracing, G2 graph viewer and list goes on …are sheer 
examples of visionary PTL.

Congratulations to Damjan!. Both of you share bigger chunk of what VPP is 
today. Best of luck

Thanks,
Nitin

From: vpp-dev@lists.fd.io  On Behalf Of Dave Barach
Sent: Tuesday, September 29, 2020 12:40 AM
To: 'Florin Coras' ; 'Damjan Marion (damarion)' 

Cc: 'Benoit Ganne (bganne)' ; 'Dave Barach (dbarach)' 
; vpp-committ...@lists.fd.io; vpp-dev@lists.fd.io
Subject: [EXT] Re: [vpp-committers] [vpp-dev] VPP committers: VPP PTL vote

External Email

Thanks for the kind words... It’s been fun, but now it’s time for others to 
take the ball and run with it... Congrats to Damjan, he’ll do a great job!

Dave

From: vpp-committ...@lists.fd.io 
mailto:vpp-committ...@lists.fd.io>> On Behalf Of 
Florin Coras
Sent: Monday, September 28, 2020 2:28 PM
To: Damjan Marion (damarion) mailto:damar...@cisco.com>>
Cc: Benoit Ganne (bganne) mailto:bga...@cisco.com>>; Dave 
Barach (dbarach) mailto:dbar...@cisco.com>>; 
vpp-committ...@lists.fd.io; 
vpp-dev@lists.fd.io
Subject: Re: [vpp-committers] [vpp-dev] VPP committers: VPP PTL vote

Congrats, Damjan!! Those are some huge “shoes" you’ll have to fill but I’m sure 
they’ll fit ;-)

Dave, a few sentences won’t do all your contributions as PTL any justice! 
Nonetheless, thank you for starting the project, for diligently working towards 
growing it and for entrusting us, as a community, with its future!

Florin

On Sep 28, 2020, at 4:47 AM, Damjan Marion via 
lists.fd.io
 mailto:damarion=cisco@lists.fd.io>> wrote:



Now when we have votes from all 12 committers (me excluded) I would like to 
thank you all for your +1s.
It is nice to be elected by unanimous decision :)

—
Damjan


On 28.09.2020., at 09:44, Benoit Ganne (bganne) via 
lists.fd.io
 mailto:bganne=cisco@lists.fd.io>> wrote:

+1

And I take this opportunity to say a big thank you Dave for your efforts to 
build an healthy community, and all the best for Damjan in his (I hope) future 
new role 

Best
ben

-Original Message-
From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Dave Barach
via 
lists.fd.io
Sent: vendredi 25 septembre 2020 21:14
To: vpp-committ...@lists.fd.io
Cc: vpp-dev@lists.fd.io
Subject: [vpp-dev] VPP committers: VPP PTL vote

Folks,



The self-nomination period closed yesterday. We had one self-nomination,
from Damjan Marion. At this point, we can proceed with a vote.



I’m sure that Damjan will do a great job, so let me start:



Damjan Marion as VPP PTL: +1



Please vote +1, 0, -1. For once, the “reply-all” button is everyone’s
friend.



Thanks... Dave



-Original Message-
From: d...@barachs.net 
mailto:d...@barachs.net>>
Sent: Thursday, September 17, 2020 10:32 AM
To: 'vpp-dev@lists.fd.io' 
mailto:vpp-dev@lists.fd.io>>; 
't...@lists.fd.io'
mailto:t...@lists.fd.io>>
Subject: Happy Trails to Me...



Folks,



I’m departing the employment rolls towards the end of next month. Although
I intend to remain active in the 
fd.io
 vpp community as a coder,
committer, and resident greybeard, it’s time for the community to pick a
new PTL.



According to the project governance document,
https://fd.io/docs/tsc/FD.IO-Technical-Community-Document-12-12-2017.pdf

Re: [vpp-dev] Create big tables on huge-page

[Nitin Saxena]

Hi Lijian,

+1 on the finding. It would be interested to know how much is the performance 
gain.
Having said that, correct me if I am wrong,  I think pmalloc module works only 
on single hugepage size (pm->def_log2_page_sz) which means either 1G or 2M and 
not both

Thanks,
Nitin

From: vpp-dev@lists.fd.io  On Behalf Of Honnappa 
Nagarahalli
Sent: Thursday, July 23, 2020 10:53 PM
To: Damjan Marion 
Cc: Lijian Zhang ; vpp-dev ; nd 
; Govindarajan Mohandoss ; 
Jieqiang Wang ; Honnappa Nagarahalli 

Subject: [EXT] Re: [vpp-dev] Create big tables on huge-page

External Email

Sure. We will create couple of patches (in the areas we are analyzing 
currently) and we can decide from there.
Thanks,
Honnappa

From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, July 23, 2020 12:17 PM
To: Honnappa Nagarahalli 
mailto:honnappa.nagaraha...@arm.com>>
Cc: Lijian Zhang mailto:lijian.zh...@arm.com>>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>; nd 
mailto:n...@arm.com>>; Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
Jieqiang Wang mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] Create big tables on huge-page



Hard to say without seeing the patch. Can you summarize what the changes will 
be in each particular .c file?


On 23 Jul 2020, at 18:00, Honnappa Nagarahalli 
mailto:honnappa.nagaraha...@arm.com>> wrote:

Hi Damjan,
Thank you. Till your patch is ready, would you accept patches 
that would enable creating these tables in 1G huge pages as temporary solution?

Thanks,
Honnappa

From: Damjan Marion mailto:dmar...@me.com>>
Sent: Thursday, July 23, 2020 7:15 AM
To: Lijian Zhang mailto:lijian.zh...@arm.com>>
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>; nd 
mailto:n...@arm.com>>; Honnappa Nagarahalli 
mailto:honnappa.nagaraha...@arm.com>>; 
Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
Jieqiang Wang mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] Create big tables on huge-page


I started working on patch which addresses most of this points, few weeks ago, 
but likely I will not have it completed for 20.09.
Even if it is completed, it is probably bad idea to merge it so late in the 
release process….

—
Damjan



On 23 Jul 2020, at 10:45, Lijian Zhang 
mailto:lijian.zh...@arm.com>> wrote:

Hi Maintainers,
From VPP source code, ip4-mtrie table is created on huge-page only when below 
parameters are set in configuration file.
While adjacency table is created on normal-page always.
  36 ip {
  37   heap-size 256M
  38   mtrie-hugetlb
  39 }
In the 10K flow testing, I configured 10K routing entries in ip4-mtrie and 10K 
entries in adjacency table.
By creating ip4-mtrie table on 1G huge-page with above parameters set and 
similarly create adjacency table on 1G huge-page, although I don’t observe 
obvious throughput performance improvement, but TLB misses are dramatically 
reduced.
Do you think configuration of 10K routing entries + 10K adjacency entries is a 
reasonable and possible config, or normally it would be 10K routing entries + 
only several adjacency entries?
Does it make sense to create adjacency table on huge-pages?
Another problem is although above assigned heap-size is 256M, but on 1G 
huge-page system, it seems to occupy a huge-page completely, other memory space 
within that huge-page seems will not be used by other tables.

Same as the bihash based tables, only 2M huge-page system is supported. To 
support creating bihash based tables on 1G huge-page system, each table will 
occupy a 1G huge-page completely, but that will waste a lot of memories.
Is it possible just like pmalloc module, reserving a big memory space on 1G/2M 
huge-pages in initialization stage, and then allocate memory pieces per 
requirement for Bihash, ip4-mtrie and adjacency tables, so that all tables 
could be created on huge-pages and will fully utilize the huge-pages.
I tried to create MAC table on 1G huge-page, and it does improve throughput 
performance.
vpp# show bihash
Name Actual Configured
GBP Endpoints - MAC/BD   1m 1m
b4s 64m 64m
b4s 64m 64m
in2out   10.12m 10.12m
in2out   10.12m 10.12m
ip4-dr   2m 2m
ip4-dr   2m 2m
ip6 FIB fwding table32m 32m
ip6 FIB non-fwding table32m 32m
ip6 mFIB table  32m 32m
l2fib mac table512m 512m
mapping_by_as4  64m 64m
out2in 128m 128m
out2in 128m 128m
out2in   10.12m 10.12m
out2in   10.12m 10.12m
pppoe link table 8m 8m
pppoe session table  8m 8m
static_mapping_by_external  64m 64m
static_mapping_by_local 64m 64m
stn addresses1m 1m
users  

Re: [EXT] [vpp-dev] New perfmon plugin

[Nitin Saxena]

Hi Damjan,

ARM defines two sets of performance monitoring counters and extension
1. Common Event number and micro-architecture events defined by ARM which every 
chip vendor should implement.   
2. Chip vendor specific PMU counters other than (1)

I am not in ThunderX2 BU but I think the kernel driver you are referring to 
seems to be a PMU extension which falls under the category of (2) above. See 
below for OCTEONTX2 output

So for ARM to be enabled in perfmon plugin, I am thinking, 
- we need common bundle to register common ARM PMU events. This should be first 
step and include most of the useful/important events
- chip vendor specific bundle should also be allowed to "implementation 
defined" PMU events 

One of the key differentiation in ARM is a kernel driver needs to be hooked at 
runtime to allow VPP to get hold of PMU counters (which is not the case with 
x86)

>>> Can you capture contents of /sys/bus/event_source/devices/ from one system?
I do not have ThunderX2 access but here is the output of OCTEONTX2

$ ls -ltr /sys/bus/event_source/devices/
total 0
lrwxrwxrwx 1 root root 0 Dec 14 06:48 software -> ../../../devices/software
lrwxrwxrwx 1 root root 0 Dec 14 06:48 cs_etm -> ../../../devices/cs_etm
lrwxrwxrwx 1 root root 0 Dec 14 06:48 breakpoint -> ../../../devices/breakpoint
lrwxrwxrwx 1 root root 0 Dec 14 06:48 tracepoint -> ../../../devices/tracepoint
lrwxrwxrwx 1 root root 0 Dec 14 06:48 armv8_cavium_thunder -> 
../../../devices/armv8_cavium_thunder

Thanks,
Nitin




> -Original Message-
> From: Damjan Marion 
> Sent: Monday, December 14, 2020 4:19 PM
> To: Nitin Saxena 
> Cc: vpp-dev 
> Subject: Re: [EXT] [vpp-dev] New perfmon plugin
> 
> 
> Isn’t there also uncore PMU? I can see some thunderx2 specific driver in
> kernel source.
> 
> Can you capture contents of /sys/bus/event_source/devices/ from one
> system?
> 
> Thanks,
> 
> —
> Damjan
> 
> 
> > On 14.12.2020., at 09:09, Nitin Saxena  wrote:
> >
> > Yes most of the ARM processors including ThunderX2, OCTEONTX2 has
> PMU as per AARCH64 specifications. I did some analysis to add ARM support
> in older perfmon plugin and should be easy to port to this new one. This is
> something in TODO list which is much needed for us and overall ARM
> >
> > Thanks,
> > Nitin
> >
> >> -Original Message-
> >> From: Damjan Marion 
> >> Sent: Saturday, December 12, 2020 7:46 PM
> >> To: Nitin Saxena 
> >> Cc: vpp-dev 
> >> Subject: Re: [EXT] [vpp-dev] New perfmon plugin
> >>
> >>
> >> cool, if I got it right ThunderX2 have
> >> own PMU so we can add it as new source and create specific bundles.
> >>
> >> --
> >> Damjan
> >>
> >>> On 12.12.2020., at 11:07, Nitin Saxena  wrote:
> >>>
> >>> Hi Damjan,
> >>>
> >>> I was already fan of older perfmon plugin and new one seems superset
> >>> of the older one (at-least from video)
> >>>
> >>> Nice addition
> >>>
> >>> Thanks,
> >>> Nitin
> >>>
> >>>> -Original Message-
> >>>> From: vpp-dev@lists.fd.io  On Behalf Of Damjan
> >>>> Marion via lists.fd.io
> >>>> Sent: Friday, December 11, 2020 9:44 PM
> >>>> To: vpp-dev 
> >>>> Subject: [EXT] [vpp-dev] New perfmon plugin
> >>>>
> >>>> External Email
> >>>>
> >>>> -
> >>>> -
> >>>>
> >>>> Guys,
> >>>>
> >>>> I just submitted patch with the new perfmon plugin: [1]
> >>>>
> >>>> It takes significantly different approach compared to current one.
> >>>>
> >>>> - it support multiple sources of perf counters (linux, intel core,
> >>>> intel uncore) and it is extensible to other vendors
> >>>> - it have concept instances so it can monitor multiple instances of
> >>>> specific PMU (DRAM channels, UPI/QPU links, ..)
> >>>> - it supports node, thread and system metrics
> >>>> - different metrics are organized in bundles, where bundle consists
> >>>> of multiple counters and format functions which calculates and
> presents
> >> metric.
> >>>> Yuo can find example of bundle here [2]
> >>>>
> >>>> To se how this looks in action, I captured small asciinema video: [3]
> >>>>
> >>>> As this new plugin is signif

Re: [EXT] [vpp-dev] New perfmon plugin

[Nitin Saxena]

Hi Damjan,

I was already fan of older perfmon plugin and new one seems superset of the 
older one (at-least from video)

Nice addition

Thanks,
Nitin

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Damjan
> Marion via lists.fd.io
> Sent: Friday, December 11, 2020 9:44 PM
> To: vpp-dev 
> Subject: [EXT] [vpp-dev] New perfmon plugin
> 
> External Email
> 
> --
> 
> Guys,
> 
> I just submitted patch with the new perfmon plugin: [1]
> 
> It takes significantly different approach compared to current one.
> 
>  - it support multiple sources of perf counters (linux, intel core, intel 
> uncore)
> and it is extensible to other vendors
>  - it have concept instances so it can monitor multiple instances of specific
> PMU (DRAM channels, UPI/QPU links, ..)
>  - it supports node, thread and system metrics
>  - different metrics are organized in bundles, where bundle consists of
> multiple counters and format functions which calculates and presents metric.
> Yuo can find example of bundle here [2]
> 
> To se how this looks in action, I captured small asciinema video: [3]
> 
> As this new plugin is significantly different than old one, I wonder if anyone
> thinks we should keep old une.
> Also, any other feedback is wellcome.
> 
> Thanks,
> 
> Damjan
> 
> 
> [1] https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__gerrit.fd.io_r_c_vpp_-
> 2B_30186=DwIFAg=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3
> IkGduCfk9LbZMPOAecQGDzWV0=YLtsj5dRrH71jbO5dLDi4wZVjd_BZiWves
> g69AEyXxg=4tTXQv2yVPh7WGV5S1eL8TKk7hzW5hs9HV5HffGAZWg=
> [2] https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__gerrit.fd.io_r_c_vpp_-
> 2B_30186_12_src_plugins_perfmon_intel_bundle_load-
> 5Fblocks.c=DwIFAg=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3
> IkGduCfk9LbZMPOAecQGDzWV0=YLtsj5dRrH71jbO5dLDi4wZVjd_BZiWves
> g69AEyXxg=4vBJsKjTB2jgQ_knZlMqSPL9zu9r61mn_EDKfPaxP30=
> [3] https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__asciinema.org_a_aFN5rMFYw0RPvGOZiFsziXV5w=DwIFAg=nKjWec
> 2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0
> =YLtsj5dRrH71jbO5dLDi4wZVjd_BZiWvesg69AEyXxg=OOSrE-
> xR_qMePcPBdjlV1FSyjeguxIFhIl7_W98ghFk=


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18314): https://lists.fd.io/g/vpp-dev/message/18314
Mute This Topic: https://lists.fd.io/mt/78904118/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] New perfmon plugin

[Nitin Saxena]

Yes most of the ARM processors including ThunderX2, OCTEONTX2 has PMU as per 
AARCH64 specifications. I did some analysis to add ARM support in older perfmon 
plugin and should be easy to port to this new one. This is something in TODO 
list which is much needed for us and overall ARM

Thanks,
Nitin

> -Original Message-
> From: Damjan Marion 
> Sent: Saturday, December 12, 2020 7:46 PM
> To: Nitin Saxena 
> Cc: vpp-dev 
> Subject: Re: [EXT] [vpp-dev] New perfmon plugin
> 
> 
> cool, if I got it right ThunderX2 have
> own PMU so we can add it as new source and create specific bundles.
> 
> --
> Damjan
> 
> > On 12.12.2020., at 11:07, Nitin Saxena  wrote:
> >
> > Hi Damjan,
> >
> > I was already fan of older perfmon plugin and new one seems superset
> > of the older one (at-least from video)
> >
> > Nice addition
> >
> > Thanks,
> > Nitin
> >
> >> -Original Message-
> >> From: vpp-dev@lists.fd.io  On Behalf Of Damjan
> >> Marion via lists.fd.io
> >> Sent: Friday, December 11, 2020 9:44 PM
> >> To: vpp-dev 
> >> Subject: [EXT] [vpp-dev] New perfmon plugin
> >>
> >> External Email
> >>
> >> -
> >> -
> >>
> >> Guys,
> >>
> >> I just submitted patch with the new perfmon plugin: [1]
> >>
> >> It takes significantly different approach compared to current one.
> >>
> >> - it support multiple sources of perf counters (linux, intel core,
> >> intel uncore) and it is extensible to other vendors
> >> - it have concept instances so it can monitor multiple instances of
> >> specific PMU (DRAM channels, UPI/QPU links, ..)
> >> - it supports node, thread and system metrics
> >> - different metrics are organized in bundles, where bundle consists
> >> of multiple counters and format functions which calculates and presents
> metric.
> >> Yuo can find example of bundle here [2]
> >>
> >> To se how this looks in action, I captured small asciinema video: [3]
> >>
> >> As this new plugin is significantly different than old one, I wonder
> >> if anyone thinks we should keep old une.
> >> Also, any other feedback is wellcome.
> >>
> >> Thanks,
> >>
> >> Damjan
> >>
> >>
> >> [1] https://urldefense.proofpoint.com/v2/url?u=https-
> >> 3A__gerrit.fd.io_r_c_vpp_-
> >>
> 2B_30186=DwIFAg=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3
> >>
> IkGduCfk9LbZMPOAecQGDzWV0=YLtsj5dRrH71jbO5dLDi4wZVjd_BZiWves
> >> g69AEyXxg=4tTXQv2yVPh7WGV5S1eL8TKk7hzW5hs9HV5HffGAZWg=
> >> [2] https://urldefense.proofpoint.com/v2/url?u=https-
> >> 3A__gerrit.fd.io_r_c_vpp_-
> >> 2B_30186_12_src_plugins_perfmon_intel_bundle_load-
> >>
> 5Fblocks.c=DwIFAg=nKjWec2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3
> >>
> IkGduCfk9LbZMPOAecQGDzWV0=YLtsj5dRrH71jbO5dLDi4wZVjd_BZiWves
> >> g69AEyXxg=4vBJsKjTB2jgQ_knZlMqSPL9zu9r61mn_EDKfPaxP30=
> >> [3] https://urldefense.proofpoint.com/v2/url?u=https-
> >>
> 3A__asciinema.org_a_aFN5rMFYw0RPvGOZiFsziXV5w=DwIFAg=nKjWec
> >>
> 2b6R0mOyPaz7xtfQ=S4H7jibYAtA5YOvfL3IkGduCfk9LbZMPOAecQGDzWV0
> >> =YLtsj5dRrH71jbO5dLDi4wZVjd_BZiWvesg69AEyXxg=OOSrE-
> >> xR_qMePcPBdjlV1FSyjeguxIFhIl7_W98ghFk=
> >
> >
> > 
> >

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18324): https://lists.fd.io/g/vpp-dev/message/18324
Mute This Topic: https://lists.fd.io/mt/78904118/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] Re: [vpp-dev] Prefetches improvement for VPP Arm generic image

[Nitin Saxena]

Hi Damjan,

+1. This proposal completely aligns with our thought process. i.e 

-  Data structure alignment to 128B cache line and separate macro for 
prefetching which can be 64B or 128B depending on underlying SoC

It also allow us to have single AARCH64 binary in distros for any cloud 
deployments. We can provide our feedback on the changes for any performance 
impact on our SoC

Thanks,
Nitin

> -Original Message-
> From: Damjan Marion 
> Sent: Wednesday, July 14, 2021 11:18 PM
> To: vpp-dev 
> Cc: Honnappa Nagarahalli ; Govindarajan
> Mohandoss ; Tianyu Li
> ; Nitin Saxena ; Lijian Zhang
> 
> Subject: [EXT] Re: [vpp-dev] Prefetches improvement for VPP Arm generic
> image
> 
> External Email
> 
> --
> 
> I spent a bit of time to look at this and come up with some reasonable
> solution.
> 
> First, 128-byte cacheline is not dead, recently announced Marvell Octeon 10
> have 128 byte cacheline.
> 
> In current code cacheline size defines both amount of data prefetch
> instruction prefetches and alignment of data in data structures needed to
> avoid false sharing.
> 
> So I think ideally we should have following:
> 
> - on x86:
>   - number of bytes prefetch instruction prefetches set to 64
>   - data structures should be aligned to 64 bytes
>   - due the fact that there is adjacent cacehline prefetcher on x86 it may be
> worth
> investigating if aligning to 128 brings some value
> 
> - on AArch64
>   - number of bytes prefetch instruction prefetches set to 64 or 128, based on
> multiarch variant running
>   - data structures should be aligned to 128 bytes as that value prevents 
> false
> sharing for both 64 and 128 byte cacheline systems
> 
> Main problem is abuse of CLIB_PREFETCH() macro in our codebase.
> Original idea of it was good, somebody wanted to provide macro which
> transparently emits 1-4 prefetch instructions based on data size recognising
> that there may be systems with different cacheline size
> 
> Like:
>   CLIB_PREFETCH (p, sizeof (ip6_header_t), LOAD);
> 
> But reality is, most of the time we have:
>   CLIB_PREFETCH (p, CLIB_CACHE_LINE_BYTES, LOAD);
> 
> Where it is assumed that cacheline size is 64 and that just wasted resources
> on system with 128-byte cacheline.
> 
> Also, most of places in our codebase are perfectly fine with whatever
> cacheline size is, so I’m thinking about following:
> 
> 1. set CLIB_CACHE_LINE_BYTES to 64 on x86 and 128 on ARM, that will make
> sure false sharing is not happening
> 
> 2. introduce CLIB_CACHE_PREFETCH_BYTES which can be set to different
> value for different multiarch variant (64 for N1, 128 ThinderX2)
> 
> 3. modify CLIB_PREFETCH macro to use CLIB_CACHE_PREFETCH_BYTES to
> emit proper number of prefetch instructions for cases where data size is
> specified
> 
> 4. take the stub and replace all `CLIB_PREFETCH (p, CLIB_CACHE_LINE_BYTES,
> LOAD);` with `clib_prefetch_load (p);`.
>There may be exceptions but those lines typically mean: "i want to prefetch
> few (<=64) bytes at this address and i really don’t care what the cache line
> size is”.
> 
> 5. analyse remaining few cases where CLIB_PREFETCH() is used with size
> specified by CLIB_CACHE_LINE_BYTES.
> 
> Thoughts?
> 
> —
> Damjan
> 
> > On 06.07.2021., at 03:48, Lijian Zhang  wrote:
> >
> > Thanks Damjan for your comments. Some replies in lines.
> > 
> > Hi Lijian,
> >
> > It will be good to know if 128 byte cacheline is something ARM platforms
> will be using in the future or it is just historical.
> > [Lijian] Existing ThunderX1 and OcteonTX2 CPUs are 128 byte cache-line. To
> my knowledge, there may be more CPUs with 128 byte cache-line in the
> future.
> >
> > Cacheline size problem is not just about prefetching, even bigger issue is
> false sharing, so we need to address both.
> > [Lijian] Yes, there may be false-sharing issue when running VPP image with
> 64B definition on 128B cache-line CPUs. We will do some scalability testing
> with that case, and check the multi-core performance.
> >
> > Probably best solution is to have 2 VPP images, one for 128 and one for 64
> byte cacheline size.
> > [Lijian] For native built image, that’s fine. But I’m not sure if it’s 
> > possible for
> cloud binaries installed via “apt-get install”.
> >
> > Going across the whole codebase and replacing prefetch macros is
> something we should definitely avoid.
> > [Lijian] I got your concerns on large scope replacement. My concern is
> when CLIB_PREFETCH() is used to prefetch packet content into cache as
> below example, cache-li

[vpp-dev] IPSec fixes missing in stable/2106 branch

[Nitin Saxena]

Hi,

We are using v21.06 release and recently encountered crash in async crypto 
framework which is fixed by [1]
This patch is missing in stable/2106 branch. I searched more and find out that 
there are more IPSec fixes which are not part of stable/2106 ([2], [3],[4])

Is there any plan to merge them to stable/2106 or movement to stable/2110 is 
recommended?
Note: [1], [2], [3] are easily cherry-pickable in v2106 but not [4]

[1]: https://gerrit.fd.io/r/c/vpp/+/32596
[2]: https://gerrit.fd.io/r/c/vpp/+/32725
[3]: https://gerrit.fd.io/r/c/vpp/+/32611
[4]: https://gerrit.fd.io/r/c/vpp/+/32938

Thanks,
Nitin

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20687): https://lists.fd.io/g/vpp-dev/message/20687
Mute This Topic: https://lists.fd.io/mt/88263120/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-