Re: [vpp-dev] Q: how best to avoid locking for cleanup.

[Govindarajan Mohandoss]

Hi Chris,

>>>>I do wonder how many other cases of "state associated with
>>>> in-flight packets" there might be, and if more sophisticated
>>>> general solution might be useful.
>>>>> RCU mechanism is the general solution to avoid Reader locks in data 
>>>>> plane. With this scheme, there is no lock needed in the data plane. The 
>>>>> in-flight packets can be tracked through the counters like the way you 
>>>>> described (or) through an alternate scheme explained below. The control 
>>>>> plane can do a delayed free of the shared memory objects (Shared with 
>>>>> data plane) based on these counters.
If the scope of the counters are widened across the entire packet processing 
path in run to completion model, then all the data structures like Bihash, 
IPSec SA and other unknown cases can be covered.

Proposal:
===
Control Plane:
===
1. Maintain a common running counter across all the control plane threads.
2. Delete the shared memory objects from Bihash/IPSec SA table.
3. Set the running counter in shared memory.
4. Wait till all the data plane threads catch up with this counter value 
(Maintain a catch up counter per data plane thread).
 Optimization Note: Instead of waiting for data plane sync, a pending list 
can be maintained to hold the memory objects that needs to be freed after all 
the inflight packets are flushed out.
The memory objects in the Pending list 
can then be freed through next data plane table entry add/delete APIs.
5. Free the shared memory objects once all the data plane threads catch up.

In Fast path:
==
While (1) /* packet processing loop in run-to-completion model */
 {
  Ethernet processing,
 IP Processing,
IPSec Processing,
  Etc.,
  /* End of packet processing */
 Catch up counter = counter value set by control plane. <<<< Data 
plane/Control plane sync
}

There is a solution to this problem already in DPDK. We can evaluate it and 
decide whether it will fit VPP requirements (or) we can do a new design for VPP.
https://doc.dpdk.org/guides/prog_guide/rcu_lib.html

Thanks
Govind

> -Original Message-
> From: Andrew  Yourtchenko 
> Sent: Friday, February 28, 2020 5:50 AM
> To: Honnappa Nagarahalli 
> Cc: Benoit Ganne (bganne) ; cho...@chopps.org; vpp-
> d...@lists.fd.io; nd ; Govindarajan Mohandoss
> ; Lijian Zhang 
> Subject: Re: [vpp-dev] Q: how best to avoid locking for cleanup.
>
> On 2/28/20, Honnappa Nagarahalli 
> wrote:
>
> >> On the other hand, if you do modify shared data structures in the
> >> datapath, you are on your own - you need to take care of the data
> >> consistency.
> >> Again, the way we usually deal with that is to do a "rpc" to the main
> >> thread - then the main thread can request the worker barrier, etc.
> >>
> >> Or do you refer to other situations?
> > I was looking at the bi-hash library on a standalone basis. The
> > entries are deleted and buckets are freed without any synchronization
> > between the writer
>
> FWIW, the above statement is incorrect if all we are talking is pure bihash
> operation with values that are not used as keys for subsequent memory
> accesses in other data structures. This code in
> include/vppinfra/bihash_template.c might be of interest:
>
> static inline int BV (clib_bihash_add_del_inline)
>   (BVT (clib_bihash) * h, BVT (clib_bihash_kv) * add_v, int is_add,
>int (*is_stale_cb) (BVT (clib_bihash_kv) *, void *), void *arg) {
>   u32 bucket_index;
>
> ...
>
>
>   BV (clib_bihash_lock_bucket) (b);   <- LOCK
>
>   /* First elt in the bucket? */
>   if (BV (clib_bihash_bucket_is_empty) (b))
> {
>   if (is_add == 0)
> {
>   BV (clib_bihash_unlock_bucket) (b);
>   return (-1);
> }
>
> 
>
>   /* Move readers to a (locked) temp copy of the bucket */
>   BV (clib_bihash_alloc_lock) (h);<- LOCK
>   BV (make_working_copy) (h, b);
>
> -
>
> and so on.
>
> when performing the actual bihash operations as a user of the bihash, you
> most definitely do *not* need any extra locking, the bihash is doing it for 
> you
> behind the scenes.
>
> There is only one transient condition that I had seen - under intense
> add/delete workload, the readers in other threads may see the lookup
> successful but the value returned being ~0.
> That is fairly easy to deal with.
>
> But of course there is a race in case you are using bihash to store secondary
> indices into your own data structures - if you are deleting a bihash entry,

Re: [vpp-dev] Q: how best to avoid locking for cleanup.

[Govindarajan Mohandoss]

Please find inline.

Thanks
Govind

> -Original Message-
> From: Dave Barach (dbarach) 
> Sent: Friday, February 28, 2020 4:17 PM
> To: Govindarajan Mohandoss ;
> Andrew  Yourtchenko ; Honnappa Nagarahalli
> 
> Cc: Benoit Ganne (bganne) ; cho...@chopps.org; vpp-
> d...@lists.fd.io; nd ; Lijian Zhang 
> Subject: RE: [vpp-dev] Q: how best to avoid locking for cleanup.
>
> On the data plane side, please use vm->main_loop_count. Mark the variable
> volatile in src/vlib/main.h.
>
> Atomically update data structures, include a memory barrier.
> Foreach_thread: snapshoot vm->main_loop_count.
> Delay until all vm->main_loop_count values have changed.
   2 consecutive snapshots can be obtained by introducing delay between the 
"vm->main_loop_count" polls. The delay timer value (??) will have an impact on 
the API configuration time.
   A delay timer can be replaced with a spinning loop till the 
"vm->main_loop_count" changes. This will be relatively better than hard coding 
a delay timer value. But, this will also have an impact on API config time. The 
API config time may not be crucial for IPSec SA deletion / IP route deletion, 
but it is sensitive to GTPu kind of protocols. As mentioned before, this 
problem can be solved by introducing a Pending list scheme in control plane.
" Optimization Note: Instead of waiting for data plane sync, a pending list can 
be maintained to hold the memory objects that needs to be freed after all the 
inflight packets are flushed out. The 
memory objects in the Pending list can then be freed through next data plane 
table entry add/delete APIs."

  More details about API latency sensitive applications:
  
  I noticed GTPu plugin in VPP. Not sure about the VPP use case of GTPu.
  https://github.com/FDio/vpp/tree/master/src/plugins/gtpu
  In general, GTPu is heavily used in Radio access Networks for LTE and 5G 
Transport.  The GTPu Tunnel setup/tear down (Based on UE call setup/teardown) 
will happen at higher rate. Considering VPP to run on LTE/5G Transport nodes 
like eNodeB/gNB in the future (Not sure whether VPP is already used in eNodeB 
?), we have to take API latency also as an important requirement and address it.

> Clean up old data structures.
>
> FWIW... Dave
>
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan
> Mohandoss
> Sent: Friday, February 28, 2020 4:55 PM
> To: Andrew  Yourtchenko ; Honnappa Nagarahalli
> 
> Cc: Benoit Ganne (bganne) ; cho...@chopps.org; vpp-
> d...@lists.fd.io; nd ; Lijian Zhang 
> Subject: Re: [vpp-dev] Q: how best to avoid locking for cleanup.
>
> Hi Chris,
> 
> >>>>I do wonder how many other cases of "state associated with in-flight
> >>>>packets" there might be, and if more sophisticated general solution
> >>>>might be useful.
> >>>>> RCU mechanism is the general solution to avoid Reader locks in data
> plane. With this scheme, there is no lock needed in the data plane. The in-
> flight packets can be tracked through the counters like the way you described
> (or) through an alternate scheme explained below. The control plane can do
> a delayed free of the shared memory objects (Shared with data plane) based
> on these counters.
> If the scope of the counters are widened across the entire packet processing
> path in run to completion model, then all the data structures like Bihash,
> IPSec SA and other unknown cases can be covered.
>
> Proposal:
> ===
> Control Plane:
> ===
> 1. Maintain a common running counter across all the control plane threads.
> 2. Delete the shared memory objects from Bihash/IPSec SA table.
> 3. Set the running counter in shared memory.
> 4. Wait till all the data plane threads catch up with this counter value
> (Maintain a catch up counter per data plane thread).
>  Optimization Note: Instead of waiting for data plane sync, a pending list
> can be maintained to hold the memory objects that needs to be freed after
> all the inflight packets are flushed out.
> The memory objects in the Pending 
> list can then be
> freed through next data plane table entry add/delete APIs.
> 5. Free the shared memory objects once all the data plane threads catch up.
>
> In Fast path:
> ==
> While (1) /* packet processing loop in run-to-completion model */  {
>   Ethernet processing,
>  IP Processing,
> IPSec Processing,
>   Etc.,
>   /* End of packet processing */
>  Catch up counter = counter value set by control plane. <<<< Data
> plane/Control plane sync }
>
> The

[vpp-dev] ACL question

[Govindarajan Mohandoss]

Hello ACL Maintainer,

  We want to measure and optimize the ACL performance for ARM servers.  As per 
the foll. link, there are 4 different implementation of ACLs in VPP.

  https://fd.io/docs/vpp/master/usecases/acls.html

  We would like to start with most commonly used ACL implementation in VPP 
which can cover L2, L3 and L4 fields. As per the link above and CSIT reports 
(link below), it looks like ACL plugin is the right match.

  Can you please confirm ? ACL plugin has 2 variants - Stateful & Stateless. 
Which is common and widely used in VPP ?

  
https://docs.fd.io/csit/master/report/detailed_test_results/vpp_performance_results/index.html



Thanks

Govind

IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15871): https://lists.fd.io/g/vpp-dev/message/15871
Mute This Topic: https://lists.fd.io/mt/72544608/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Hi Andrew,
  Thanks for the document.
  Can you please share the documents related to ACL plugin CLI config for both 
stateful & stateless modes ?

   I tried the following commands for input ACL in VAT CLI. Not sure whether 
this is SL / SF ?
“
vat# acl_add_replace -1 ipv4 permit dst 192.82.1.1/32
vl_api_acl_add_replace_reply_t_handler:70: ACL index: 0
vat# acl_interface_set_acl_list TenGigabitEthernet13/0/0 input 0
vat# acl_interface_list_dump TenGigabitEthernet13/0/0
vl_api_acl_interface_list_details_t_handler:115: sw_if_index: 3, count: 1, 
n_input: 1
   input 0

vat# help acl_add_replace
usage: acl_add_replace  [] 
 [src IP/plen] [dst IP/plen] [sport X-Y] 
[dport X-Y] [proto P] [tcpflags FL MASK], ... , ...
“

Thanks
Govind

From: Andrew  Yourtchenko 
Sent: Thursday, March 26, 2020 4:49 AM
To: Govindarajan Mohandoss 
Cc: vpp-dev@lists.fd.io; Lijian Zhang ; Jieqiang Wang 
; nd 
Subject: Re: [vpp-dev] ACL question

As an acl plugin author I can say both stateful and stateless ACLs are used for 
different consumers.

Various matching implementations in vpp are used in different use cases... and 
there is not a single silver bullet magic answer, because the trade offs are 
different.

 https://nonsns.github.io/paper/rossi19ton.pdf

Is a reasonable read on the subject - also because it relates to VPP and the 
real project that we did a while ago.

--a


On 25 Mar 2020, at 17:26, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:


Hello ACL Maintainer,

  We want to measure and optimize the ACL performance for ARM servers.  As per 
the foll. link, there are 4 different implementation of ACLs in VPP.

  https://fd.io/docs/vpp/master/usecases/acls.html

  We would like to start with most commonly used ACL implementation in VPP 
which can cover L2, L3 and L4 fields. As per the link above and CSIT reports 
(link below), it looks like ACL plugin is the right match.

  Can you please confirm ? ACL plugin has 2 variants – Stateful & Stateless. 
Which is common and widely used in VPP ?

  
https://docs.fd.io/csit/master/report/detailed_test_results/vpp_performance_results/index.html



Thanks

Govind
IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you. 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15886): https://lists.fd.io/g/vpp-dev/message/15886
Mute This Topic: https://lists.fd.io/mt/72544608/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Hi Andrew,
   I just found out that ACL action differentiates SF or SL. Following command 
enables SF and provides better performance.

   “acl_add_replace -1 ipv4 permit+reflect dst 192.82.1.1/32”



   Few more questions:

   =

   Choosing between VPP Classifiers and ACL Plugin:

   
https://lists.fd.io/g/vpp-dev/message/5716?p=,,,20,0,0,0::relevance,,ACL,20,2,60,10641995

  *   You mentioned that VPP classifiers are faster than ACL plugin.

  1.  For  field based classification, which one provides better 
data plane perf ?
  2.  Does classifier support ranges ?
  3.  Which one is better if the rate of ACL rule add/del is high / low?
  4.  Whether ACL rule priority is supported in both the schemes ?
  5.  Whether ACL Plugin SF mode will perform better than classifier ? Whether 
classifier also has SF mode ?

   ACL Plugin:

  1.  SF mode – How much of extra memory is needed compared to SL mode ?

Thanks
Govind

From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan 
Mohandoss via Lists.Fd.Io
Sent: Thursday, March 26, 2020 12:37 PM
To: Andrew  Yourtchenko 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] ACL question

Hi Andrew,
  Thanks for the document.
  Can you please share the documents related to ACL plugin CLI config for both 
stateful & stateless modes ?

   I tried the following commands for input ACL in VAT CLI. Not sure whether 
this is SL / SF ?
“
vat# acl_add_replace -1 ipv4 permit dst 192.82.1.1/32
vl_api_acl_add_replace_reply_t_handler:70: ACL index: 0
vat# acl_interface_set_acl_list TenGigabitEthernet13/0/0 input 0
vat# acl_interface_list_dump TenGigabitEthernet13/0/0
vl_api_acl_interface_list_details_t_handler:115: sw_if_index: 3, count: 1, 
n_input: 1
   input 0

vat# help acl_add_replace
usage: acl_add_replace  [] 
 [src IP/plen] [dst IP/plen] [sport X-Y] 
[dport X-Y] [proto P] [tcpflags FL MASK], ... , ...
“

Thanks
Govind

From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
Sent: Thursday, March 26, 2020 4:49 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: Re: [vpp-dev] ACL question

As an acl plugin author I can say both stateful and stateless ACLs are used for 
different consumers.

Various matching implementations in vpp are used in different use cases... and 
there is not a single silver bullet magic answer, because the trade offs are 
different.

 https://nonsns.github.io/paper/rossi19ton.pdf

Is a reasonable read on the subject - also because it relates to VPP and the 
real project that we did a while ago.

--a

On 25 Mar 2020, at 17:26, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:


Hello ACL Maintainer,

  We want to measure and optimize the ACL performance for ARM servers.  As per 
the foll. link, there are 4 different implementation of ACLs in VPP.

  https://fd.io/docs/vpp/master/usecases/acls.html

  We would like to start with most commonly used ACL implementation in VPP 
which can cover L2, L3 and L4 fields. As per the link above and CSIT reports 
(link below), it looks like ACL plugin is the right match.

  Can you please confirm ? ACL plugin has 2 variants – Stateful & Stateless. 
Which is common and widely used in VPP ?

  
https://docs.fd.io/csit/master/report/detailed_test_results/vpp_performance_results/index.html



Thanks

Govind
IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15890): https://lists.fd.io/g/vpp-dev/message/15890
Mute This Topic: https://lists.fd.io/mt/72544608/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Thank you very much Andrew !! I will do some benchmarks and get back to you to 
understand it better.

Thanks
Govind

> -Original Message-
> From: Andrew  Yourtchenko 
> Sent: Friday, March 27, 2020 7:52 AM
> To: Govindarajan Mohandoss 
> Cc: vpp-dev@lists.fd.io; nd 
> Subject: Re: [vpp-dev] ACL question
> 
> > On 27 Mar 2020, at 00:47, Govindarajan Mohandoss
>  wrote:
> >
> > 
> >
> > Hi Andrew,
> >
> >    I just found out that ACL action differentiates SF or SL. Following
> command enables SF and provides better performance.
> >
> >    “acl_add_replace -1 ipv4 permit+reflect dst 192.82.1.1/32”
> >
> >
> >
> >    Few more questions:
> >
> >    =
> >
> >    Choosing between VPP Classifiers and ACL Plugin:
> >
> >
> > https://lists.fd.io/g/vpp-dev/message/5716?p=,,,20,0,0,0::relevance,,A
> > CL,20,2,60,10641995
> >
> > You mentioned that VPP classifiers are faster than ACL plugin.
> > For  field based classification, which one provides better data
> plane perf ?
> 
> 
> It depends. If you wanna simultaneously match on all three, there is
> currently no mechanism to generically do so.
> 
> But then every time I looked at the use cases claiming to require that, turned
> out it was a bad idea to represent the data this way - because of
> combinatorial explosion. Even ACLs themselves suffer from this issue - N
> sources times M destinations times K servces equal N*M*K rules, which
> quickly skyrockets.
> 
> > Does classifier support ranges ?
> 
> 
> Classifier supports chained masked lookups. You might emulate ranges there.
> 
> That said, I had seen ranges used only in a tiny percentage of the cases. So
> they are a corner case imho.
> 
> 
> > Which one is better if the rate of ACL rule add/del is high / low?
> 
> 
> Classifier single table is your best bet probably. ACL plugin deliberately 
> does
> not have an API to add/del a single rule - you always download the entire
> ACL.
> 
> > Whether ACL rule priority is supported in both the schemes ?
> 
> 
> First match for Acl and multi table classify case. Single table is just a hash
> lookup because the entries don’t overlap by definition
> 
> > Whether ACL Plugin SF mode will perform better than classifier ?
> 
> 
> I did not benchmark them. It's somewhat different use cases.
> 
> > Whether classifier also has SF mode ?
> 
> 
> Nope.
> 
> >
> >
> >    ACL Plugin:
> >
> > SF mode – How much of extra memory is needed compared to SL mode ?
> 
> 
> Depending on the number of active sessions... each session creates two
> binash table entries, and consumes an entry in the session pool. The default
> values in the code for the bihash memory usage have been tested with half a
> million sessions - so you can extrapolate from those with some ballpark
> (though bihash memory usage is not linear wrt the entries, and also there is
> some extra memory churn due to bucket reallocations when the size
> increases).
> 
> —a
> 
> >
> >
> > Thanks
> >
> > Govind
> >
> >
> >
> > From: vpp-dev@lists.fd.io  On Behalf Of
> > Govindarajan Mohandoss via Lists.Fd.Io
> > Sent: Thursday, March 26, 2020 12:37 PM
> > To: Andrew  Yourtchenko 
> > Cc: vpp-dev@lists.fd.io
> > Subject: Re: [vpp-dev] ACL question
> >
> >
> >
> > Hi Andrew,
> >
> >   Thanks for the document.
> >
> >   Can you please share the documents related to ACL plugin CLI config for
> both stateful & stateless modes ?
> >
> >
> >
> >    I tried the following commands for input ACL in VAT CLI. Not sure
> whether this is SL / SF ?
> >
> > “
> >
> > vat# acl_add_replace -1 ipv4 permit dst 192.82.1.1/32
> >
> > vl_api_acl_add_replace_reply_t_handler:70: ACL index: 0
> >
> > vat# acl_interface_set_acl_list TenGigabitEthernet13/0/0 input 0
> >
> > vat# acl_interface_list_dump TenGigabitEthernet13/0/0
> >
> > vl_api_acl_interface_list_details_t_handler:115: sw_if_index: 3,
> > count: 1, n_input: 1
> >
> >    input 0
> >
> >
> >
> > vat# help acl_add_replace
> >
> > usage: acl_add_replace  []
>  [src IP/plen] [dst IP/plen] [sport X-Y]
> [dport X-Y] [proto P] [tcpflags FL MASK], ... , ...
> >
> > “
> >
> >
> >
> > Thanks
> >
> > Govind
> >
> >
> >
> > From: Andrew  Yourtchenko 
> > Sent: Thursday, March 26, 2020 4:49 AM
> > To: Govindarajan Mohand

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Hi Andrew,

  I am working on ACL plugin SF+SL optimization on ARM servers.

  I am finding prefetches in ACL node is becoming bottle neck. I see 
performance improvements on both SL & SF mode, when SF mode bihash table 
related prefetching is disabled.

  I need some help with right ACL config to verify my patch.



 I did the testing with Ingress ACL -- 1 Rule and 50 Rules (Rule:  - DPORT is incremented). The Traffic match all the 50 rules.



  When I tried to add 100 rules on the same rule set in SF mode:

  "acl_add_replace -1 ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 
proto 17 sport 100 dport 1,

   ... ,

   ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 sport 100 
dport 100",



   I see only 48 rules in show tables and 48th rule is added as “permit” all 
and not “permit + reflect”. Does it mean <0 – 47> rules will be SF and the rest 
will be in SL mode ?



"

vpp# show acl-plugin acl

acl-index 0 count 49 tag {}

   0: ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 
sport 100 dport 1

   

  47: ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 
sport 100 dport 48

  48: ipv4 permit src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 
0-65535

  applied inbound on sw_if_index: 1

  used in lookup context index: 0

"



  1.  Is there a limit of 48 on number of rules that can be added into the Rule 
table (acl-index 0) in SF mode ?
  2.  Whether 48 rules in a ruleset is good enough to verify my optimization 
patch (Traffic flow will match all the 48 rules) ?
  3.  Can I associate more than 1 ACL rule set to an ingress interface (like 
“vat# acl_interface_set_acl_list TenGigabitEthernet1/0/0 input 0 1 2”) ? Each 
Rule set 0, 1, 2 will have different ACL rules. Do I need to test this case 
also to study the performance gain ?
  4.  In SL mode, When I tried to add 100 rules, only 53 rules are seen in show 
table. 53rd rule is added as permit all (Should I read it as permit all ?). Is 
there a limit on number of rules in SL mode ?

“

vpp# show acl-plugin acl

acl-index 0 count 54 tag {}

  0: ipv4 permit src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 sport 100 
dport 1

  ….

 52: ipv4 permit src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 sport 100 
dport 53

 53: ipv4 permit src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 
dport 0-65535

  applied inbound on sw_if_index: 1

  used in lookup context index: 0

“



Thanks

Govind



> -Original Message-

> From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan

> Mohandoss via Lists.Fd.Io

> Sent: Friday, March 27, 2020 11:32 AM

> To: Andrew  Yourtchenko 

> Cc: vpp-dev@lists.fd.io

> Subject: Re: [vpp-dev] ACL question

>

> Thank you very much Andrew !! I will do some benchmarks and get back to

> you to understand it better.

>

> Thanks

> Govind

>

> > -Original Message-

> > From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>

> > Sent: Friday, March 27, 2020 7:52 AM

> > To: Govindarajan Mohandoss 
> > mailto:govindarajan.mohand...@arm.com>>

> > Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
> > mailto:n...@arm.com>>

> > Subject: Re: [vpp-dev] ACL question

> >

> > > On 27 Mar 2020, at 00:47, Govindarajan Mohandoss

> > mailto:govindarajan.mohand...@arm.com>> 
> > wrote:

> > >

> > >

> > >

> > > Hi Andrew,

> > >

> > >I just found out that ACL action differentiates SF or SL.

> > > Following

> > command enables SF and provides better performance.

> > >

> > >“acl_add_replace -1 ipv4 permit+reflect dst 192.82.1.1/32”

> > >

> > >

> > >

> > >Few more questions:

> > >

> > >=

> > >

> > >Choosing between VPP Classifiers and ACL Plugin:

> > >

> > >

> > > https://lists.fd.io/g/vpp-dev/message/5716?p=,,,20,0,0,0::relevance,

> > > ,A

> > > CL,20,2,60,10641995

> > >

> > > You mentioned that VPP classifiers are faster than ACL plugin.

> > > For  field based classification, which one provides

> > > better data

> > plane perf ?

> >

> >

> > It depends. If you wanna simultaneously match on all three, there is

> > currently no mechanism to generically do so.

> >

> > But then every time I looked at the use cases claiming to require

> > that, turned out it was a bad idea to represent the data this way -

> > because of combinatorial explosion. Even ACLs themselves suffer from

> > this issue - N sources times M destinations times K servces equal

> > N*M*K rules, which qui

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Hi Andrew,
   I have to work on make test test case. Before that, I would like to confirm 
whether this is a problem (or) misconfiguration.
   I added 50 rules using acl_add_replace in VAT CLI.  In the ACL dump (show 
acl-plugin acl 0), only 48 rules are present. 2 rules are missing and a default 
rule of “permit all” is also getting added.
   I have put the ACL config and ACL dump info in the attached file.


Thanks
Govind

From: John Lo (loj) 
Sent: Tuesday, April 28, 2020 10:38 PM
To: Govindarajan Mohandoss ; Paul Vinciguerra 

Cc: Andrew  Yourtchenko ; vpp-dev@lists.fd.io; nd 
; Lijian Zhang ; Jieqiang Wang 
; nd 
Subject: RE: [vpp-dev] ACL question

Try “make test TEST=acl_plugin”.   -John

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Tuesday, April 28, 2020 11:22 PM
To: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: Re: [vpp-dev] ACL question

Hi Paul,
  How can I selectively run only the test_acl_plugin.py instead of running make 
test ?

Thanks
Govind

From: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Sent: Tuesday, April 28, 2020 9:22 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

See: src/plugins/acl/test/test_acl_plugin.py

On Tue, Apr 28, 2020 at 7:19 PM Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Sure Andrew. Is there a unit test case for ACL plugin ?

From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
Sent: Tuesday, April 28, 2020 4:57 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

1-3: no.
4: please make a “make test” test case illustrating the problem and share it.
--a

On 28 Apr 2020, at 22:37, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:


Hi Andrew,

  I am working on ACL plugin SF+SL optimization on ARM servers.

  I am finding prefetches in ACL node is becoming bottle neck. I see 
performance improvements on both SL & SF mode, when SF mode bihash table 
related prefetching is disabled.

  I need some help with right ACL config to verify my patch.



 I did the testing with Ingress ACL -- 1 Rule and 50 Rules (Rule:  - DPORT is incremented). The Traffic match all the 50 rules.



  When I tried to add 100 rules on the same rule set in SF mode:

  "acl_add_replace -1 ipv4 permit+reflect src 
192.81.1.1/32<http://192.81.1.1/32> dst 192.82.1.1/32<http://192.82.1.1/32> 
proto 17 sport 100 dport 1,

   ... ,

   ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 100",



   I see only 48 rules in show tables and 48th rule is added as “permit” all 
and not “permit + reflect”. Does it mean <0 – 47> rules will be SF and the rest 
will be in SL mode ?



"

vpp# show acl-plugin acl

acl-index 0 count 49 tag {}

   0: ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 1

   

  47: ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 48

  48: ipv4 permit src 0.0.0.0/0<http://0.0.0.0/0> dst 
0.0.0.0/0<http://0.0.0.0/0> proto 0 sport 0-65535 dport 0-65535

  applied inbound on sw_if_index: 1

  used in lookup context index: 0

"



  1.  Is there a limit of 48 on number of rules that can be added into the Rule 
table (acl-index 0) in SF mode ?
  2.  Whether 48 rules in a ruleset is good enough to verify my optimization 
patch (Traffic flow will match all the 48 rules) ?
  3.  Can I associate more than 1 ACL rule set to an ingress interface (like 
“vat# acl_interface_set_acl_list TenGigabitEthernet1/0/0 input 0 1 2”) ? Each 
Rule set 0, 1, 2 will have different ACL rules. Do I need to test this case 
also to study the performance gain ?
  4.  In SL mode, When I tried to add 100 rules, only 53 rules are seen in show 
table. 53rd rule is added as permit all (Should I read it as permit all ?). Is 
the

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Thanks Paul !

From: Paul Vinciguerra 
Sent: Tuesday, April 28, 2020 9:22 PM
To: Govindarajan Mohandoss 
Cc: Andrew  Yourtchenko ; vpp-dev@lists.fd.io; nd 
; Lijian Zhang ; Jieqiang Wang 

Subject: Re: [vpp-dev] ACL question

See: src/plugins/acl/test/test_acl_plugin.py

On Tue, Apr 28, 2020 at 7:19 PM Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Sure Andrew. Is there a unit test case for ACL plugin ?

From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
Sent: Tuesday, April 28, 2020 4:57 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

1-3: no.
4: please make a “make test” test case illustrating the problem and share it.
--a

On 28 Apr 2020, at 22:37, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:


Hi Andrew,

  I am working on ACL plugin SF+SL optimization on ARM servers.

  I am finding prefetches in ACL node is becoming bottle neck. I see 
performance improvements on both SL & SF mode, when SF mode bihash table 
related prefetching is disabled.

  I need some help with right ACL config to verify my patch.



 I did the testing with Ingress ACL -- 1 Rule and 50 Rules (Rule:  - DPORT is incremented). The Traffic match all the 50 rules.



  When I tried to add 100 rules on the same rule set in SF mode:

  "acl_add_replace -1 ipv4 permit+reflect src 
192.81.1.1/32<http://192.81.1.1/32> dst 192.82.1.1/32<http://192.82.1.1/32> 
proto 17 sport 100 dport 1,

   ... ,

   ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 100",



   I see only 48 rules in show tables and 48th rule is added as “permit” all 
and not “permit + reflect”. Does it mean <0 – 47> rules will be SF and the rest 
will be in SL mode ?



"

vpp# show acl-plugin acl

acl-index 0 count 49 tag {}

   0: ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 1

   

  47: ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 48

  48: ipv4 permit src 0.0.0.0/0<http://0.0.0.0/0> dst 
0.0.0.0/0<http://0.0.0.0/0> proto 0 sport 0-65535 dport 0-65535

  applied inbound on sw_if_index: 1

  used in lookup context index: 0

"



  1.  Is there a limit of 48 on number of rules that can be added into the Rule 
table (acl-index 0) in SF mode ?
  2.  Whether 48 rules in a ruleset is good enough to verify my optimization 
patch (Traffic flow will match all the 48 rules) ?
  3.  Can I associate more than 1 ACL rule set to an ingress interface (like 
“vat# acl_interface_set_acl_list TenGigabitEthernet1/0/0 input 0 1 2”) ? Each 
Rule set 0, 1, 2 will have different ACL rules. Do I need to test this case 
also to study the performance gain ?
  4.  In SL mode, When I tried to add 100 rules, only 53 rules are seen in show 
table. 53rd rule is added as permit all (Should I read it as permit all ?). Is 
there a limit on number of rules in SL mode ?

“

vpp# show acl-plugin acl

acl-index 0 count 54 tag {}

  0: ipv4 permit src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 1

  ….

 52: ipv4 permit src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 53

 53: ipv4 permit src 0.0.0.0/0<http://0.0.0.0/0> dst 
0.0.0.0/0<http://0.0.0.0/0> proto 0 sport 0-65535 dport 0-65535

  applied inbound on sw_if_index: 1

  used in lookup context index: 0

“



Thanks

Govind



> -Original Message-

> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan

> Mohandoss via Lists.Fd.Io<http://Lists.Fd.Io>

> Sent: Friday, March 27, 2020 11:32 AM

> To: Andrew  Yourtchenko mailto:ayour...@gmail.com>>

> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>

> Subject: Re: [vpp-dev] ACL question

>

> Thank you very much Andrew !! I will do some benchmarks and get back to

> you to understand it better.

>

> Thanks

> Govind

>

> > -Original Message-

> > From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>

> > Sent: Friday, March 27, 2020 7:52 AM

> > To: Govindarajan Mohandoss 
> > mailto:govindarajan.mohand...@arm.com>>

> > Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
> > mailto:n...@arm.com>>

> > Subject: Re: [vpp-d

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Hi Paul,
  How can I selectively run only the test_acl_plugin.py instead of running make 
test ?

Thanks
Govind

From: Paul Vinciguerra 
Sent: Tuesday, April 28, 2020 9:22 PM
To: Govindarajan Mohandoss 
Cc: Andrew  Yourtchenko ; vpp-dev@lists.fd.io; nd 
; Lijian Zhang ; Jieqiang Wang 

Subject: Re: [vpp-dev] ACL question

See: src/plugins/acl/test/test_acl_plugin.py

On Tue, Apr 28, 2020 at 7:19 PM Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Sure Andrew. Is there a unit test case for ACL plugin ?

From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
Sent: Tuesday, April 28, 2020 4:57 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

1-3: no.
4: please make a “make test” test case illustrating the problem and share it.
--a

On 28 Apr 2020, at 22:37, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:


Hi Andrew,

  I am working on ACL plugin SF+SL optimization on ARM servers.

  I am finding prefetches in ACL node is becoming bottle neck. I see 
performance improvements on both SL & SF mode, when SF mode bihash table 
related prefetching is disabled.

  I need some help with right ACL config to verify my patch.



 I did the testing with Ingress ACL -- 1 Rule and 50 Rules (Rule:  - DPORT is incremented). The Traffic match all the 50 rules.



  When I tried to add 100 rules on the same rule set in SF mode:

  "acl_add_replace -1 ipv4 permit+reflect src 
192.81.1.1/32<http://192.81.1.1/32> dst 192.82.1.1/32<http://192.82.1.1/32> 
proto 17 sport 100 dport 1,

   ... ,

   ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 100",



   I see only 48 rules in show tables and 48th rule is added as “permit” all 
and not “permit + reflect”. Does it mean <0 – 47> rules will be SF and the rest 
will be in SL mode ?



"

vpp# show acl-plugin acl

acl-index 0 count 49 tag {}

   0: ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 1

   

  47: ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 48

  48: ipv4 permit src 0.0.0.0/0<http://0.0.0.0/0> dst 
0.0.0.0/0<http://0.0.0.0/0> proto 0 sport 0-65535 dport 0-65535

  applied inbound on sw_if_index: 1

  used in lookup context index: 0

"



  1.  Is there a limit of 48 on number of rules that can be added into the Rule 
table (acl-index 0) in SF mode ?
  2.  Whether 48 rules in a ruleset is good enough to verify my optimization 
patch (Traffic flow will match all the 48 rules) ?
  3.  Can I associate more than 1 ACL rule set to an ingress interface (like 
“vat# acl_interface_set_acl_list TenGigabitEthernet1/0/0 input 0 1 2”) ? Each 
Rule set 0, 1, 2 will have different ACL rules. Do I need to test this case 
also to study the performance gain ?
  4.  In SL mode, When I tried to add 100 rules, only 53 rules are seen in show 
table. 53rd rule is added as permit all (Should I read it as permit all ?). Is 
there a limit on number of rules in SL mode ?

“

vpp# show acl-plugin acl

acl-index 0 count 54 tag {}

  0: ipv4 permit src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 1

  ….

 52: ipv4 permit src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 53

 53: ipv4 permit src 0.0.0.0/0<http://0.0.0.0/0> dst 
0.0.0.0/0<http://0.0.0.0/0> proto 0 sport 0-65535 dport 0-65535

  applied inbound on sw_if_index: 1

  used in lookup context index: 0

“



Thanks

Govind



> -Original Message-

> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan

> Mohandoss via Lists.Fd.Io<http://Lists.Fd.Io>

> Sent: Friday, March 27, 2020 11:32 AM

> To: Andrew  Yourtchenko mailto:ayour...@gmail.com>>

> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>

> Subject: Re: [vpp-dev] ACL question

>

> Thank you very much Andrew !! I will do some benchmarks and get back to

> you to understand it better.

>

> Thanks

> Govind

>

> > -Original Message-

> > From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>

> > Sent: Friday, March 27, 2020 7:52 AM

> > To: Govindarajan Mohandoss 
> > mailto:govindarajan.mohand...@arm.com>>

> > Cc: vpp-dev@lists.fd.io<mailto

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Thanks John.

From: John Lo (loj) 
Sent: Tuesday, April 28, 2020 10:38 PM
To: Govindarajan Mohandoss ; Paul Vinciguerra 

Cc: Andrew  Yourtchenko ; vpp-dev@lists.fd.io; nd 
; Lijian Zhang ; Jieqiang Wang 
; nd 
Subject: RE: [vpp-dev] ACL question

Try “make test TEST=acl_plugin”.   -John

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Tuesday, April 28, 2020 11:22 PM
To: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: Re: [vpp-dev] ACL question

Hi Paul,
  How can I selectively run only the test_acl_plugin.py instead of running make 
test ?

Thanks
Govind

From: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Sent: Tuesday, April 28, 2020 9:22 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

See: src/plugins/acl/test/test_acl_plugin.py

On Tue, Apr 28, 2020 at 7:19 PM Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Sure Andrew. Is there a unit test case for ACL plugin ?

From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
Sent: Tuesday, April 28, 2020 4:57 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

1-3: no.
4: please make a “make test” test case illustrating the problem and share it.
--a

On 28 Apr 2020, at 22:37, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:


Hi Andrew,

  I am working on ACL plugin SF+SL optimization on ARM servers.

  I am finding prefetches in ACL node is becoming bottle neck. I see 
performance improvements on both SL & SF mode, when SF mode bihash table 
related prefetching is disabled.

  I need some help with right ACL config to verify my patch.



 I did the testing with Ingress ACL -- 1 Rule and 50 Rules (Rule:  - DPORT is incremented). The Traffic match all the 50 rules.



  When I tried to add 100 rules on the same rule set in SF mode:

  "acl_add_replace -1 ipv4 permit+reflect src 
192.81.1.1/32<http://192.81.1.1/32> dst 192.82.1.1/32<http://192.82.1.1/32> 
proto 17 sport 100 dport 1,

   ... ,

   ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 100",



   I see only 48 rules in show tables and 48th rule is added as “permit” all 
and not “permit + reflect”. Does it mean <0 – 47> rules will be SF and the rest 
will be in SL mode ?



"

vpp# show acl-plugin acl

acl-index 0 count 49 tag {}

   0: ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 1

   

  47: ipv4 permit+reflect src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 48

  48: ipv4 permit src 0.0.0.0/0<http://0.0.0.0/0> dst 
0.0.0.0/0<http://0.0.0.0/0> proto 0 sport 0-65535 dport 0-65535

  applied inbound on sw_if_index: 1

  used in lookup context index: 0

"



  1.  Is there a limit of 48 on number of rules that can be added into the Rule 
table (acl-index 0) in SF mode ?
  2.  Whether 48 rules in a ruleset is good enough to verify my optimization 
patch (Traffic flow will match all the 48 rules) ?
  3.  Can I associate more than 1 ACL rule set to an ingress interface (like 
“vat# acl_interface_set_acl_list TenGigabitEthernet1/0/0 input 0 1 2”) ? Each 
Rule set 0, 1, 2 will have different ACL rules. Do I need to test this case 
also to study the performance gain ?
  4.  In SL mode, When I tried to add 100 rules, only 53 rules are seen in show 
table. 53rd rule is added as permit all (Should I read it as permit all ?). Is 
there a limit on number of rules in SL mode ?

“

vpp# show acl-plugin acl

acl-index 0 count 54 tag {}

  0: ipv4 permit src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 1

  ….

 52: ipv4 permit src 192.81.1.1/32<http://192.81.1.1/32> dst 
192.82.1.1/32<http://192.82.1.1/32> proto 17 sport 100 dport 53

 53: ipv4 per

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Thanks Neale.

From: Neale Ranns (nranns) 
Sent: Wednesday, April 29, 2020 4:24 AM
To: Andrew Yourtchenko ; Govindarajan Mohandoss 

Cc: John Lo (loj) ; Paul Vinciguerra 
; vpp-dev@lists.fd.io; nd ; Lijian 
Zhang ; Jieqiang Wang 
Subject: Re: [vpp-dev] ACL question


Or in the latest version you can create ACLs on the CLI:
  set acl-plugin acl ?
  set acl-plugin interface ?

/neale

From: mailto:vpp-dev@lists.fd.io>> on behalf of Andrew 
Yourtchenko mailto:ayour...@gmail.com>>
Date: Wednesday 29 April 2020 at 10:59
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: "John Lo (loj)" mailto:l...@cisco.com>>, Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>, 
"vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" 
mailto:vpp-dev@lists.fd.io>>, nd 
mailto:n...@arm.com>>, Lijian Zhang 
mailto:lijian.zh...@arm.com>>, Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

Hi Govind,

1) make an api trace and inspect the message there - whether it contains the 
entries you are expecting.

1a) If it does, then you can trivially recreate the same message using the 
python api just by hacking an existing testcase.

1b) if it doesn’t - run the vat itself under gdb and trace how the api message 
is built before it’s sent. I suspect what you are seeing is some issue between 
the command line that you are putting in and the api message being formed.

Also, it may give more clues if you do the above side by side on x86 and arm 
and compare the behaviors.

--a

On 29 Apr 2020, at 06:27, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Hi Andrew,
   I have to work on make test test case. Before that, I would like to confirm 
whether this is a problem (or) misconfiguration.
   I added 50 rules using acl_add_replace in VAT CLI.  In the ACL dump (show 
acl-plugin acl 0), only 48 rules are present. 2 rules are missing and a default 
rule of “permit all” is also getting added.
   I have put the ACL config and ACL dump info in the attached file.


Thanks
Govind

From: John Lo (loj) mailto:l...@cisco.com>>
Sent: Tuesday, April 28, 2020 10:38 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; Paul 
Vinciguerra mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: RE: [vpp-dev] ACL question

Try “make test TEST=acl_plugin”.   -John

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Tuesday, April 28, 2020 11:22 PM
To: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: Re: [vpp-dev] ACL question

Hi Paul,
  How can I selectively run only the test_acl_plugin.py instead of running make 
test ?

Thanks
Govind

From: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Sent: Tuesday, April 28, 2020 9:22 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

See: src/plugins/acl/test/test_acl_plugin.py

On Tue, Apr 28, 2020 at 7:19 PM Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Sure Andrew. Is there a unit test case for ACL plugin ?

From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
Sent: Tuesday, April 28, 2020 4:57 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

1-3: no.
4: please make a “make test” test case illustrating the problem and share it.
--a

On 28 Apr 2020, at 22:37, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:

Hi Andrew,

  I am working on ACL plugin SF+SL optimization on ARM servers.

  I am finding prefetches in ACL node is becoming bottle neck. I see 
performance improvements on both SL & SF mode, when SF mode bihash table 
related prefetching is disabled.

  I need some help with right ACL config to verify my patch.



 I did

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Thanks Andrew. I will investigate more based on your suggestion.

I am seeing the issue in x86 also. I am trying to find similar config 
(tc01-64B-1c-avf-ethip4udp-ip4base-iacl50sf-10kflows-mrr) in CSIT logs. It 
doesn’t have “show acl-plugin acl" dump.
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-mrr-daily-master-2n-skx/888/archives/log.html#s1-s1-s1-s2-s15

But, I am able to create > 48 rules by creating multiple rule tables and 
associate them with ingress interface.

Thanks
Govind

From: Andrew  Yourtchenko 
Sent: Wednesday, April 29, 2020 3:58 AM
To: Govindarajan Mohandoss 
Cc: John Lo (loj) ; Paul Vinciguerra 
; vpp-dev@lists.fd.io; nd ; Lijian 
Zhang ; Jieqiang Wang 
Subject: Re: [vpp-dev] ACL question

Hi Govind,

1) make an api trace and inspect the message there - whether it contains the 
entries you are expecting.

1a) If it does, then you can trivially recreate the same message using the 
python api just by hacking an existing testcase.

1b) if it doesn’t - run the vat itself under gdb and trace how the api message 
is built before it’s sent. I suspect what you are seeing is some issue between 
the command line that you are putting in and the api message being formed.

Also, it may give more clues if you do the above side by side on x86 and arm 
and compare the behaviors.

--a


On 29 Apr 2020, at 06:27, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:

Hi Andrew,
   I have to work on make test test case. Before that, I would like to confirm 
whether this is a problem (or) misconfiguration.
   I added 50 rules using acl_add_replace in VAT CLI.  In the ACL dump (show 
acl-plugin acl 0), only 48 rules are present. 2 rules are missing and a default 
rule of “permit all” is also getting added.
   I have put the ACL config and ACL dump info in the attached file.


Thanks
Govind

From: John Lo (loj) mailto:l...@cisco.com>>
Sent: Tuesday, April 28, 2020 10:38 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; Paul 
Vinciguerra mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: RE: [vpp-dev] ACL question

Try “make test TEST=acl_plugin”.   -John

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Tuesday, April 28, 2020 11:22 PM
To: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: Re: [vpp-dev] ACL question

Hi Paul,
  How can I selectively run only the test_acl_plugin.py instead of running make 
test ?

Thanks
Govind

From: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Sent: Tuesday, April 28, 2020 9:22 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

See: src/plugins/acl/test/test_acl_plugin.py

On Tue, Apr 28, 2020 at 7:19 PM Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Sure Andrew. Is there a unit test case for ACL plugin ?

From: Andrew  Yourtchenko mailto:ayour...@gmail.com>>
Sent: Tuesday, April 28, 2020 4:57 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

1-3: no.
4: please make a “make test” test case illustrating the problem and share it.
--a

On 28 Apr 2020, at 22:37, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:


Hi Andrew,

  I am working on ACL plugin SF+SL optimization on ARM servers.

  I am finding prefetches in ACL node is becoming bottle neck. I see 
performance improvements on both SL & SF mode, when SF mode bihash table 
related prefetching is disabled.

  I need some help with right ACL config to verify my patch.



 I did the testing with Ingress ACL -- 1 Rule and 50 Rules (Rule:  - DPORT is incremented). The Traffic match all the 50 rules.



  When I tried to add 100 rules on the same rule set in SF mode:

  "acl_add_replace -1 ipv4 permit+reflect src 
192.81.

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Hi Neale,
  I tried to use the CLI for ACL in master. But the following command is not 
associating the ACL to ingress interface.
 Show command is not listing anything and “show run time” doesn’t show the ACL 
node.
  Am I missing any config ?

vpp# set acl-plugin acl permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 
proto 17 sport 100 dport 1
ACL index:0
vpp# show acl-plugin acl
acl-index 0 count 1 tag {cli}
  0: ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 
sport 100 dport 1
vpp# set acl-plugin ?
  set acl-plugin acl   set acl-plugin acl  src 
 dst  proto X sport X-Y dport X-Y [tag FOO] {use comma 
separated list for multiple rules}
  set acl-plugin interface set acl-plugin interface  
  [del]
  set acl-plugin   set acl-plugin session timeout {{udp 
idle}|tcp {idle|transient}} 
vpp# set acl-plugin interface TenGigabitEthernet7/0/0 input 0
vpp# show acl-plugin interface   <<< No output.

Thanks
Govind

From: Neale Ranns (nranns) 
Sent: Wednesday, April 29, 2020 4:24 AM
To: Andrew Yourtchenko ; Govindarajan Mohandoss 

Cc: John Lo (loj) ; Paul Vinciguerra 
; vpp-dev@lists.fd.io; nd ; Lijian 
Zhang ; Jieqiang Wang 
Subject: Re: [vpp-dev] ACL question


Or in the latest version you can create ACLs on the CLI:
  set acl-plugin acl ?
  set acl-plugin interface ?

/neale

From: mailto:vpp-dev@lists.fd.io>> on behalf of Andrew 
Yourtchenko mailto:ayour...@gmail.com>>
Date: Wednesday 29 April 2020 at 10:59
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: "John Lo (loj)" mailto:l...@cisco.com>>, Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>, 
"vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" 
mailto:vpp-dev@lists.fd.io>>, nd 
mailto:n...@arm.com>>, Lijian Zhang 
mailto:lijian.zh...@arm.com>>, Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

Hi Govind,

1) make an api trace and inspect the message there - whether it contains the 
entries you are expecting.

1a) If it does, then you can trivially recreate the same message using the 
python api just by hacking an existing testcase.

1b) if it doesn’t - run the vat itself under gdb and trace how the api message 
is built before it’s sent. I suspect what you are seeing is some issue between 
the command line that you are putting in and the api message being formed.

Also, it may give more clues if you do the above side by side on x86 and arm 
and compare the behaviors.

--a

On 29 Apr 2020, at 06:27, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Hi Andrew,
   I have to work on make test test case. Before that, I would like to confirm 
whether this is a problem (or) misconfiguration.
   I added 50 rules using acl_add_replace in VAT CLI.  In the ACL dump (show 
acl-plugin acl 0), only 48 rules are present. 2 rules are missing and a default 
rule of “permit all” is also getting added.
   I have put the ACL config and ACL dump info in the attached file.


Thanks
Govind

From: John Lo (loj) mailto:l...@cisco.com>>
Sent: Tuesday, April 28, 2020 10:38 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; Paul 
Vinciguerra mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: RE: [vpp-dev] ACL question

Try “make test TEST=acl_plugin”.   -John

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Tuesday, April 28, 2020 11:22 PM
To: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: Re: [vpp-dev] ACL question

Hi Paul,
  How can I selectively run only the test_acl_plugin.py instead of running make 
test ?

Thanks
Govind

From: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Sent: Tuesday, April 28, 2020 9:22 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

See: src/plugins/acl/test/test_acl_plugin.py

On Tue, Apr 28, 2020 at 7:19 PM Govindarajan Mohandoss 
mailto:govindarajan.mohand...@a

Re: [vpp-dev] ACL question

[Govindarajan Mohandoss]

Thanks Neale. It works now.

From: Neale Ranns (nranns) 
Sent: Saturday, May 2, 2020 8:15 AM
To: Govindarajan Mohandoss ; Andrew Yourtchenko 

Cc: John Lo (loj) ; Paul Vinciguerra 
; vpp-dev@lists.fd.io; nd ; Lijian 
Zhang ; Jieqiang Wang 
Subject: Re: [vpp-dev] ACL question



From: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Date: Friday 1 May 2020 at 21:15
To: "Neale Ranns (nranns)" mailto:nra...@cisco.com>>, Andrew 
Yourtchenko mailto:ayour...@gmail.com>>
Cc: "John Lo (loj)" mailto:l...@cisco.com>>, Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>, 
"vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" 
mailto:vpp-dev@lists.fd.io>>, nd 
mailto:n...@arm.com>>, Lijian Zhang 
mailto:lijian.zh...@arm.com>>, Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>, nd 
mailto:n...@arm.com>>
Subject: RE: [vpp-dev] ACL question

Hi Neale,
  I tried to use the CLI for ACL in master. But the following command is not 
associating the ACL to ingress interface.
 Show command is not listing anything and “show run time” doesn’t show the ACL 
node.
  Am I missing any config ?

vpp# set acl-plugin acl permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 
proto 17 sport 100 dport 1
ACL index:0
vpp# show acl-plugin acl
acl-index 0 count 1 tag {cli}
  0: ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 
sport 100 dport 1
vpp# set acl-plugin ?
  set acl-plugin acl   set acl-plugin acl  src 
 dst  proto X sport X-Y dport X-Y [tag FOO] {use comma 
separated list for multiple rules}
  set acl-plugin interface set acl-plugin interface  
  [del]
  set acl-plugin   set acl-plugin session timeout {{udp 
idle}|tcp {idle|transient}} 
vpp# set acl-plugin interface TenGigabitEthernet7/0/0 input 0

you have to specify which ACL you want to bind to the interface:
 set acl-plugin interface TenGigabitEthernet7/0/0 input 0 acl 0

/neale

vpp# show acl-plugin interface   <<< No output.

Thanks
Govind

From: Neale Ranns (nranns) mailto:nra...@cisco.com>>
Sent: Wednesday, April 29, 2020 4:24 AM
To: Andrew Yourtchenko mailto:ayour...@gmail.com>>; 
Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: John Lo (loj) mailto:l...@cisco.com>>; Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question


Or in the latest version you can create ACLs on the CLI:
  set acl-plugin acl ?
  set acl-plugin interface ?

/neale

From: mailto:vpp-dev@lists.fd.io>> on behalf of Andrew 
Yourtchenko mailto:ayour...@gmail.com>>
Date: Wednesday 29 April 2020 at 10:59
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Cc: "John Lo (loj)" mailto:l...@cisco.com>>, Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>, 
"vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" 
mailto:vpp-dev@lists.fd.io>>, nd 
mailto:n...@arm.com>>, Lijian Zhang 
mailto:lijian.zh...@arm.com>>, Jieqiang Wang 
mailto:jieqiang.w...@arm.com>>
Subject: Re: [vpp-dev] ACL question

Hi Govind,

1) make an api trace and inspect the message there - whether it contains the 
entries you are expecting.

1a) If it does, then you can trivially recreate the same message using the 
python api just by hacking an existing testcase.

1b) if it doesn’t - run the vat itself under gdb and trace how the api message 
is built before it’s sent. I suspect what you are seeing is some issue between 
the command line that you are putting in and the api message being formed.

Also, it may give more clues if you do the above side by side on x86 and arm 
and compare the behaviors.

--a

On 29 Apr 2020, at 06:27, Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>> wrote:
Hi Andrew,
   I have to work on make test test case. Before that, I would like to confirm 
whether this is a problem (or) misconfiguration.
   I added 50 rules using acl_add_replace in VAT CLI.  In the ACL dump (show 
acl-plugin acl 0), only 48 rules are present. 2 rules are missing and a default 
rule of “permit all” is also getting added.
   I have put the ACL config and ACL dump info in the attached file.


Thanks
Govind

From: John Lo (loj) mailto:l...@cisco.com>>
Sent: Tuesday, April 28, 2020 10:38 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; Paul 
Vinciguerra mailto:pvi...@vinciconsulting.com>>
Cc: Andrew  Yourtchenko mailto:ayour...@gmail.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; nd 
mailto:n...@arm.com>>; Lijian Zhang 
mailto:lijian.zh...@arm.com>>; Jieqiang Wang 
mailto:jieqiang.w...@arm.com&g

Re: [vpp-dev] Help with creating patch

[Govindarajan Mohandoss]

Hi Chris,
  I didn't create a local branch.  Thanks !!
  I didn’t change the subject thinking that it could be related to code freeze. 
Sorry for that.

Thanks
Govind

> -Original Message-
> From: Luke, Chris 
> Sent: Wednesday, May 13, 2020 9:12 PM
> To: Govindarajan Mohandoss ; vpp-
> dev 
> Cc: nd 
> Subject: RE: [vpp-dev] Help with creating patch
> 
> Govind,
> 
> Did you create a branch locally before making a commit? It looks like you
> tried to push to master which won't work. A typical workflow involves
> creating a local branch, making some changes and commits and then pushing
> to Gerrit.
> 
> Also, I changed the email subject; you should really have started a new
> thread instead of replying to an existing thread with something unrelated.
> 
> Chris.
> 
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan
> Mohandoss
> Sent: Wednesday, May 13, 2020 19:24
> To: ayour...@gmail.com; vpp-dev 
> Cc: nd 
> Subject: [EXTERNAL] Re: [vpp-dev] VPP 20.05 RC1 milestone is complete! RC2
> - on Wednesday 20th May
> 
> Hello Maintainers,
>  I am doing the patch submission for the first time.
>  I am following the page
> https://urldefense.com/v3/__https://wiki.fd.io/view/VPP/Pulling,_Building,_
> Running,_Hacking_and_Pushing_VPP_Code*Pulling__;Iw!!CQl3mcHX2A!X_Yl
> Df6H02w8Ew6AQDrBpiMP7UZ5XJeDWGNgAaY0wqMSqos0VyWPgbGH8cP27P
> ol6w$  and getting the error below. Can you please help to fix this ?
> 
> #:~/vpp_external/vpp$ git review
> remote: error: branch refs/publish/master:
> remote: You need 'Create' rights to create new references.
> remote: User: mgovind
> remote: Contact an administrator to fix the permissions
> remote:
> remote: Processing changes: refs: 1
> remote: Processing changes: refs: 1, done To ssh://gerrit.fd.io:29418/vpp
>  ! [remote rejected] HEAD -> refs/publish/master (prohibited by Gerrit: 
> not
> permitted: create)
> error: failed to push some refs to 'ssh://mgov...@gerrit.fd.io:29418/vpp'
> 
> Thanks
> Govind
> 
> > -Original Message-
> > From: vpp-dev@lists.fd.io  On Behalf Of Andrew
> > Yourtchenko via lists.fd.io
> > Sent: Wednesday, May 13, 2020 6:05 PM
> > To: vpp-dev 
> > Subject: [vpp-dev] VPP 20.05 RC1 milestone is complete! RC2 - on
> > Wednesday 20th May
> >
> > Hi all,
> >
> > This is to announce that the VPP 20.05 RC1 milestone is complete!
> >
> > The newly created stable/2005 branch is ready for your fixes in
> > preparation for the RC2 milestone.
> >
> > They need to have a Jira ticket for the issue, and to avoid forgetting
> > adding them to master, where practical, *should* be first merged there
> > and then cherry-picked into the stable/2005 branch - but as soon as
> > the Jira ticket is mentioned in the commit message and the fix ends up
> > in both master and
> > stable/2005 (and if it is important/urgent - maybe earlier branches),
> > then either order is fine.
> >
> > The installation packages for the RC1 for Ubuntu 18.04 and Centos 7
> > from the new branch are available on
> > https://urldefense.com/v3/__https://packagecloud.io/fdio/2005/__;!!CQl
> >
> 3mcHX2A!X_YlDf6H02w8Ew6AQDrBpiMP7UZ5XJeDWGNgAaY0wqMSqos0Vy
> WPgbGH8cM1me
> > QxkA$
> >
> > The master branch is open for all commits.
> >
> > Our next milestone for VPP 20.05 is RC2, happening next Wednesday 20th
> > May.
> >
> > Thanks a lot to Vanessa Valderrama, Dave Wallace and Ed Warnicke for
> > the help!
> >
> > --a
> > /* Your friendly 2005 release manager */
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16372): https://lists.fd.io/g/vpp-dev/message/16372
Mute This Topic: https://lists.fd.io/mt/74197290/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP 20.05 RC1 milestone is complete! RC2 - on Wednesday 20th May

[Govindarajan Mohandoss]

Hello Maintainers,
 I am doing the patch submission for the first time.
 I am following the page 
https://wiki.fd.io/view/VPP/Pulling,_Building,_Running,_Hacking_and_Pushing_VPP_Code#Pulling
 and getting the error below. Can you please help to fix this ?

#:~/vpp_external/vpp$ git review
remote: error: branch refs/publish/master:
remote: You need 'Create' rights to create new references.
remote: User: mgovind
remote: Contact an administrator to fix the permissions
remote:
remote: Processing changes: refs: 1
remote: Processing changes: refs: 1, done
To ssh://gerrit.fd.io:29418/vpp
 ! [remote rejected] HEAD -> refs/publish/master (prohibited by Gerrit: not 
permitted: create)
error: failed to push some refs to 'ssh://mgov...@gerrit.fd.io:29418/vpp'

Thanks
Govind

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Andrew
> Yourtchenko via lists.fd.io
> Sent: Wednesday, May 13, 2020 6:05 PM
> To: vpp-dev 
> Subject: [vpp-dev] VPP 20.05 RC1 milestone is complete! RC2 - on Wednesday
> 20th May
> 
> Hi all,
> 
> This is to announce that the VPP 20.05 RC1 milestone is complete!
> 
> The newly created stable/2005 branch is ready for your fixes in preparation
> for the RC2 milestone.
> 
> They need to have a Jira ticket for the issue, and to avoid forgetting adding
> them to master, where practical, *should* be first merged there and then
> cherry-picked into the stable/2005 branch - but as soon as the Jira ticket is
> mentioned in the commit message and the fix ends up in both master and
> stable/2005 (and if it is important/urgent - maybe earlier branches), then
> either order is fine.
> 
> The installation packages for the RC1 for Ubuntu 18.04 and Centos 7 from the
> new branch are available on https://packagecloud.io/fdio/2005/
> 
> The master branch is open for all commits.
> 
> Our next milestone for VPP 20.05 is RC2, happening next Wednesday 20th
> May.
> 
> Thanks a lot to Vanessa Valderrama, Dave Wallace and Ed Warnicke for the
> help!
> 
> --a
> /* Your friendly 2005 release manager */
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16370): https://lists.fd.io/g/vpp-dev/message/16370
Mute This Topic: https://lists.fd.io/mt/74194208/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Performance on the Macchiatobin

[Govindarajan Mohandoss]

Hi Shmuel,
  We measured IPv4 forwarding performance in the beginning of 2019 (with VPP 
Master branch) and got 5.77 MPPS with single core @ 1.6 GHz. We used Marvell 
I/O Plugin (PMD) and not DPDK. We don’t have the latest performance numbers.  
Currently, our Macchiatobin boards are not functional.  Please contact Nitin 
Saxena for latest updates.

Thanks
Govind

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Shmuel H. via
> lists.fd.io
> Sent: Thursday, May 14, 2020 9:53 AM
> To: vpp-dev@lists.fd.io
> Subject: [vpp-dev] Performance on the Macchiatobin
> 
> Hi,
> 
> I have noticed that there has been some work on the Macchiatobin on vpp
> (as a part of the aarach64 project).
> 
> However, I have not managed to find any performance results for the
> Macchiatobin as it is not a part of CSIT.
> 
> I have found this related(?) [1] JIRA task from 2019/2018, but with no actual
> results.
> 
> If someone has some information about what performance I should expect
> from the Macchiatobin, I would appreciate it a lot.
> 
> [1]: https://jira.fd.io/browse/VPP-1267
> 
> Regards,
> --
> - Shmuel Hazan
> 
> mailto:s...@tkos.co.il | tel:+972-523-746-435 | http://tkos.co.il

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16391): https://lists.fd.io/g/vpp-dev/message/16391
Mute This Topic: https://lists.fd.io/mt/74206564/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Performance on the Macchiatobin

[Govindarajan Mohandoss]

Packet size is 64B.

From: Mrityunjay Kumar 
Sent: Thursday, May 14, 2020 2:18 PM
To: Christian Hopps 
Cc: Govindarajan Mohandoss ; s...@tkos.co.il; 
vpp-dev@lists.fd.io; Nitin Saxena ; nd 
Subject: Re: [vpp-dev] Performance on the Macchiatobin


  >  We measured IPv4 forwarding performance in the beginning of 2019 (with VPP 
Master branch) and got 5.77 MPPS with single core @ 1.6 GHz
---
Friends, what was the packet size with this MPPS. Good if you share some more 
details.

Regards,
Mrityunjay Kumar.
Mobile: +91 - 9731528504


On Fri, May 15, 2020 at 12:20 AM Christian Hopps 
mailto:cho...@chopps.org>> wrote:
We've also seen good numbers from these boxes; however, we have had issues with 
connecting them back-to-back (they don't work that way the interfaces flap), 
they seem to work when connected through a *non-marvel* based switch. IOW 
there's something weird going on with the marvel chip talking to itself (or the 
parts they use).

Then we experienced problems with the HW encryption offload, getting into 
non-recoverable failures. This could be sw bugs either in DPDK or in the 
marvell mvsam support code, or perhaps in the HW, who knows.

-> can you dump the details of issue over the group? if bug with dpdk, will try 
to address.


Not seeing any recent development and being unable to get support from anyone 
for the boxes caused us to stop using them. It's a shame the performance was 
fantastic with the encryption HW so it was a hard choice to make.

If you have time to debug these issues then they might be good HW, otherwise ...

Thanks,
Chris.



> On May 14, 2020, at 2:07 PM, Govindarajan Mohandoss 
> mailto:govindarajan.mohand...@arm.com>> wrote:
>
> Hi Shmuel,
>  We measured IPv4 forwarding performance in the beginning of 2019 (with VPP 
> Master branch) and got 5.77 MPPS with single core @ 1.6 GHz. We used Marvell 
> I/O Plugin (PMD) and not DPDK. We don’t have the latest performance numbers.  
> Currently, our Macchiatobin boards are not functional.  Please contact Nitin 
> Saxena for latest updates.
>
> Thanks
> Govind
>
>> -Original Message-
>> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
>> mailto:vpp-dev@lists.fd.io>> On Behalf Of Shmuel H. via
>> lists.fd.io<http://lists.fd.io>
>> Sent: Thursday, May 14, 2020 9:53 AM
>> To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
>> Subject: [vpp-dev] Performance on the Macchiatobin
>>
>> Hi,
>>
>> I have noticed that there has been some work on the Macchiatobin on vpp
>> (as a part of the aarach64 project).
>>
>> However, I have not managed to find any performance results for the
>> Macchiatobin as it is not a part of CSIT.
>>
>> I have found this related(?) [1] JIRA task from 2019/2018, but with no actual
>> results.
>>
>> If someone has some information about what performance I should expect
>> from the Macchiatobin, I would appreciate it a lot.
>>
>> [1]: https://jira.fd.io/browse/VPP-1267
>>
>> Regards,
>> --
>> - Shmuel Hazan
>>
>> mailto:s...@tkos.co.il<mailto:s...@tkos.co.il> | tel:+972-523-746-435 | 
>> http://tkos.co.il
>
>


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16396): https://lists.fd.io/g/vpp-dev/message/16396
Mute This Topic: https://lists.fd.io/mt/74206564/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] GTPu Question

[Govindarajan Mohandoss]

Thanks Andreas !!


From: Andreas Schultz 
Sent: Tuesday, May 26, 2020 4:14 PM
To: Govindarajan Mohandoss 
Cc: vpp-dev@lists.fd.io; nd 
Subject: Re: [vpp-dev] GTPu Question

Hi Govind,

I'm NOT one of the GTPu maintainers, but I do work on a UPF implementation 
based on VPP.

The GTPu plugin is IMHO mostly useless and broken by design in its current 
state. It assumes that TEIDs are symmetric which they never are in real 3GPP 
settings.
There is an outdated change in gerrit (https://gerrit.fd.io/r/c/vpp/+/13134) 
that corrects that and other deficits.

AFAIK there is no GTP-C control interface that can control the GTP-U plugin 
through VPPs binaries API. Only static tunnels through the CLI are doable (I 
have not tested that myself). A real PGW/GGSN is therefore not doable.

The plugin also has no support for 3GPP compliant charging and QoS. This would 
be a major problem in case you want to evaluate performance as those areas are 
the ones that introduce the highest complexity. Performance tests on pure GTP 
encap/decap are IMHO useless for real world GTP use cases.

Regards,
Andreas

Am Di., 26. Mai 2020 um 05:18 Uhr schrieb Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>:

Dear GTPu Owners,

   I need some help in creating GTPu Origination and Termination config in DUT 
(Running VPP) as described below.



GTPu Origination:



[cid:image005.png@01D63420.3CBD6010]



GTPu Termination:



[cid:image006.png@01D63420.3CBD6010]

Whether GTPu plugin has the support to do such mapping (or) I need to write a 
test plugin to do such mapping ?



I found some information in the below link explaining about GTPu Tunneling.

https://wiki.fd.io/view/VPP/Per-feature_Notes#VRF.2C_Table_Ids_and_Routing

As per the example, foll. are the VPP CLI commands to create a GTPu Tunnel.  
But I don’t follow the commands. Please see inline.



“

loop create

set int state loop0 up

set int ip addr loop0 1.1.1.1/32<http://1.1.1.1/32>   << Can the IP address be 
created on a physical interface connecting the next node in GTPu Origination 
Topology mentioned above ?

create gtpu tunnel src 1.1.1.1 dst 10.6.6.6 teid  decap-next ip4 << What 
does “decap-next” mean ?

set int ip addr gptu_tunnel0 1.1.1.2/31<http://1.1.1.2/31> << Why the IP 
address is assigned to GTPu Tunnel interface ?

“



For the GTPu origination case:

How can I associate the incoming Ethernet traffic to GTPu Tunnel config ?



It will be great if you can share some document / CLI config (or) test case 
which is similar to Origination & Termination topology.



Thanks

Govind



--

Andreas Schultz
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16537): https://lists.fd.io/g/vpp-dev/message/16537
Mute This Topic: https://lists.fd.io/mt/74471469/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] ACL plugin optimization

[Govindarajan Mohandoss]

Hi Andrew,

  While profiling the ACL plugin node using perf tool in ARM Neoverse platform, 
Bihash related prefetches were shown as bottleneck.

Performance improvement is seen in ARM N1, TX2 and Intel Skylake servers after 
removing those prefetches. Testing is done with Ingress ACL/IPv4 forwarding in 
both SF and SL modes.

As the code change is common for Ingress/Egress ACL for both IPv4 and IPv6, 
performance improvement is expected for those cases also.

Following are the test results for Ingress ACL / IPv4 / 1 core / 64B @ MRR in 
ARM N1, TX2 and Intel Skylake servers:



Legend:

===

N1 - ARM Neoverse

TX2 - ARM Thunder X2

SKX - Intel Skylake

SL: % imp - Performance improvement in stateless mode

SF: % imp - Performance improvement in stateful mode






SKX
N1
TX2
Num Rules
Matching Rules
SL: Avg % imp
SF: Avg % imp
SL: % imp
SF: % imp
SL: % imp
SF: % imp
1
1
0.99
12.09
8.38
10.41
4.48
4.63
50
1 (50th)
0.79
9.63
8.76
10.06
5.32
4.63
100
1 (100th)
4.34
10.75
8.60
10.06
6.98
4.63
1000
1(1000th)
4.18
13.06
8.61
11.14
6.17
5.58
100
100
3.70
11.70
6.65
14
2.82
6.53
1000
1000
1.84
15.96
5.52
27.72
4.72
8.69





Please find the patch here: https://gerrit.fd.io/r/c/vpp/+/27167



I ran per patch regression on ARM Taishan server in CSIT lab. Following are the 
results for Stateless and Stateful modes:

1.  perftest-3n-tsh acl_statelessAND1cAND64b:

 https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-tsh/23/consoleFull

 In the log, I can see the comparative numbers between parent and current 
(my patch) for 45 test cases.

 I searched for "Difference of averages relative to parent" in the log -  
41/45 test cases have shown around 4% improvement with the patch. Rest of the 4 
test cases stayed neutral.



2. perftest-3n-tsh acl_statefulAND1cAND64b:

https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-tsh/25/

Performance improvement is seen in all 36 test cases.



Please provide your comments.



Thanks

Govind


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16539): https://lists.fd.io/g/vpp-dev/message/16539
Mute This Topic: https://lists.fd.io/mt/74507621/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] GTPu Question

[Govindarajan Mohandoss]

Dear GTPu Owners,

   I need some help in creating GTPu Origination and Termination config in DUT 
(Running VPP) as described below.



GTPu Origination:



[cid:image002.png@01D632DC.1A086500]



GTPu Termination:



[cid:image004.png@01D632DD.4A324CE0]

Whether GTPu plugin has the support to do such mapping (or) I need to write a 
test plugin to do such mapping ?



I found some information in the below link explaining about GTPu Tunneling.

https://wiki.fd.io/view/VPP/Per-feature_Notes#VRF.2C_Table_Ids_and_Routing

As per the example, foll. are the VPP CLI commands to create a GTPu Tunnel.  
But I don't follow the commands. Please see inline.



"

loop create

set int state loop0 up

set int ip addr loop0 1.1.1.1/32   << Can the IP address be created on a 
physical interface connecting the next node in GTPu Origination Topology 
mentioned above ?

create gtpu tunnel src 1.1.1.1 dst 10.6.6.6 teid  decap-next ip4 << What 
does "decap-next" mean ?

set int ip addr gptu_tunnel0 1.1.1.2/31 << Why the IP address is assigned to 
GTPu Tunnel interface ?

"



For the GTPu origination case:

How can I associate the incoming Ethernet traffic to GTPu Tunnel config ?



It will be great if you can share some document / CLI config (or) test case 
which is similar to Origination & Termination topology.



Thanks

Govind


image001.emz
Description: image001.emz


oledata.mso
Description: oledata.mso


image003.emz
Description: image003.emz
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16484): https://lists.fd.io/g/vpp-dev/message/16484
Mute This Topic: https://lists.fd.io/mt/74471469/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ACL plugin optimization

[Govindarajan Mohandoss]

Thanks Andrew. I will fix the issue and get back to you.

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Andrew
> Yourtchenko via lists.fd.io
> Sent: Wednesday, May 27, 2020 4:51 PM
> To: Govindarajan Mohandoss 
> Cc: vpp-dev@lists.fd.io; Lijian Zhang ; Jieqiang
> Wang ; Honnappa Nagarahalli
> ; nd 
> Subject: Re: [vpp-dev] ACL plugin optimization
> 
> Hi Govind,
> 
> 1) According to Jenkins, this patch permits some of the packets that should
> be denied, hence JJB voted "-1".
> 
> 2) If you suspect merely the prefetches are the issue, just commenting out
> the body of prefetch_session_entry() in the original code should turn it into 
> a
> no-op that doesn't break anything.
> 
> Hard to say anything else given the functionality is not correct.
> 
> In general - ensure you run "EXTENDED_TESTS=y TEST=acl* make test" as a
> sanity check before extensive perf-tests. It's not a 100% guarantee but it 
> does
> catch a few naughty cases.
> 
> Also - take a look at f1cd92d8d9, which got about 30% improvement back in
> the day, and is the source of much of the trickiness in that node.
> 
> --a
> 
> 
> On 5/27/20, Govindarajan Mohandoss
>  wrote:
> > Hi Andrew,
> >
> >   While profiling the ACL plugin node using perf tool in ARM Neoverse
> > platform, Bihash related prefetches were shown as bottleneck.
> >
> > Performance improvement is seen in ARM N1, TX2 and Intel Skylake
> > servers after removing those prefetches. Testing is done with Ingress
> > ACL/IPv4 forwarding in both SF and SL modes.
> >
> > As the code change is common for Ingress/Egress ACL for both IPv4 and
> > IPv6, performance improvement is expected for those cases also.
> >
> > Following are the test results for Ingress ACL / IPv4 / 1 core / 64B @
> > MRR in ARM N1, TX2 and Intel Skylake servers:
> >
> >
> >
> > Legend:
> >
> > ===
> >
> > N1 - ARM Neoverse
> >
> > TX2 - ARM Thunder X2
> >
> > SKX - Intel Skylake
> >
> > SL: % imp - Performance improvement in stateless mode
> >
> > SF: % imp - Performance improvement in stateful mode
> >
> >
> >
> >
> >
> >
> > SKX
> > N1
> > TX2
> > Num Rules
> > Matching Rules
> > SL: Avg % imp
> > SF: Avg % imp
> > SL: % imp
> > SF: % imp
> > SL: % imp
> > SF: % imp
> > 1
> > 1
> > 0.99
> > 12.09
> > 8.38
> > 10.41
> > 4.48
> > 4.63
> > 50
> > 1 (50th)
> > 0.79
> > 9.63
> > 8.76
> > 10.06
> > 5.32
> > 4.63
> > 100
> > 1 (100th)
> > 4.34
> > 10.75
> > 8.60
> > 10.06
> > 6.98
> > 4.63
> > 1000
> > 1(1000th)
> > 4.18
> > 13.06
> > 8.61
> > 11.14
> > 6.17
> > 5.58
> > 100
> > 100
> > 3.70
> > 11.70
> > 6.65
> > 14
> > 2.82
> > 6.53
> > 1000
> > 1000
> > 1.84
> > 15.96
> > 5.52
> > 27.72
> > 4.72
> > 8.69
> >
> >
> >
> >
> >
> > Please find the patch here: https://gerrit.fd.io/r/c/vpp/+/27167
> >
> >
> >
> > I ran per patch regression on ARM Taishan server in CSIT lab.
> > Following are the results for Stateless and Stateful modes:
> >
> > 1.  perftest-3n-tsh acl_statelessAND1cAND64b:
> >
> >
> > https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-tsh/23/consol
> > eFull
> >
> >  In the log, I can see the comparative numbers between parent and
> > current (my patch) for 45 test cases.
> >
> >  I searched for "Difference of averages relative to parent" in the
> > log -
> >  41/45 test cases have shown around 4% improvement with the patch.
> > Rest of the 4 test cases stayed neutral.
> >
> >
> >
> > 2. perftest-3n-tsh acl_statefulAND1cAND64b:
> >
> > https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-tsh/25/
> >
> > Performance improvement is seen in all 36 test cases.
> >
> >
> >
> > Please provide your comments.
> >
> >
> >
> > Thanks
> >
> > Govind
> >
> >
> >
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16582): https://lists.fd.io/g/vpp-dev/message/16582
Mute This Topic: https://lists.fd.io/mt/74507621/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ACL plugin optimization

[Govindarajan Mohandoss]

Thanks Neale. If will fix it and recheck.

> -Original Message-
> From: Neale Ranns (nranns) 
> Sent: Thursday, May 28, 2020 1:56 AM
> To: Andrew Yourtchenko ; Govindarajan Mohandoss
> 
> Cc: vpp-dev@lists.fd.io; Lijian Zhang ; Jieqiang
> Wang ; Honnappa Nagarahalli
> ; nd 
> Subject: Re: [vpp-dev] ACL plugin optimization
> 
> 
> Hi Govind,
> 
> As well as removing the prefetches, you've also removed the per packet call
> to acl_fa_find_session_with_hash(). So IIUC you've removed the per-packet
> session lookup and instead re-use the lookup of packet 0 each time. that'll
> make things quicker but it's not functionally correct.
> 
> /neale
> 
> On 27/05/2020 23:51, "vpp-dev@lists.fd.io on behalf of Andrew
> Yourtchenko"  wrote:
> 
> Hi Govind,
> 
> 1) According to Jenkins, this patch permits some of the packets that
> should be denied, hence JJB voted "-1".
> 
> 2) If you suspect merely the prefetches are the issue, just commenting
> out the body of prefetch_session_entry() in the original code should
> turn it into a no-op that doesn't break anything.
> 
> Hard to say anything else given the functionality is not correct.
> 
> In general - ensure you run "EXTENDED_TESTS=y TEST=acl* make test" as
> a sanity check before extensive perf-tests. It's not a 100% guarantee
> but it does catch a few naughty cases.
> 
> Also - take a look at f1cd92d8d9, which got about 30% improvement back
> in the day, and is the source of much of the trickiness in that node.
> 
> --a
> 
> 
> On 5/27/20, Govindarajan Mohandoss
>  wrote:
> > Hi Andrew,
> >
> >   While profiling the ACL plugin node using perf tool in ARM Neoverse
> > platform, Bihash related prefetches were shown as bottleneck.
> >
> > Performance improvement is seen in ARM N1, TX2 and Intel Skylake
> servers
> > after removing those prefetches. Testing is done with Ingress ACL/IPv4
> > forwarding in both SF and SL modes.
> >
> > As the code change is common for Ingress/Egress ACL for both IPv4 and
> IPv6,
> > performance improvement is expected for those cases also.
> >
> > Following are the test results for Ingress ACL / IPv4 / 1 core / 64B @ 
> MRR
> > in ARM N1, TX2 and Intel Skylake servers:
> >
> >
> >
> > Legend:
> >
> > ===
> >
> > N1 - ARM Neoverse
> >
> > TX2 - ARM Thunder X2
> >
> > SKX - Intel Skylake
> >
> > SL: % imp - Performance improvement in stateless mode
> >
> > SF: % imp - Performance improvement in stateful mode
> >
> >
> >
> >
> >
> >
> > SKX
> > N1
> > TX2
> > Num Rules
> > Matching Rules
> > SL: Avg % imp
> > SF: Avg % imp
> > SL: % imp
> > SF: % imp
> > SL: % imp
> > SF: % imp
> > 1
> > 1
> > 0.99
> > 12.09
> > 8.38
> > 10.41
> > 4.48
> > 4.63
> > 50
> > 1 (50th)
> > 0.79
> > 9.63
> > 8.76
> > 10.06
> > 5.32
> > 4.63
> > 100
> > 1 (100th)
> > 4.34
> > 10.75
> > 8.60
> > 10.06
> > 6.98
> > 4.63
> > 1000
> > 1(1000th)
> > 4.18
> > 13.06
> > 8.61
> > 11.14
> > 6.17
> > 5.58
> > 100
> > 100
> > 3.70
> > 11.70
> > 6.65
> > 14
> > 2.82
> > 6.53
> > 1000
> > 1000
> > 1.84
> > 15.96
> > 5.52
> > 27.72
> > 4.72
> > 8.69
> >
> >
> >
> >
> >
> > Please find the patch here: https://gerrit.fd.io/r/c/vpp/+/27167
> >
> >
> >
> > I ran per patch regression on ARM Taishan server in CSIT lab. Following
> are
> > the results for Stateless and Stateful modes:
> >
> > 1.  perftest-3n-tsh acl_statelessAND1cAND64b:
> >
> >
> > https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-
> tsh/23/consoleFull
> >
> >  In the log, I can see the comparative numbers between parent and
> > current (my patch) for 45 test cases.
> >
> >  I searched for "Difference of averages relative to parent" in the 
> log -
> >  41/45 test cases have shown around 4% improvement with the patch.
> Rest of
> > the 4 test cases stayed neutral.
> >
> >
> >
> > 2. perftest-3n-tsh acl_statefulAND1cAND64b:
> >
> > https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-tsh/25/
> >
> > Performance improvement is seen in all 36 test cases.
> >
> >
> >
> > Please provide your comments.
> >
> >
> >
> > Thanks
> >
> > Govind
> >
> >
> >

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16581): https://lists.fd.io/g/vpp-dev/message/16581
Mute This Topic: https://lists.fd.io/mt/74507621/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXT] [vpp-dev] New perfmon plugin

[Govindarajan Mohandoss]

Please find the list of perf events on TX2 and N1SDP attached.

Thanks
Govind

From: vpp-dev@lists.fd.io  On Behalf Of Damjan Marion via 
lists.fd.io
Sent: Monday, December 14, 2020 8:03 AM
To: Honnappa Nagarahalli 
Cc: nsax...@marvell.com; vpp-dev ; nd 
Subject: Re: [EXT] [vpp-dev] New perfmon plugin


Thanks both,

So on N1SDP and ThunderX2 we have: armv8_pmuv3_0
In addition ThunderX2 have few uncore_* PMUs

On OcteonTX2 we have armv8_cavium_thunder which seems to be different than 
armv8_pmuv3_0.

Will be interesting to see output of “grep . events/*“ so we can compare 
counter names and IDs….

—
Damjan


> On 14.12.2020., at 14:54, Honnappa Nagarahalli 
> mailto:honnappa.nagaraha...@arm.com>> wrote:
>
> 
>
>>
>> Hi Damjan,
>>
>> ARM defines two sets of performance monitoring counters and extension
>> 1. Common Event number and micro-architecture events defined by ARM
>> which every chip vendor should implement.
>> 2. Chip vendor specific PMU counters other than (1)
>>
>> I am not in ThunderX2 BU but I think the kernel driver you are referring to
>> seems to be a PMU extension which falls under the category of (2) above. See
>> below for OCTEONTX2 output
>>
>> So for ARM to be enabled in perfmon plugin, I am thinking,
>> - we need common bundle to register common ARM PMU events. This should
>> be first step and include most of the useful/important events
>> - chip vendor specific bundle should also be allowed to "implementation
>> defined" PMU events
>>
>> One of the key differentiation in ARM is a kernel driver needs to be hooked 
>> at
>> runtime to allow VPP to get hold of PMU counters (which is not the case with
>> x86)
>>
> Can you capture contents of /sys/bus/event_source/devices/ from one
>> system?
>> I do not have ThunderX2 access but here is the output of OCTEONTX2
> On thunderx2:
> honnag01@2u-thunderx2:~$ ls /sys/bus/event_source/devices/
> armv8_pmuv3_0  breakpoint  kprobe  software  tracepoint  uncore_dmc_0  
> uncore_dmc_1  uncore_l3c_0  uncore_l3c_1  uprobe
>
> 2u-thunderx2:~$ ls /sys/bus/event_source/devices/uncore_dmc_0/
> cpumask  events  format  perf_event_mux_interval_ms  power  subsystem  type  
> uevent
>
>>
>> $ ls -ltr /sys/bus/event_source/devices/ total 0 lrwxrwxrwx 1 root root 0 Dec
>> 14 06:48 software -> ../../../devices/software lrwxrwxrwx 1 root root 0 Dec 
>> 14
>> 06:48 cs_etm -> ../../../devices/cs_etm lrwxrwxrwx 1 root root 0 Dec 14 06:48
>> breakpoint -> ../../../devices/breakpoint lrwxrwxrwx 1 root root 0 Dec 14 
>> 06:48
>> tracepoint -> ../../../devices/tracepoint lrwxrwxrwx 1 root root 0 Dec 14 
>> 06:48
>> armv8_cavium_thunder -> ../../../devices/armv8_cavium_thunder
>>
>> Thanks,
>> Nitin
>>
>>
>>
>>
>>> -Original Message-
>>> From: Damjan Marion mailto:dmar...@me.com>>
>>> Sent: Monday, December 14, 2020 4:19 PM
>>> To: Nitin Saxena mailto:nsax...@marvell.com>>
>>> Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>
>>> Subject: Re: [EXT] [vpp-dev] New perfmon plugin
>>>
>>>
>>> Isn’t there also uncore PMU? I can see some thunderx2 specific driver
>>> in kernel source.
>>>
>>> Can you capture contents of /sys/bus/event_source/devices/ from one
>>> system?
>>>
>>> Thanks,
>>>
>>> —
>>> Damjan
>>>
>>>
 On 14.12.2020., at 09:09, Nitin Saxena 
 mailto:nsax...@marvell.com>> wrote:

 Yes most of the ARM processors including ThunderX2, OCTEONTX2 has
>>> PMU as per AARCH64 specifications. I did some analysis to add ARM
>>> support in older perfmon plugin and should be easy to port to this new
>>> one. This is something in TODO list which is much needed for us and
>>> overall ARM

 Thanks,
 Nitin

> -Original Message-
> From: Damjan Marion mailto:dmar...@me.com>>
> Sent: Saturday, December 12, 2020 7:46 PM
> To: Nitin Saxena mailto:nsax...@marvell.com>>
> Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>
> Subject: Re: [EXT] [vpp-dev] New perfmon plugin
>
>
> cool, if I got it right ThunderX2 have own PMU so we can add it as
> new source and create specific bundles.
>
> --
> Damjan
>
>> On 12.12.2020., at 11:07, Nitin Saxena 
>> mailto:nsax...@marvell.com>> wrote:
>>
>> Hi Damjan,
>>
>> I was already fan of older perfmon plugin and new one seems
>> superset of the older one (at-least from video)
>>
>> Nice addition
>>
>> Thanks,
>> Nitin
>>
>>> -Original Message-
>>> From: vpp-dev@lists.fd.io 
>>> mailto:vpp-dev@lists.fd.io>> On Behalf Of
>>> Damjan Marion via lists.fd.io
>>> Sent: Friday, December 11, 2020 9:44 PM
>>> To: vpp-dev mailto:vpp-dev@lists.fd.io>>
>>> Subject: [EXT] [vpp-dev] New perfmon plugin
>>>
>>> External Email
>>>
>>> -
>>> 
>>> -
>>>
>>> Guys,
>>>
>>> I just submitted patch with the new perfmon plugin: [1]
>>>
>>> It takes 

Re: [vpp-dev] Make test help

[Govindarajan Mohandoss]

Thanks Klement.

> -Original Message-
> From: Klement Sekera -X (ksekera - PANTHEON TECH SRO at Cisco)
> 
> Sent: Monday, April 26, 2021 8:00 AM
> To: Govindarajan Mohandoss 
> Cc: vpp-dev ; nd 
> Subject: Re: [vpp-dev] Make test help
> 
> Hi Govind,
> 
> there is no explicit startup.conf used by test framework. All arguments are
> passed using VPP command line built setUpConstants() function of
> VppTestCase.
> 
> Regards,
> Klement
> 
> > On 23 Apr 2021, at 18:13, Govindarajan Mohandoss
>  wrote:
> >
> > Dear Maintainers,
> >  I would like to enable a field in "startup.conf" through "make test". How
> can I do that ?
> >
> > Thanks
> > Govind
> >
> > 
> >


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19284): https://lists.fd.io/g/vpp-dev/message/19284
Mute This Topic: https://lists.fd.io/mt/82315155/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp_papi.py error

[Govindarajan Mohandoss]

Hi Hemant,
  We saw this issue and submitted a patch.
   https://gerrit.fd.io/r/c/vpp/+/32172

Thanks
Govind

From: vpp-dev@lists.fd.io  On Behalf Of hemant via 
lists.fd.io
Sent: Friday, April 30, 2021 3:12 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] vpp_papi.py error

I git cloned vpp gerrit repo yesterday and during using "sudo make 
build-release" I run into this error.  Is a bug filed for this issue?  Python 
2.7 doesn't like a "*" for an arg.

make[3]: Entering directory '/home/hemant/vpp/build-root/build-vpp-native/vpp'
  File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_papi.py", line 389
def __init__(self, *, apifiles=None, testmode=False, async_thread=True,
^
SyntaxError: invalid syntax

Regards,

Hemant

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19306): https://lists.fd.io/g/vpp-dev/message/19306
Mute This Topic: https://lists.fd.io/mt/82490885/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp_papi.py error

[Govindarajan Mohandoss]

Thanks Paul. That was a quick fix that worked for us. We will drop it.

From: Paul Vinciguerra 
Sent: Friday, April 30, 2021 5:16 PM
To: hem...@mnkcg.com
Cc: Govindarajan Mohandoss ; vpp-dev 
; Zachary Leaf ; nd 
Subject: Re: [vpp-dev] vpp_papi.py error

Please see [0] from Dec.

This proposed change has a breaking impact further down the line.
It has been a more reliable fix for me at least to instead modify the makefile 
to do `python3 -m pip install cmake`, which is a proven technique and has been 
used with older distros in the CI for a good while now.

Paul

https://gerrit.fd.io/r/c/vpp/+/30401/6/src/vpp-api/python/CMakeLists.txt


On Fri, Apr 30, 2021 at 5:45 PM hemant via lists.fd.io<http://lists.fd.io> 
mailto:mnkcg@lists.fd.io>> wrote:
Govind,

Thanks!

Hemant

From: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Sent: Friday, April 30, 2021 5:43 PM
To: hem...@mnkcg.com<mailto:hem...@mnkcg.com>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Cc: Zachary Leaf mailto:zachary.l...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: RE: [vpp-dev] vpp_papi.py error

Hi Hemant,
  We saw this issue and submitted a patch.
   https://gerrit.fd.io/r/c/vpp/+/32172

Thanks
Govind

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of hemant via 
lists.fd.io<http://lists.fd.io>
Sent: Friday, April 30, 2021 3:12 PM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [vpp-dev] vpp_papi.py error

I git cloned vpp gerrit repo yesterday and during using “sudo make 
build-release” I run into this error.  Is a bug filed for this issue?  Python 
2.7 doesn’t like a “*” for an arg.

make[3]: Entering directory '/home/hemant/vpp/build-root/build-vpp-native/vpp'
  File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_papi.py", line 389
def __init__(self, *, apifiles=None, testmode=False, async_thread=True,
^
SyntaxError: invalid syntax

Regards,

Hemant



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19309): https://lists.fd.io/g/vpp-dev/message/19309
Mute This Topic: https://lists.fd.io/mt/82490885/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Make test help

[Govindarajan Mohandoss]

Dear Maintainers,
  I would like to enable a field in "startup.conf" through "make test". How can 
I do that ?

Thanks
Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19280): https://lists.fd.io/g/vpp-dev/message/19280
Mute This Topic: https://lists.fd.io/mt/82315155/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] IPSec proposal to improve "ipsec4-output-feature" node performance

[Govindarajan Mohandoss]

Hi Neale,
   I have completed the flow cache implementation for SPD lookup in IPv4/IPSec 
outbound direction.  Performance numbers are good with null encryption on 
single core/64B @ MRR. Please provide your comments.
https://gerrit.fd.io/r/c/vpp/+/31694

  Summary of flow cache implementation:

  1.  Based on Bihash without collision handling. This will avoid the overhead 
to recycle/age out old entries in flow cache. Whenever collision occurs, the 
old entry will be overwritten by new entry in data plane.
  2.  Size of flow cache is fixed. Currently set to handle 1 million flows. 
This can be made configurable as a next step.
  3.  Whenever an SPD rule is added/deleted from the control plane, flow cache 
entries are flushed from control plane. Before flushing, the data plane is put 
in fall back mode to bypass flow cache and do linear lookup. Flushing is done 
only after the inflight packets are sent out from all the worker cores.

Thanks
Govind

From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Wednesday, March 3, 2021 7:55 PM
To: Neale Ranns ; vpp-dev 
Cc: nd ; nd 
Subject: Re: [vpp-dev] IPSec proposal to improve "ipsec4-output-feature" node 
performance

Hi Neale,
  Thank you for your comments. I know you would have thought about it already. 
I can work with you to implement the right solution to improve performance.
  Please see my response inline.

Thanks
Govind

From: Neale Ranns mailto:ne...@graphiant.com>>
Sent: Wednesday, March 3, 2021 8:45 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec proposal to improve "ipsec4-output-feature" node 
performance


Hi Govind,

Flow caches always perform well, but they are more difficult to use than they 
first appear. Consider asking yourself these questions:
1 - how many entries can the cache contain?
>> This can be made configurable as per the system need. By default, we can 
>> allocate the hash table size to hold 10K entries.

2 - what do you do when the cache is full? How do you age or recycle old flows?
>> If the flow cache is implemented using a hash table without collision 
>> handling, then age out mechanism is not needed. Whenever a collision occurs,
old entry can be overwritten with new entry. Worst case will be 255 overwrites, 
if all the 256 packets per batch result in same hash value.

3 - how do you flush the cache when the policy set changes?
>> Whenever an SPD rule is deleted, the flow cache will be flushed completely 
>> in the control plane. An IPSec module level flag will be introduced and set 
>> by the
control plane to put the data plane in fall back mode to use linear search. 
This flag will be reset once the control plane flush the flow cache and delete 
the
SPD rule from SPD table. Also, data plane will not add new entry into the flow 
cache during SPD rule deletion.
I have added this logic in my prototype. Please find the changes attached.

In general, what is the rate at which an SPD rule will be deleted by the 
application ? If the deletion rate is low, then we can take the hit of flushing 
the flow cache in control plane.

I had considered in the past changing an SPD definition to use IP subnets 
(rather than IP ranges) and then re-use the tuple-sort/merge algorithm used by 
ACLs. This approach would not need you to answer the awkward questions about a 
cache and it would break the linear dependent lookup (it has other 
dependencies, but they are much better). Two reasons I didn't do this 1) no 
time 2) ipsec is a vnet component and ACL is a plugin, a vnet -> plugin 
dependency is a no-no. If you're lucky some-one might volunteer to make IPsec a 
plugin and this will go away...
>> Please correct my understanding.
In this method, the mask have to be created for every SPD rule and stored in an 
array. On every packet arrival, the mask will be picked up in linear fashion 
and hash will
be computed based on mask and packet header fields. Then bihash will be looked 
up with that hash value. This reduces the overhead of comparing the ranges 
during
linear search. But the mask lookup is still linear. I agree that there will be 
a performance improvement because the range comparison is avoided for every SPD 
entry.
Is there a way to implement it without creating IPSec plugin and without 
depending on ACL plugin ?


/neale


From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> on behalf of Govindarajan 
Mohandoss via lists.fd.io 
mailto:Govindarajan.mohandoss=arm@lists.fd.io>>
Date: Wednesday, 3 March 2021 at 06:57
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: nd mailto:n...@arm.com>>
Subject: [vpp-dev] IPSec proposal to improve "ipsec4-output-feature" node 
performance
Hi Neale,
   I am working on optimizing "ipsec

Re: [vpp-dev] IPSec ESP Tunnel mode config

[Govindarajan Mohandoss]

Thanks Neale. I will try it out.

From: Neale Ranns 
Sent: Thursday, February 25, 2021 3:16 AM
To: Govindarajan Mohandoss ; vpp-dev 

Cc: nd 
Subject: Re: [vpp-dev] IPSec ESP Tunnel mode config

Hi Govind,

Please see:
  https://wiki.fd.io/view/VPP/IPSec

/neale

From: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Date: Wednesday, 24 February 2021 at 20:34
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>, Neale 
Ranns mailto:ne...@graphiant.com>>, vpp-dev 
mailto:vpp-dev@lists.fd.io>>
Cc: nd mailto:n...@arm.com>>, nd 
mailto:n...@arm.com>>
Subject: RE: [vpp-dev] IPSec ESP Tunnel mode config
Hi Neale,
  I was wrong. I did a packet capture in null-encryption mode and the packet 
format is of ESP Transport mode type.
   Can you please help me to config ESP Tunnel mode ? Do you have any 
logs/document to refer ?

NULL encryption config:

vpp# create ipip tunnel src 192.83.1.1 dst 192.83.1.2
ipip0
vpp# ipsec sa add 20 spi 1000 esp crypto-alg none integ-alg none
vpp# ipsec tunnel protect ipip0 sa-out 20 add

Thanks
Govind

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Wednesday, February 24, 2021 10:00 AM
To: ne...@graphiant.com<mailto:ne...@graphiant.com>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>
Cc: nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec ESP Tunnel mode config

Thank you Neale. Following set of commands worked.  I hope it is correct.

vpp# create ipip tunnel src 192.83.1.1 dst 192.83.1.2
ipip0
vpp# ipsec sa add 20 spi 1000 crypto-alg aes-gcm-256 crypto-key 
0123456789012345678901234567890101234567890123456789012345678901 salt 0x12345678
vpp# ipsec tunnel protect ipip0 sa-out 20

Foll. command didn't work:
ipsec sa add 20 spi 1000 esp crypto-alg aes-gcm-128 crypto-key 
4a506a794f574265564551694d653768 salt 0x12345678 tunnel src 192.83.1.1 dst 
192.83.1.2

Thanks
Govind

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Neale Ranns via 
lists.fd.io
Sent: Wednesday, February 24, 2021 9:20 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] IPSec ESP Tunnel mode config

Dear Govind,

The tunnel parameters are parsed separately in recent versions. Try:

  ipsec sa add 20 spi 1000 esp crypto-alg aes-gcm-128 crypto-key 
4a506a794f574265564551694d653768 salt 0x12345678 tunnel src 192.83.1.1 dst 
192.83.1.2

/neale

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> on behalf of Govindarajan 
Mohandoss via lists.fd.io 
mailto:Govindarajan.mohandoss=arm@lists.fd.io>>
Date: Wednesday, 24 February 2021 at 15:59
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: nd mailto:n...@arm.com>>, nd 
mailto:n...@arm.com>>
Subject: [vpp-dev] IPSec ESP Tunnel mode config

Dear Maintainers,

   I need help to fix ESP Tunnel mode configuration using debug CLI.

   Following command is throwing parse error. Can you please share the latest 
CLI command ?



vpp# ipsec sa add 20 spi 1000 esp tunnel-src 192.83.1.1 tunnel-dst 192.83.1.2 
crypto-alg aes-gcm-128 crypto-key 4a506a794f574265564551694d653768 salt 
0x12345678

ipsec sa: parse error: '-src 192.83.1.1 tunnel-dst 192...'



Thanks

Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18806): https://lists.fd.io/g/vpp-dev/message/18806
Mute This Topic: https://lists.fd.io/mt/80878044/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] IPSec ESP Tunnel mode config

[Govindarajan Mohandoss]

Dear Maintainers,

   I need help to fix ESP Tunnel mode configuration using debug CLI.

   Following command is throwing parse error. Can you please share the latest 
CLI command ?



vpp# ipsec sa add 20 spi 1000 esp tunnel-src 192.83.1.1 tunnel-dst 192.83.1.2 
crypto-alg aes-gcm-128 crypto-key 4a506a794f574265564551694d653768 salt 
0x12345678

ipsec sa: parse error: '-src 192.83.1.1 tunnel-dst 192...'



Thanks

Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18795): https://lists.fd.io/g/vpp-dev/message/18795
Mute This Topic: https://lists.fd.io/mt/80878044/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] IPSec ESP Tunnel mode config

[Govindarajan Mohandoss]

Hi Neale,
  I was wrong. I did a packet capture in null-encryption mode and the packet 
format is of ESP Transport mode type.
   Can you please help me to config ESP Tunnel mode ? Do you have any 
logs/document to refer ?

NULL encryption config:

vpp# create ipip tunnel src 192.83.1.1 dst 192.83.1.2
ipip0
vpp# ipsec sa add 20 spi 1000 esp crypto-alg none integ-alg none
vpp# ipsec tunnel protect ipip0 sa-out 20 add

Thanks
Govind

From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Wednesday, February 24, 2021 10:00 AM
To: ne...@graphiant.com; vpp-dev 
Cc: nd 
Subject: Re: [vpp-dev] IPSec ESP Tunnel mode config

Thank you Neale. Following set of commands worked.  I hope it is correct.

vpp# create ipip tunnel src 192.83.1.1 dst 192.83.1.2
ipip0
vpp# ipsec sa add 20 spi 1000 crypto-alg aes-gcm-256 crypto-key 
0123456789012345678901234567890101234567890123456789012345678901 salt 0x12345678
vpp# ipsec tunnel protect ipip0 sa-out 20

Foll. command didn't work:
ipsec sa add 20 spi 1000 esp crypto-alg aes-gcm-128 crypto-key 
4a506a794f574265564551694d653768 salt 0x12345678 tunnel src 192.83.1.1 dst 
192.83.1.2

Thanks
Govind

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Neale Ranns via 
lists.fd.io
Sent: Wednesday, February 24, 2021 9:20 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] IPSec ESP Tunnel mode config

Dear Govind,

The tunnel parameters are parsed separately in recent versions. Try:

  ipsec sa add 20 spi 1000 esp crypto-alg aes-gcm-128 crypto-key 
4a506a794f574265564551694d653768 salt 0x12345678 tunnel src 192.83.1.1 dst 
192.83.1.2

/neale

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> on behalf of Govindarajan 
Mohandoss via lists.fd.io 
mailto:Govindarajan.mohandoss=arm@lists.fd.io>>
Date: Wednesday, 24 February 2021 at 15:59
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: nd mailto:n...@arm.com>>, nd 
mailto:n...@arm.com>>
Subject: [vpp-dev] IPSec ESP Tunnel mode config

Dear Maintainers,

   I need help to fix ESP Tunnel mode configuration using debug CLI.

   Following command is throwing parse error. Can you please share the latest 
CLI command ?



vpp# ipsec sa add 20 spi 1000 esp tunnel-src 192.83.1.1 tunnel-dst 192.83.1.2 
crypto-alg aes-gcm-128 crypto-key 4a506a794f574265564551694d653768 salt 
0x12345678

ipsec sa: parse error: '-src 192.83.1.1 tunnel-dst 192...'



Thanks

Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18799): https://lists.fd.io/g/vpp-dev/message/18799
Mute This Topic: https://lists.fd.io/mt/80878044/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] IPSec proposal to improve "ipsec4-output-feature" node performance

[Govindarajan Mohandoss]

Hi Neale,
   I am working on optimizing "ipsec4-output-feature" node on ARM based 
systems. Towards that, I saw an opportunity to supplement SPD table lookup 
(linear search) with Bihash based flow cache.
   This approach is similar to ACL plugin stateful mode implementation. This 
approach will consume extra memory for Bihash and provide O(1) performance for 
SPD rules added at different indices.
   I did a very basic prototype and got good results. Please find the prototype 
patch attached.
   Before I start the actual implementation, I would like to get your feedback. 
It will be great if you can give your comments.
  
Following is the idea at high level. Flow cache will be augmented with 
existing linear search based SPD table lookup.

Enhanced SPD Table lookup logic:
-
One every packet arrival, following lookup will be done in 
"ipsec4-output-feature" node:
   1. found = Lookup <5 tuple: Bihash based flow cache> 
   2. if (!found) {
found = Lookup <5 tuple: Linear search>
  if (found) { 
Add an entry into <5 tuple: Bihash based flow cache> 
  }
   }

Linear search will happen only for 1st packet in a flow and from 2nd packet 
onwards, match will succeed in bihash table.
I did a basic prototype and got O(1) performance as expected, when IPv4 5 
tuple rule is added at different indices <1, 10, 100, 1000> in SPD table.

Following are the per core performance numbers with IPSec NULL encryption 
configuration in ESP Tunnel mode, in ARM CPU based system @MRR with 64B packets:

Baseline based on existing linear search

SPD index Performance
---
1st match 5.2  MPPS
10th match  4.51 MPPS
100th match2.05 MPPS
1000th match  266  KPPS

   With Bihash based flow cache (Basic prototype results):
   ==
   SPD index Performance
   ---
   1st match4.88 MPPS
   10th match 4.88 MPPS
   100th match   4.88 MPPS
   1000th match 4.88 MPPS

   As you can see, we are getting constant performance numbers even when rules 
are added at different indices.
   If you are fine with this approach, I would like to proceed with actual 
implementation. 

   I am making an assumption that SPD table will not be populated frequently by 
the application. Please correct me if I am wrong.
   Whenever application add/delete/modify an entry in SPD table, flow cache 
will be purged in the data plane through an interface level flag. I will work 
on this case and send another update.
   
Thanks
Govind  



  



spd_with_flow_cache_prototype.diff
Description: spd_with_flow_cache_prototype.diff

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18839): https://lists.fd.io/g/vpp-dev/message/18839
Mute This Topic: https://lists.fd.io/mt/81046304/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] IPSec proposal to improve "ipsec4-output-feature" node performance

[Govindarajan Mohandoss]

Hi Neale,
  Thank you for your comments. I know you would have thought about it already. 
I can work with you to implement the right solution to improve performance.
  Please see my response inline.

Thanks
Govind

From: Neale Ranns 
Sent: Wednesday, March 3, 2021 8:45 AM
To: Govindarajan Mohandoss ; vpp-dev 

Cc: nd 
Subject: Re: [vpp-dev] IPSec proposal to improve "ipsec4-output-feature" node 
performance


Hi Govind,

Flow caches always perform well, but they are more difficult to use than they 
first appear. Consider asking yourself these questions:
1 - how many entries can the cache contain?
>> This can be made configurable as per the system need. By default, we can 
>> allocate the hash table size to hold 10K entries.

2 - what do you do when the cache is full? How do you age or recycle old flows?
>> If the flow cache is implemented using a hash table without collision 
>> handling, then age out mechanism is not needed. Whenever a collision occurs,
old entry can be overwritten with new entry. Worst case will be 255 overwrites, 
if all the 256 packets per batch result in same hash value.

3 - how do you flush the cache when the policy set changes?
>> Whenever an SPD rule is deleted, the flow cache will be flushed completely 
>> in the control plane. An IPSec module level flag will be introduced and set 
>> by the
control plane to put the data plane in fall back mode to use linear search. 
This flag will be reset once the control plane flush the flow cache and delete 
the
SPD rule from SPD table. Also, data plane will not add new entry into the flow 
cache during SPD rule deletion.
I have added this logic in my prototype. Please find the changes attached.

In general, what is the rate at which an SPD rule will be deleted by the 
application ? If the deletion rate is low, then we can take the hit of flushing 
the flow cache in control plane.

I had considered in the past changing an SPD definition to use IP subnets 
(rather than IP ranges) and then re-use the tuple-sort/merge algorithm used by 
ACLs. This approach would not need you to answer the awkward questions about a 
cache and it would break the linear dependent lookup (it has other 
dependencies, but they are much better). Two reasons I didn't do this 1) no 
time 2) ipsec is a vnet component and ACL is a plugin, a vnet -> plugin 
dependency is a no-no. If you're lucky some-one might volunteer to make IPsec a 
plugin and this will go away...
>> Please correct my understanding.
In this method, the mask have to be created for every SPD rule and stored in an 
array. On every packet arrival, the mask will be picked up in linear fashion 
and hash will
be computed based on mask and packet header fields. Then bihash will be looked 
up with that hash value. This reduces the overhead of comparing the ranges 
during
linear search. But the mask lookup is still linear. I agree that there will be 
a performance improvement because the range comparison is avoided for every SPD 
entry.
Is there a way to implement it without creating IPSec plugin and without 
depending on ACL plugin ?

/neale


From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> on behalf of Govindarajan 
Mohandoss via lists.fd.io 
mailto:Govindarajan.mohandoss=arm@lists.fd.io>>
Date: Wednesday, 3 March 2021 at 06:57
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: nd mailto:n...@arm.com>>
Subject: [vpp-dev] IPSec proposal to improve "ipsec4-output-feature" node 
performance
Hi Neale,
   I am working on optimizing "ipsec4-output-feature" node on ARM based 
systems. Towards that, I saw an opportunity to supplement SPD table lookup 
(linear search) with Bihash based flow cache.
   This approach is similar to ACL plugin stateful mode implementation. This 
approach will consume extra memory for Bihash and provide O(1) performance for 
SPD rules added at different indices.
   I did a very basic prototype and got good results. Please find the prototype 
patch attached.
   Before I start the actual implementation, I would like to get your feedback. 
It will be great if you can give your comments.

Following is the idea at high level. Flow cache will be augmented with 
existing linear search based SPD table lookup.

Enhanced SPD Table lookup logic:
-
One every packet arrival, following lookup will be done in 
"ipsec4-output-feature" node:
   1. found = Lookup <5 tuple: Bihash based flow cache>
   2. if (!found) {
 found = Lookup <5 tuple: Linear search>
  if (found) {
Add an entry into <5 tuple: Bihash based flow cache>
  }
   }

Linear search will happen only for 1st packet in a flow and from 2nd packet 
onwards, match will succeed in bihash table.
I did a basic prototype and got O(1) performance as expected,

Re: [vpp-dev] High packet drop under high binary-api call rate #binapi #vpp #vpp-dev #vapi

[Govindarajan Mohandoss]

Would there be any suggestions that can achieve a lower packet drop rate under 
a high binary-API call rate?

Implementing ready copy update scheme in VPP can address this issue.
https://doc.dpdk.org/guides/prog_guide/rcu_lib.html

Thanks
Govind

From: vpp-dev@lists.fd.io  On Behalf Of benleungbuild via 
lists.fd.io
Sent: Wednesday, July 14, 2021 1:14 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] High packet drop under high binary-api call rate #binapi 
#vpp #vpp-dev #vapi


Hi vpp developers,




**Backgound:**



base on my understanding, the handling of binary API (not-thread-safe) requires 
to stop all other worker threads.

I did a test to study the impact of binary-API call to the packet-drop rate.



topology:

- (dpdk traffic in) > VPP1  (memif) > VPP2  -- (dpdk 
traffic out) ->



The test was run for around 10 minutes, under 8.5 Gbps traffic between 2 vpps 
and around 100 request/second binary API call (classify-add-del) to vpp2 (by 
ligato-vpp-agent), overall around 15-20% of packets are dropped in the path 
from vpp1 to vpp2.

If there is no binary API call, no packet is dropped.




**Questions:**

1. Would there be any suggestions that can achieve a lower packet drop rate 
under a high binary-API call rate?

2. Are there any plan in future vpp release that can improve the "locking" of 
worker thread for non-thread-safe binary API call?



Best

Ben


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19766): https://lists.fd.io/g/vpp-dev/message/19766
Mute This Topic: https://lists.fd.io/mt/84196118/21656
Mute #vpp:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp
Mute #binapi:https://lists.fd.io/g/vpp-dev/mutehashtag/binapi
Mute #vapi:https://lists.fd.io/g/vpp-dev/mutehashtag/vapi
Mute #vpp-dev:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-dev
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Mellanox NIC and Intel QAT support

[Govindarajan Mohandoss]

Hi Fan Zhang,
   With DPDK plugin, VPP does the DMA page map in IOMMU, only when DPDK 
supported ethernet devices are present. As a result, Mellanox NIC and QAT combo 
doesn't work. 
   This issue is fixed by adding DPDK supported crypto device check do the DMA 
page map.
   https://gerrit.fd.io/r/c/vpp/+/35634

  Please provide your comments.

Thanks
Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21012): https://lists.fd.io/g/vpp-dev/message/21012
Mute This Topic: https://lists.fd.io/mt/89779686/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Mellanox NIC and Intel QAT support

[Govindarajan Mohandoss]

Thanks Damjan for merging it.

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan
> Mohandoss via lists.fd.io
> Sent: Monday, March 14, 2022 12:54 PM
> To: vpp-dev 
> Cc: nd ; nd 
> Subject: [vpp-dev] Mellanox NIC and Intel QAT support
> 
> Hi Fan Zhang,
>With DPDK plugin, VPP does the DMA page map in IOMMU, only when
> DPDK supported ethernet devices are present. As a result, Mellanox NIC and
> QAT combo doesn't work.
>This issue is fixed by adding DPDK supported crypto device check do the
> DMA page map.
>https://gerrit.fd.io/r/c/vpp/+/35634
> 
>   Please provide your comments.
> 
> Thanks
> Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21014): https://lists.fd.io/g/vpp-dev/message/21014
Mute This Topic: https://lists.fd.io/mt/89779686/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] IPSec/QAT offload config

[Govindarajan Mohandoss]

Thanks Fan.

Get Outlook for iOS<https://aka.ms/o0ukef>

From: Zhang, Roy Fan 
Sent: Wednesday, February 16, 2022 4:02:10 AM
To: Govindarajan Mohandoss ; vpp-dev 

Cc: Yoan Picchi ; nd ; nd 
Subject: RE: [vpp-dev] IPSec/QAT offload config


Yeah saw it thank you very much!



From: Govindarajan Mohandoss 
Sent: Wednesday, February 16, 2022 1:23 AM
To: Zhang, Roy Fan ; vpp-dev 
Cc: Yoan Picchi ; nd ; nd 
Subject: RE: [vpp-dev] IPSec/QAT offload config



Hi Fan,

I have submitted the patch in gerrit. Please review it.

https://gerrit.fd.io/r/c/vpp/+/35338



Thanks

Govind



From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Fan Zhang via 
lists.fd.io
Sent: Wednesday, February 9, 2022 6:09 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config



Hi Govind,



Sorry for the late reply.

We managed to verify your fix proposal and it is valid.

Could you send a patch to gerrit and add me as reviewer?



Regards,

Fan



From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Monday, January 24, 2022 1:38 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config



Hi Experts,

It worked after making the following changes. I enabled “crypto_native_plugin” 
& qat in startup.  If this is a real issue and if my Crypto/QAT config is 
correct, I can submit this fix for review. Please let me know.



diff --git a/src/vnet/crypto/crypto.c b/src/vnet/crypto/crypto.c

index 9f437cfcd..ed9a9d1be 100644

--- a/src/vnet/crypto/crypto.c

+++ b/src/vnet/crypto/crypto.c

@@ -333,6 +333,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)

   for (i = 0; i < VNET_CRYPTO_ASYNC_OP_N_IDS; i++)

 {

   otd = cm->async_opt_data + i;

+  if (otd->active_engine_index_async == ~0)

+   continue;

   e = cm->engines + otd->active_engine_index_async;

   if (!e->dequeue_handler)

continue;

@@ -345,6 +347,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)

 {

   if (ei[0] == last_ei)

continue;

+  if (ei[0] == ~0)

+   continue;



   e = cm->engines + ei[0];

   vec_add1 (cm->dequeue_handlers, e->dequeue_handler);





Startup conf:

-

plugins

{

…

plugin crypto_native_plugin.so

  {

enable

  }

}



cpu

{

 corelist-workers 9  /* Single worker */

  main-core 0

}



dpdk

{

  dev :af:00.0 #NIC

  dev :39:01.0 #QAT

  log-level debug

  dev default

  {

num-rx-desc 256

num-tx-desc 256

  }



Thanks

Govind



From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Friday, January 21, 2022 1:19 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config



Once more than 1 worker core is added in startup conf, crash is not observed 
during init phase.  But when I set the async mode on, it crashes in same place. 
Do I need to set async mode on to use QAT ? Do I need to enable any specific 
plugin ?



cpu

{

 corelist-workers 9-12

  main-core 13

}



DBGvpp# set ipsec async mode on



Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

337   if (!e->dequeue_handler)

(gdb)





> -Original Message-

> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan

> Mohandoss via lists.fd.io

> Sent: Friday, January 21, 2022 12:24 PM

> To: vpp-dev mailto:vpp-dev@lists.fd.io>>

> Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
> mailto:n...@arm.com>>; nd

> mailto:n...@arm.com>>

> Subject: [vpp-dev] IPSec/QAT offload config

>

> Hi Experts,

>   We are trying to run IPSec with QAT offload and did the following dpdk

> config in startup conf. When we run VPP, it crashes in the init phase (Before

> reaching out to VPP shell). Can you please help us with proper config to

> enable QAT ?

> We did a sanity test with standalone DPDK IPSec appl

Re: [vpp-dev] Building VPP + External DPDK

[Govindarajan Mohandoss]

Hi Experts,
  I would like to access DPDK symbols while running VPP in gdb. I made the 
following change and it didn't help. How can I make the DPDK symbols available ?
 

diff --git a/build/external/packages/dpdk.mk b/build/external/packages/dpdk.mk
index 720682618..cf62c3f5b 100644
--- a/build/external/packages/dpdk.mk
+++ b/build/external/packages/dpdk.mk
@@ -13,7 +13,7 @@

 DPDK_PKTMBUF_HEADROOM?= 128
 DPDK_USE_LIBBSD  ?= n
-DPDK_DEBUG   ?= n
+DPDK_DEBUG   ?= y
 DPDK_MLX4_PMD?= n
 DPDK_MLX5_PMD?= n
 DPDK_MLX5_COMMON_PMD ?= n

Thanks
Govind

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan
> Mohandoss via lists.fd.io
> Sent: Sunday, February 27, 2022 7:31 PM
> To: vpp-dev 
> Cc: nd ; Yoan Picchi ; nd
> 
> Subject: [vpp-dev] Building VPP + External DPDK
> 
> Hi all,
>   Can you please share the steps to build VPP with external DPDK ?
> 
> Thanks
> Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20931): https://lists.fd.io/g/vpp-dev/message/20931
Mute This Topic: https://lists.fd.io/mt/89441756/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] IPSec/QAT offload config

[Govindarajan Mohandoss]

Hi Fan,
I have submitted the patch in gerrit. Please review it.
https://gerrit.fd.io/r/c/vpp/+/35338

Thanks
Govind

From: vpp-dev@lists.fd.io  On Behalf Of Fan Zhang via 
lists.fd.io
Sent: Wednesday, February 9, 2022 6:09 AM
To: Govindarajan Mohandoss ; vpp-dev 

Cc: Yoan Picchi ; nd ; nd 
Subject: Re: [vpp-dev] IPSec/QAT offload config

Hi Govind,

Sorry for the late reply.
We managed to verify your fix proposal and it is valid.
Could you send a patch to gerrit and add me as reviewer?

Regards,
Fan

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Monday, January 24, 2022 1:38 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config

Hi Experts,
It worked after making the following changes. I enabled "crypto_native_plugin" 
& qat in startup.  If this is a real issue and if my Crypto/QAT config is 
correct, I can submit this fix for review. Please let me know.

diff --git a/src/vnet/crypto/crypto.c b/src/vnet/crypto/crypto.c
index 9f437cfcd..ed9a9d1be 100644
--- a/src/vnet/crypto/crypto.c
+++ b/src/vnet/crypto/crypto.c
@@ -333,6 +333,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
   for (i = 0; i < VNET_CRYPTO_ASYNC_OP_N_IDS; i++)
 {
   otd = cm->async_opt_data + i;
+  if (otd->active_engine_index_async == ~0)
+   continue;
   e = cm->engines + otd->active_engine_index_async;
   if (!e->dequeue_handler)
continue;
@@ -345,6 +347,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
 {
   if (ei[0] == last_ei)
continue;
+  if (ei[0] == ~0)
+   continue;

   e = cm->engines + ei[0];
   vec_add1 (cm->dequeue_handlers, e->dequeue_handler);


Startup conf:
-
plugins
{
...
plugin crypto_native_plugin.so
  {
enable
  }
}

cpu
{
 corelist-workers 9  /* Single worker */
  main-core 0
}

dpdk
{
  dev :af:00.0 #NIC
  dev :39:01.0 #QAT
  log-level debug
  dev default
  {
num-rx-desc 256
num-tx-desc 256
  }

Thanks
Govind

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Friday, January 21, 2022 1:19 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config


Once more than 1 worker core is added in startup conf, crash is not observed 
during init phase.  But when I set the async mode on, it crashes in same place. 
Do I need to set async mode on to use QAT ? Do I need to enable any specific 
plugin ?



cpu

{

 corelist-workers 9-12

  main-core 13

}



DBGvpp# set ipsec async mode on



Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

337   if (!e->dequeue_handler)

(gdb)





> -Original Message-

> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan

> Mohandoss via lists.fd.io

> Sent: Friday, January 21, 2022 12:24 PM

> To: vpp-dev mailto:vpp-dev@lists.fd.io>>

> Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
> mailto:n...@arm.com>>; nd

> mailto:n...@arm.com>>

> Subject: [vpp-dev] IPSec/QAT offload config

>

> Hi Experts,

>   We are trying to run IPSec with QAT offload and did the following dpdk

> config in startup conf. When we run VPP, it crashes in the init phase (Before

> reaching out to VPP shell). Can you please help us with proper config to

> enable QAT ?

> We did a sanity test with standalone DPDK IPSec application and it works fine

> with QAT card.

>

> dpdk

> {

>   dev :af:00.0 #NIC

>   dev :39:01.0 #QAT

>   log-level debug

>   dev default

>   {

> num-rx-desc 1024

> num-tx-desc 1024

>   }

> }

>

>

> Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

> 0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> 337   if (!e->dequeue_handler)

> (gdb) bt

> #0  0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> #1  0x77371d69 in vnet_crypto_request_async_mode (is_enable=1)

> at /home/govmoh01/vpp_qat/vpp/src

Re: [vpp-dev] Building VPP + External DPDK

[Govindarajan Mohandoss]

Thanks Ben !! After the purge it worked.

> -Original Message-
> From: Benoit Ganne (bganne) 
> Sent: Tuesday, March 1, 2022 2:43 AM
> To: Govindarajan Mohandoss ; vpp-
> dev 
> Cc: nd ; Yoan Picchi ; nd
> 
> Subject: RE: [vpp-dev] Building VPP + External DPDK
> 
> Hi Govind,
> 
> That should work, but if you have installed dpdk via 'make install-ext-dep'
> you'll have to remove the 'vpp-ext-deps' package (eg. 'apt purge vpp-ext-
> deps') otherwise it will not be rebuilt.
> 
> Best
> Ben
> 
> > -Original Message-
> > From: vpp-dev@lists.fd.io  On Behalf Of
> > Govindarajan Mohandoss
> > Sent: lundi 28 février 2022 20:12
> > To: Govindarajan Mohandoss ; vpp-
> dev
> > 
> > Cc: nd ; Yoan Picchi ; nd
> > 
> > Subject: Re: [vpp-dev] Building VPP + External DPDK
> >
> > Hi Experts,
> >   I would like to access DPDK symbols while running VPP in gdb. I made
> > the following change and it didn't help. How can I make the DPDK
> > symbols available ?
> >
> >
> > diff --git a/build/external/packages/dpdk.mk
> > b/build/external/packages/dpdk.mk index 720682618..cf62c3f5b 100644
> > --- a/build/external/packages/dpdk.mk
> > +++ b/build/external/packages/dpdk.mk
> > @@ -13,7 +13,7 @@
> >
> >  DPDK_PKTMBUF_HEADROOM?= 128
> >  DPDK_USE_LIBBSD  ?= n
> > -DPDK_DEBUG   ?= n
> > +DPDK_DEBUG   ?= y
> >  DPDK_MLX4_PMD    ?= n
> >  DPDK_MLX5_PMD?= n
> >  DPDK_MLX5_COMMON_PMD ?= n
> >
> > Thanks
> > Govind
> >
> > > -Original Message-
> > > From: vpp-dev@lists.fd.io  On Behalf Of
> > Govindarajan
> > > Mohandoss via lists.fd.io
> > > Sent: Sunday, February 27, 2022 7:31 PM
> > > To: vpp-dev 
> > > Cc: nd ; Yoan Picchi ; nd
> > > 
> > > Subject: [vpp-dev] Building VPP + External DPDK
> > >
> > > Hi all,
> > >   Can you please share the steps to build VPP with external DPDK ?
> > >
> > > Thanks
> > > Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20936): https://lists.fd.io/g/vpp-dev/message/20936
Mute This Topic: https://lists.fd.io/mt/89441756/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Building VPP + External DPDK

[Govindarajan Mohandoss]

Hi all,
  Can you please share the steps to build VPP with external DPDK ?

Thanks
Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20927): https://lists.fd.io/g/vpp-dev/message/20927
Mute This Topic: https://lists.fd.io/mt/89441756/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] IPSec/QAT offload config

[Govindarajan Mohandoss]

Sure Fan. I will submit the patch. Thanks.

From: vpp-dev@lists.fd.io  On Behalf Of Fan Zhang via 
lists.fd.io
Sent: Wednesday, February 9, 2022 6:09 AM
To: Govindarajan Mohandoss ; vpp-dev 

Cc: Yoan Picchi ; nd ; nd 
Subject: Re: [vpp-dev] IPSec/QAT offload config

Hi Govind,

Sorry for the late reply.
We managed to verify your fix proposal and it is valid.
Could you send a patch to gerrit and add me as reviewer?

Regards,
Fan

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Monday, January 24, 2022 1:38 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config

Hi Experts,
It worked after making the following changes. I enabled "crypto_native_plugin" 
& qat in startup.  If this is a real issue and if my Crypto/QAT config is 
correct, I can submit this fix for review. Please let me know.

diff --git a/src/vnet/crypto/crypto.c b/src/vnet/crypto/crypto.c
index 9f437cfcd..ed9a9d1be 100644
--- a/src/vnet/crypto/crypto.c
+++ b/src/vnet/crypto/crypto.c
@@ -333,6 +333,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
   for (i = 0; i < VNET_CRYPTO_ASYNC_OP_N_IDS; i++)
 {
   otd = cm->async_opt_data + i;
+  if (otd->active_engine_index_async == ~0)
+   continue;
   e = cm->engines + otd->active_engine_index_async;
   if (!e->dequeue_handler)
continue;
@@ -345,6 +347,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
 {
   if (ei[0] == last_ei)
continue;
+  if (ei[0] == ~0)
+   continue;

   e = cm->engines + ei[0];
   vec_add1 (cm->dequeue_handlers, e->dequeue_handler);


Startup conf:
-
plugins
{
...
plugin crypto_native_plugin.so
  {
enable
  }
}

cpu
{
 corelist-workers 9  /* Single worker */
  main-core 0
}

dpdk
{
  dev :af:00.0 #NIC
  dev :39:01.0 #QAT
  log-level debug
  dev default
  {
num-rx-desc 256
num-tx-desc 256
  }

Thanks
Govind

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Friday, January 21, 2022 1:19 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config


Once more than 1 worker core is added in startup conf, crash is not observed 
during init phase.  But when I set the async mode on, it crashes in same place. 
Do I need to set async mode on to use QAT ? Do I need to enable any specific 
plugin ?



cpu

{

 corelist-workers 9-12

  main-core 13

}



DBGvpp# set ipsec async mode on



Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

337   if (!e->dequeue_handler)

(gdb)





> -Original Message-

> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan

> Mohandoss via lists.fd.io

> Sent: Friday, January 21, 2022 12:24 PM

> To: vpp-dev mailto:vpp-dev@lists.fd.io>>

> Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
> mailto:n...@arm.com>>; nd

> mailto:n...@arm.com>>

> Subject: [vpp-dev] IPSec/QAT offload config

>

> Hi Experts,

>   We are trying to run IPSec with QAT offload and did the following dpdk

> config in startup conf. When we run VPP, it crashes in the init phase (Before

> reaching out to VPP shell). Can you please help us with proper config to

> enable QAT ?

> We did a sanity test with standalone DPDK IPSec application and it works fine

> with QAT card.

>

> dpdk

> {

>   dev :af:00.0 #NIC

>   dev :39:01.0 #QAT

>   log-level debug

>   dev default

>   {

> num-rx-desc 1024

> num-tx-desc 1024

>   }

> }

>

>

> Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

> 0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> 337   if (!e->dequeue_handler)

> (gdb) bt

> #0  0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> #1  0x77371d69 in vnet_crypto_request_async_mode (is_enable=1)

> at /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:678

> #2  0x7ffef5b0e4ff in dpdk

Re: [vpp-dev] IPSec/QAT offload config

[Govindarajan Mohandoss]

Once more than 1 worker core is added in startup conf, crash is not observed 
during init phase.  But when I set the async mode on, it crashes in same place. 
Do I need to set async mode on to use QAT ? Do I need to enable any specific 
plugin ?



cpu

{

 corelist-workers 9-12

  main-core 13

}



DBGvpp# set ipsec async mode on



Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

337   if (!e->dequeue_handler)

(gdb)





> -Original Message-

> From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan

> Mohandoss via lists.fd.io

> Sent: Friday, January 21, 2022 12:24 PM

> To: vpp-dev 

> Cc: Yoan Picchi ; nd ; nd

> 

> Subject: [vpp-dev] IPSec/QAT offload config

>

> Hi Experts,

>   We are trying to run IPSec with QAT offload and did the following dpdk

> config in startup conf. When we run VPP, it crashes in the init phase (Before

> reaching out to VPP shell). Can you please help us with proper config to

> enable QAT ?

> We did a sanity test with standalone DPDK IPSec application and it works fine

> with QAT card.

>

> dpdk

> {

>   dev :af:00.0 #NIC

>   dev :39:01.0 #QAT

>   log-level debug

>   dev default

>   {

> num-rx-desc 1024

> num-tx-desc 1024

>   }

> }

>

>

> Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

> 0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> 337   if (!e->dequeue_handler)

> (gdb) bt

> #0  0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> #1  0x77371d69 in vnet_crypto_request_async_mode (is_enable=1)

> at /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:678

> #2  0x7ffef5b0e4ff in dpdk_cryptodev_init (vm=0x7ffef685a680)

> at

> /home/govmoh01/vpp_qat/vpp/src/plugins/dpdk/cryptodev/cryptodev.c:12

> 00

> #3  0x7ffef5af1608 in dpdk_process (vm=0x7ffef685a680,

> rt=0x7ffef8176d00, f=0x0)

> at /home/govmoh01/vpp_qat/vpp/src/plugins/dpdk/device/init.c:1417

> #4  0x76e513ed in vlib_process_bootstrap (_a=140733006596280) at

> /home/govmoh01/vpp_qat/vpp/src/vlib/main.c:1235

> #5  0x76cefc38 in clib_calljmp () at

> /home/govmoh01/vpp_qat/vpp/src/vppinfra/longjmp.S:123

> #6  0x7ffef4ddc8b0 in ?? ()

> #7  0x76e50e0f in vlib_process_startup (vm=0x7ffef685a680,

> p=0x7ffef8176d00, f=0x0)

>

> Thanks

> Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20777): https://lists.fd.io/g/vpp-dev/message/20777
Mute This Topic: https://lists.fd.io/mt/88589344/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] IPSec/QAT offload config

[Govindarajan Mohandoss]

Hi Experts,
  We are trying to run IPSec with QAT offload and did the following dpdk config 
in startup conf. When we run VPP, it crashes in the init phase (Before reaching 
out to VPP shell). Can you please help us with proper config to enable QAT ?
We did a sanity test with standalone DPDK IPSec application and it works fine 
with QAT card.

dpdk
{
  dev :af:00.0 #NIC
  dev :39:01.0 #QAT
  log-level debug
  dev default
  {
num-rx-desc 1024
num-tx-desc 1024
  }
}


Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.
0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337
337   if (!e->dequeue_handler)
(gdb) bt
#0  0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337
#1  0x77371d69 in vnet_crypto_request_async_mode (is_enable=1)
at /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:678
#2  0x7ffef5b0e4ff in dpdk_cryptodev_init (vm=0x7ffef685a680)
at /home/govmoh01/vpp_qat/vpp/src/plugins/dpdk/cryptodev/cryptodev.c:1200
#3  0x7ffef5af1608 in dpdk_process (vm=0x7ffef685a680, rt=0x7ffef8176d00, 
f=0x0)
at /home/govmoh01/vpp_qat/vpp/src/plugins/dpdk/device/init.c:1417
#4  0x76e513ed in vlib_process_bootstrap (_a=140733006596280) at 
/home/govmoh01/vpp_qat/vpp/src/vlib/main.c:1235
#5  0x76cefc38 in clib_calljmp () at 
/home/govmoh01/vpp_qat/vpp/src/vppinfra/longjmp.S:123
#6  0x7ffef4ddc8b0 in ?? ()
#7  0x76e50e0f in vlib_process_startup (vm=0x7ffef685a680, 
p=0x7ffef8176d00, f=0x0)

Thanks
Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20776): https://lists.fd.io/g/vpp-dev/message/20776
Mute This Topic: https://lists.fd.io/mt/88589344/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] IPSec/QAT offload config

[Govindarajan Mohandoss]

Hi Experts,
It worked after making the following changes. I enabled "crypto_native_plugin" 
& qat in startup.  If this is a real issue and if my Crypto/QAT config is 
correct, I can submit this fix for review. Please let me know.

diff --git a/src/vnet/crypto/crypto.c b/src/vnet/crypto/crypto.c
index 9f437cfcd..ed9a9d1be 100644
--- a/src/vnet/crypto/crypto.c
+++ b/src/vnet/crypto/crypto.c
@@ -333,6 +333,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
   for (i = 0; i < VNET_CRYPTO_ASYNC_OP_N_IDS; i++)
 {
   otd = cm->async_opt_data + i;
+  if (otd->active_engine_index_async == ~0)
+   continue;
   e = cm->engines + otd->active_engine_index_async;
   if (!e->dequeue_handler)
continue;
@@ -345,6 +347,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
 {
   if (ei[0] == last_ei)
continue;
+  if (ei[0] == ~0)
+   continue;

   e = cm->engines + ei[0];
   vec_add1 (cm->dequeue_handlers, e->dequeue_handler);


Startup conf:
-
plugins
{
...
plugin crypto_native_plugin.so
  {
enable
  }
}

cpu
{
 corelist-workers 9  /* Single worker */
  main-core 0
}

dpdk
{
  dev :af:00.0 #NIC
  dev :39:01.0 #QAT
  log-level debug
  dev default
  {
num-rx-desc 256
num-tx-desc 256
  }

Thanks
Govind

From: vpp-dev@lists.fd.io  On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Friday, January 21, 2022 1:19 PM
To: Govindarajan Mohandoss ; vpp-dev 

Cc: Yoan Picchi ; nd ; nd 
Subject: Re: [vpp-dev] IPSec/QAT offload config


Once more than 1 worker core is added in startup conf, crash is not observed 
during init phase.  But when I set the async mode on, it crashes in same place. 
Do I need to set async mode on to use QAT ? Do I need to enable any specific 
plugin ?



cpu

{

 corelist-workers 9-12

  main-core 13

}



DBGvpp# set ipsec async mode on



Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

337   if (!e->dequeue_handler)

(gdb)





> -Original Message-

> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan

> Mohandoss via lists.fd.io

> Sent: Friday, January 21, 2022 12:24 PM

> To: vpp-dev mailto:vpp-dev@lists.fd.io>>

> Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
> mailto:n...@arm.com>>; nd

> mailto:n...@arm.com>>

> Subject: [vpp-dev] IPSec/QAT offload config

>

> Hi Experts,

>   We are trying to run IPSec with QAT offload and did the following dpdk

> config in startup conf. When we run VPP, it crashes in the init phase (Before

> reaching out to VPP shell). Can you please help us with proper config to

> enable QAT ?

> We did a sanity test with standalone DPDK IPSec application and it works fine

> with QAT card.

>

> dpdk

> {

>   dev :af:00.0 #NIC

>   dev :39:01.0 #QAT

>   log-level debug

>   dev default

>   {

> num-rx-desc 1024

> num-tx-desc 1024

>   }

> }

>

>

> Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

> 0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> 337   if (!e->dequeue_handler)

> (gdb) bt

> #0  0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> #1  0x77371d69 in vnet_crypto_request_async_mode (is_enable=1)

> at /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:678

> #2  0x7ffef5b0e4ff in dpdk_cryptodev_init (vm=0x7ffef685a680)

> at

> /home/govmoh01/vpp_qat/vpp/src/plugins/dpdk/cryptodev/cryptodev.c:12

> 00

> #3  0x7ffef5af1608 in dpdk_process (vm=0x7ffef685a680,

> rt=0x7ffef8176d00, f=0x0)

> at /home/govmoh01/vpp_qat/vpp/src/plugins/dpdk/device/init.c:1417

> #4  0x76e513ed in vlib_process_bootstrap (_a=140733006596280) at

> /home/govmoh01/vpp_qat/vpp/src/vlib/main.c:1235

> #5  0x76cefc38 in clib_calljmp () at

> /home/govmoh01/vpp_qat/vpp/src/vppinfra/longjmp.S:123

> #6  0x7ffef4ddc8b0 in ?? ()

> #7  0x76e50e0f in vlib_process_startup (vm=0x7ffef685a680,

> p=0x7ffef8176d00, f=0x0)

>

> Thanks

> Govind

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20783): https://lists.fd.io/g/vpp-dev/message/20783
Mute This Topic: https://lists.fd.io/mt/88589344/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] IPSec/QAT offload config

[Govindarajan Mohandoss]

Hi Fan,
  Thanks for your response.
  We are observing the crash in master.
  Can you please share your startup and IPSec CLI conf ?

Thanks
Govind

From: Zhang, Roy Fan 
Sent: Wednesday, January 26, 2022 4:22 AM
To: Govindarajan Mohandoss ; vpp-dev 

Cc: Yoan Picchi ; nd ; nd 
Subject: RE: [vpp-dev] IPSec/QAT offload config

Hi Govind,

I suppose you were not using Master?
The problem is addressed recently. You shall see no failure if running on 
master. If there is still problem please let me know.

Regards,
Fan

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Monday, January 24, 2022 1:38 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config

Hi Experts,
It worked after making the following changes. I enabled "crypto_native_plugin" 
& qat in startup.  If this is a real issue and if my Crypto/QAT config is 
correct, I can submit this fix for review. Please let me know.

diff --git a/src/vnet/crypto/crypto.c b/src/vnet/crypto/crypto.c
index 9f437cfcd..ed9a9d1be 100644
--- a/src/vnet/crypto/crypto.c
+++ b/src/vnet/crypto/crypto.c
@@ -333,6 +333,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
   for (i = 0; i < VNET_CRYPTO_ASYNC_OP_N_IDS; i++)
 {
   otd = cm->async_opt_data + i;
+  if (otd->active_engine_index_async == ~0)
+   continue;
   e = cm->engines + otd->active_engine_index_async;
   if (!e->dequeue_handler)
continue;
@@ -345,6 +347,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
 {
   if (ei[0] == last_ei)
continue;
+  if (ei[0] == ~0)
+   continue;

   e = cm->engines + ei[0];
   vec_add1 (cm->dequeue_handlers, e->dequeue_handler);


Startup conf:
-
plugins
{
...
plugin crypto_native_plugin.so
  {
enable
  }
}

cpu
{
 corelist-workers 9  /* Single worker */
  main-core 0
}

dpdk
{
  dev :af:00.0 #NIC
  dev :39:01.0 #QAT
  log-level debug
  dev default
  {
num-rx-desc 256
num-tx-desc 256
  }

Thanks
Govind

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Friday, January 21, 2022 1:19 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config


Once more than 1 worker core is added in startup conf, crash is not observed 
during init phase.  But when I set the async mode on, it crashes in same place. 
Do I need to set async mode on to use QAT ? Do I need to enable any specific 
plugin ?



cpu

{

 corelist-workers 9-12

  main-core 13

}



DBGvpp# set ipsec async mode on



Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

337   if (!e->dequeue_handler)

(gdb)





> -Original Message-

> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan

> Mohandoss via lists.fd.io

> Sent: Friday, January 21, 2022 12:24 PM

> To: vpp-dev mailto:vpp-dev@lists.fd.io>>

> Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
> mailto:n...@arm.com>>; nd

> mailto:n...@arm.com>>

> Subject: [vpp-dev] IPSec/QAT offload config

>

> Hi Experts,

>   We are trying to run IPSec with QAT offload and did the following dpdk

> config in startup conf. When we run VPP, it crashes in the init phase (Before

> reaching out to VPP shell). Can you please help us with proper config to

> enable QAT ?

> We did a sanity test with standalone DPDK IPSec application and it works fine

> with QAT card.

>

> dpdk

> {

>   dev :af:00.0 #NIC

>   dev :39:01.0 #QAT

>   log-level debug

>   dev default

>   {

> num-rx-desc 1024

> num-tx-desc 1024

>   }

> }

>

>

> Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

> 0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> 337   if (!e->dequeue_handler)

> (gdb) bt

> #0  0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at

> /home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

> #1  0x77371d69 in vnet_crypto_request_async_mode (is_enable=1)

> at /home/gov

Re: [vpp-dev] IPSec/QAT offload config

[Govindarajan Mohandoss]

Sure Fan. Please find the CLI, Startup conf attached.

From: Zhang, Roy Fan 
Sent: Wednesday, January 26, 2022 8:50 AM
To: Govindarajan Mohandoss ; vpp-dev 

Cc: Yoan Picchi ; nd ; nd ; nd 

Subject: RE: [vpp-dev] IPSec/QAT offload config

Hi Govind,

We only tested the algorithm that is supported - and in our test env all 
plugins are enabled.
Could you share your CLI command instead? Maybe there is something we missed.
We will try to reproduce the problem with your CLI and with only the plugin you 
mentioned.

Regards,
Fan

From: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>
Sent: Wednesday, January 26, 2022 2:45 PM
To: Zhang, Roy Fan mailto:roy.fan.zh...@intel.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>; nd 
mailto:n...@arm.com>>
Subject: RE: [vpp-dev] IPSec/QAT offload config

Hi Fan,
  Thanks for your response.
  We are observing the crash in master.
  Can you please share your startup and IPSec CLI conf ?

Thanks
Govind

From: Zhang, Roy Fan mailto:roy.fan.zh...@intel.com>>
Sent: Wednesday, January 26, 2022 4:22 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: RE: [vpp-dev] IPSec/QAT offload config

Hi Govind,

I suppose you were not using Master?
The problem is addressed recently. You shall see no failure if running on 
master. If there is still problem please let me know.

Regards,
Fan

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss
Sent: Monday, January 24, 2022 1:38 AM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config

Hi Experts,
It worked after making the following changes. I enabled "crypto_native_plugin" 
& qat in startup.  If this is a real issue and if my Crypto/QAT config is 
correct, I can submit this fix for review. Please let me know.

diff --git a/src/vnet/crypto/crypto.c b/src/vnet/crypto/crypto.c
index 9f437cfcd..ed9a9d1be 100644
--- a/src/vnet/crypto/crypto.c
+++ b/src/vnet/crypto/crypto.c
@@ -333,6 +333,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
   for (i = 0; i < VNET_CRYPTO_ASYNC_OP_N_IDS; i++)
 {
   otd = cm->async_opt_data + i;
+  if (otd->active_engine_index_async == ~0)
+   continue;
   e = cm->engines + otd->active_engine_index_async;
   if (!e->dequeue_handler)
continue;
@@ -345,6 +347,8 @@ vnet_crypto_update_cm_dequeue_handlers (void)
 {
   if (ei[0] == last_ei)
continue;
+  if (ei[0] == ~0)
+   continue;

   e = cm->engines + ei[0];
   vec_add1 (cm->dequeue_handlers, e->dequeue_handler);


Startup conf:
-
plugins
{
...
plugin crypto_native_plugin.so
  {
enable
  }
}

cpu
{
 corelist-workers 9  /* Single worker */
  main-core 0
}

dpdk
{
  dev :af:00.0 #NIC
  dev :39:01.0 #QAT
  log-level debug
  dev default
  {
num-rx-desc 256
num-tx-desc 256
  }

Thanks
Govind

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan 
Mohandoss via lists.fd.io
Sent: Friday, January 21, 2022 1:19 PM
To: Govindarajan Mohandoss 
mailto:govindarajan.mohand...@arm.com>>; 
vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
mailto:n...@arm.com>>; nd mailto:n...@arm.com>>
Subject: Re: [vpp-dev] IPSec/QAT offload config


Once more than 1 worker core is added in startup conf, crash is not observed 
during init phase.  But when I set the async mode on, it crashes in same place. 
Do I need to set async mode on to use QAT ? Do I need to enable any specific 
plugin ?



cpu

{

 corelist-workers 9-12

  main-core 13

}



DBGvpp# set ipsec async mode on



Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x773713c9 in vnet_crypto_update_cm_dequeue_handlers () at 
/home/govmoh01/vpp_qat/vpp/src/vnet/crypto/crypto.c:337

337   if (!e->dequeue_handler)

(gdb)





> -Original Message-

> From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
> mailto:vpp-dev@lists.fd.io>> On Behalf Of Govindarajan

> Mohandoss via lists.fd.io

> Sent: Friday, January 21, 2022 12:24 PM

> To: vpp-dev mailto:vpp-dev@lists.fd.io>>

> Cc: Yoan Picchi mailto:yoan.pic...@arm.com>>; nd 
> mailto:n...@arm.com>>; nd

> mailto:n...@arm.com>>

> Subject: [vpp-dev] IPS