Hi,
Le 19/04/2020 à 16:07, Wisam Monther a écrit :
Hey Yan,
For FW steering there is HW limitation on number of flows that can be added on
it.
Which is table 0, and I think it's 65536
But to get Millions of rules use --group=1 which is SW steering.
What do you mean by SW steering?
Using group 1 we had better performance but only around 10K. I think the
whole process lacks an update feature instead of delete+create, and the
ability to batch rules.
Moreover make sure you have enough memory in the app to have good insertion
rate.
If you have enough 1G huge pages then it's ok.
If you are working with 2M pages your command should be like this:
sudo ./flow_perf -l 3-7 -n 4 -w 02:00.0,dv_flow_en=1 --socket-mem=4096 --
--ingress --group=1 --ether --ipv4 --udp --queue --flows-count=1000000
BRs,
Wisam Jaddo
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Sunday, April 19, 2020 4:58 PM
To: Yan Lei <[email protected]>
Cc: [email protected]; Wisam Monther <[email protected]>
Subject: Re: [dpdk-users] [mlx5 + DPDK 19.11] Flow insertion rate less than 4K
per sec
+Cc Wisam
16/04/2020 17:32, Yan Lei:
Hi Thomas,
I tried the patch (68057 + 68058) on DPDK 19.11/20.02 + ofed 4.7.3.
TL;DR
1. I was only able to generate 3K rules per second.
2. The maximum number of distinct rules the NIC can support seems to be
65536.
How can I increase the insertion rate? Any firmware/driver config I need to
tune? Also, is 65536 distinct flows truly a limit of the NIC? The patch
defaults to
generate 4 million distinct flows though...
Thanks in advance!
Initially, running
```
sudo ./flow_perf -l 3-7 -n 4 -w 02:00.0,dv_flow_en=1 -- --ingress
--ether --ipv4 --udp --queue --flows-count=1000000
```
failed after a few seconds and it gave
```
Flow can't be created 1 message: hardware refuses to create flow
EAL: Error - exiting with code: 1
Cause: error in creating flow
```
Then I added a small debug patch (attached) and it showed that the error
happens when creating the 65536th flow rule.
The first table is indeed limited to something around that number. But
performance are already degrading before that point. Even with OFED 5
and the firmware that comes with it.
```
Flow can't be created 1 message: hardware refuses to create flow
EAL: Error - exiting with code: 1
Cause: error in creating flow,flows generated: 65536 ```
My guess is that the NIC can only accept 65536 concurrent rules. Once I
changed the outer ip mask to 0xffff, the above command runs fine.
To see how many rules I can generate per second. I ran (with the outer
ip mask 0xffff)
```
sudo ./flow_perf -l 3-7 -n 4 -w 02:00.0,dv_flow_en=1 -- --ingress
--ether --ipv4 --udp --queue --flows-count=65536
```
and it gives
```
:: Total flow insertion rate -> 3.015922 K/Sec
:: The time for creating 65536 in flows 21.730005 seconds
:: EAGAIN counter = 0
```
So 3 rules per sec. Which is close to what I observed before.
```
sudo ./flow_perf -l 3-7 -n 4 -w 02:00.0,dv_flow_en=1 -- --ingress
--ether --ipv4 --udp --queue --flows-count=100000 ``` gives
```
:: Total flow insertion rate -> 0.949381 K/Sec
:: The time for creating 100000 in flows 105.331842 seconds
:: EAGAIN counter = 0
```
Have no idea why it's only 1k/sec in this case...
Thanks and cheers,
Lei
________________________________
From: users <[email protected]> on behalf of Yan Lei
<[email protected]>
Sent: Tuesday, April 14, 2020 1:20 PM
To: Thomas Monjalon
Cc: [email protected]
Subject: Re: [dpdk-users] [mlx5 + DPDK 19.11] Flow insertion rate less
than 4K per sec
Hi Thomas,
Thanks! I will give it a try (using DPDK 19.11 + ofed 4.7.3).
Cheers,
Lei
________________________________
From: Thomas Monjalon <[email protected]>
Sent: Tuesday, April 14, 2020 12:12:28 PM
To: Yan Lei
Cc: [email protected]
Subject: Re: [dpdk-users] [mlx5 + DPDK 19.11] Flow insertion rate less
than 4K per sec
Hi,
10/04/2020 20:11, Yan Lei:
I am doing some study that requires inserting more than 1 million
flow rules per second to the NIC. And I runs DPDK 19.11 on a ConnectX-5
NIC.
But I only managed to create around 3.3K rules per second.
Below is the code I used to measure the insertion rate:
Please could you review this new application designed for such measure?
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatc
hes.dpdk.org%2Fpatch%2F68058%2F&data=02%7C01%7Cwisamm%40m
ellanox.c
om%7Cc046523203de456919da08d7e469add0%7Ca652971c7d2e4d9ba6a4d149
256f46
1b%7C0%7C0%7C637229014854391590&sdata=dxxEmm4DWoMPeNGyM
FaYgk%2BjSE
%2FwVKLnYAwQ7QhjKGc%3D&reserved=0
Any feedback about the above patch is welcome. Feel free to try and review
it.
Tom
