Hi Maciek,
Couple of question.
1. Any idea why the bad plugins are bad - is there anything common in
their design that you suspect might be a common thread in the
performance regressions.
2. When I compare the CSIT 1710 to the 1801 test report from DPDK, it
suggests improved performance of x520? , < 24 mpps compared to > 26 mpps
with VPP 1801. Can you point us at the specific data demonstrating the
performance regression in VPP 1801.
https://docs.fd.io/csit/rls1710/report/dpdk_performance_tests/packet_throughput_graphs/l2.html#ndr-throughput
https://docs.fd.io/csit/rls1801/report/dpdk_performance_tests/packet_throughput_graphs/l2.html#ndr-throughput
Thanks,
Ray K
On 19/03/2018 23:14, Maciek Konstantynowicz (mkonstan) wrote:
Update on the four rls1801 performance regressions reported in CSIT rls1801
report[1]:
1. CSIT-925 Plugin induced NDR/PDR regression
a. All plugins loaded (VPP default) shows throughput variability of 3..5%
for all NIC2NIC tests;
b. Initial regression tests isolated a group of 6 "bad" plugins causing
variability:
- "bad" plugins: gtpu, ila, ixge, kubeproxy, l2e, sixrd;
- "good" plugins: dpdk, acl, flowprobe, ioam, lb, memif, nat, pppoe and
stn;
c. Further tests isolated it further to:
- gtpu plugin: worst case >25% performance drop on some testbed, less on
others;
- other "bad" plugins: affected by dpdk1711 + x520 NIC regression, see
CSIT-926;
d. Conclusions and next steps:
- Due to a number of issues with "bad" plugins CSIT is now loading VPP
plugins on the need basis;
- startup.conf used to load needed plugins per test;
- gtpu: requires deeper perf analysis e.g. "perf top";
2. CSIT-926 Generic NDR/PDR regression
a. Generic regression of up to -3% vs. rls1710;
b. Re-analysis of VPP daily performance trending jobs isolated the trouble
period to between two patches;
c. Further bisecting confirm problem is related to introduction of dpdk1711
for x520 NICs;
d. Conclusions and next steps:
- dpdk1711 introduced up to 3% generic performance regression to VPP for
x520 NICs
- affects some packet paths more than others;
- also affects some plugin loads, see "bad" plugins in 1b.
- x710/i40e NICs seem not be affected;
3. CSIT-927 vhost-user vring size of 1024 - NDR regression
a. Much lower NDR for vhostvr1024 tests, up to -42% vs. rls1710;
b. Root caused to bug in CSIT code, side effect of introducing Aarch64 Qemu;
c. CSIT bug fixed, https://gerrit.fd.io/r/#/c/10746/
4. CSIT-928 IPSec scale with HW QAT crypto-dev - NDR/PDR regression
a. NDR/PDR regression up to -15% vs. rls1710;
b. Root caused to bug in VPP code, QAT crypto-dev driver;
c. VPP patch by Radu Nicolau, under review/test verification.
- https://gerrit.fd.io/r/#/c/11082/
Comments?
Cheers,
-Maciek
(CSIT PTL)
[1]
https://docs.fd.io/csit/rls1801/report/vpp_performance_tests/csit_release_notes.html#known-issues
-=-=-=-=-=-=-=-=-=-=-=-
Links:
You receive all messages sent to this group.
View/Reply Online (#8607): https://lists.fd.io/g/vpp-dev/message/8607
View All Messages In Topic (2): https://lists.fd.io/g/vpp-dev/topic/15624779
Mute This Topic: https://lists.fd.io/mt/15624779/21656
New Topic: https://lists.fd.io/g/vpp-dev/post
Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: [email protected]
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
-=-=-=-=-=-=-=-=-=-=-=-
