> Added some missing negative test cases, see below. > Thanks for these Eelco, would you like me to add you as a co-author for this patch as you have code that is part of the next revison?
Regards Ian > On 15 Jul 2021, at 14:42, kumar Amber wrote: > > > From: Kumar Amber <[email protected]> > > > > Tests: > > 6: OVS-DPDK - MFEX Autovalidator > > 7: OVS-DPDK - MFEX Autovalidator Fuzzy > > 8: OVS-DPDK - MFEX Configuration > > > > Added a new directory to store the PCAP file used > > in the tests and a script to generate the fuzzy traffic > > type pcap to be used in fuzzy unit test. > > > > Signed-off-by: Kumar Amber <[email protected]> > > Acked-by: Flavio Leitner <[email protected]> > > > > --- > > v13: > > - fix -v in the command > > - added the configuration test case and supporting doc update > > v12: > > - change skip paramter for unit test > > v11: > > - fix comments from Eelco > > v7: > > - fix review comments(Eelco) > > v5: > > - fix review comments(Ian, Flavio, Eelco) > > - remove sleep from first test and added minor 5 sec sleep to fuzzy > > --- > > --- > > Documentation/topics/dpdk/bridge.rst | 56 ++++++++++++ > > tests/.gitignore | 1 + > > tests/automake.mk | 6 ++ > > tests/mfex_fuzzy.py | 33 +++++++ > > tests/pcap/mfex_test.pcap | Bin 0 -> 416 bytes > > tests/system-dpdk.at | 129 +++++++++++++++++++++++++++ > > 6 files changed, 225 insertions(+) > > create mode 100755 tests/mfex_fuzzy.py > > create mode 100644 tests/pcap/mfex_test.pcap > > > > diff --git a/Documentation/topics/dpdk/bridge.rst > b/Documentation/topics/dpdk/bridge.rst > > index 8c500c504..913b3e6f6 100644 > > --- a/Documentation/topics/dpdk/bridge.rst > > +++ b/Documentation/topics/dpdk/bridge.rst > > @@ -346,3 +346,59 @@ A compile time option is available in order to test it > with the OVS unit > > test suite. Use the following configure option :: > > > > $ ./configure --enable-mfex-default-autovalidator > > + > > +Unit Test Miniflow Extract > > +++++++++++++++++++++++++++ > > + > > +Unit test can also be used to test the workflow mentioned above by running > > +the following test-case in tests/system-dpdk.at :: > > + > > + make check-dpdk TESTSUITEFLAGS='-k MFEX' > > + OVS-DPDK - MFEX Autovalidator > > + > > +The unit test uses mulitple traffic type to test the correctness of the > > +implementaions. > > + > > +The MFEX commands can also be tested for negative and positive cases to > > +verify that the MFEX set command does not allow for incorrect parameters. > > +A user can directly run the following configuration test case in > > +tests/system-dpdk.at :: > > + > > + make check-dpdk TESTSUITEFLAGS='-k MFEX' > > + OVS-DPDK - MFEX Configuration > > + > > +Running Fuzzy test with Autovalidator > > ++++++++++++++++++++++++++++++++++++++ > > + > > +Fuzzy tests can also be done on miniflow extract with the help of > > +auto-validator and Scapy. The steps below describes the steps to > > +reproduce the setup with IP being fuzzed to generate packets. > > + > > +Scapy is used to create fuzzy IP packets and save them into a PCAP :: > > + > > + pkt = fuzz(Ether()/IP()/TCP()) > > + > > +Set the miniflow extract to autovalidator using :: > > + > > + $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator > > + > > +OVS is configured to receive the generated packets :: > > + > > + $ ovs-vsctl add-port br0 pcap0 -- \ > > + set Interface pcap0 type=dpdk options:dpdk-devargs=net_pcap0 > > + "rx_pcap=fuzzy.pcap" > > + > > +With this workflow, the autovalidator will ensure that all MFEX > > +implementations are classifying each packet in exactly the same way. > > +If an optimized MFEX implementation causes a different miniflow to be > > +generated, the autovalidator has ovs_assert and logging statements that > > +will inform about the issue. > > + > > +Unit Fuzzy test with Autovalidator > > ++++++++++++++++++++++++++++++++++++++ > > + > > +Unit test can also be used to test the workflow mentioned above by running > > +the following test-case in tests/system-dpdk.at :: > > + > > + make check-dpdk TESTSUITEFLAGS='-k MFEX' > > + OVS-DPDK - MFEX Autovalidator Fuzzy > > diff --git a/tests/.gitignore b/tests/.gitignore > > index 45b4f67b2..a3d927e5d 100644 > > --- a/tests/.gitignore > > +++ b/tests/.gitignore > > @@ -11,6 +11,7 @@ > > /ovsdb-cluster-testsuite > > /ovsdb-cluster-testsuite.dir/ > > /ovsdb-cluster-testsuite.log > > +/pcap/ > > /pki/ > > /system-afxdp-testsuite > > /system-afxdp-testsuite.dir/ > > diff --git a/tests/automake.mk b/tests/automake.mk > > index f45f8d76c..a6c15ba55 100644 > > --- a/tests/automake.mk > > +++ b/tests/automake.mk > > @@ -143,6 +143,11 @@ $(srcdir)/tests/fuzz-regression-list.at: > tests/automake.mk > > echo "TEST_FUZZ_REGRESSION([$$basename])"; \ > > done > [email protected] && mv [email protected] $@ > > > > +EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS) > > +MFEX_AUTOVALIDATOR_TESTS = \ > > + tests/pcap/mfex_test.pcap \ > > + tests/mfex_fuzzy.py > > + > > OVSDB_CLUSTER_TESTSUITE_AT = \ > > tests/ovsdb-cluster-testsuite.at \ > > tests/ovsdb-execution.at \ > > @@ -512,6 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type- > props.c > > CHECK_PYFILES = \ > > tests/appctl.py \ > > tests/flowgen.py \ > > + tests/mfex_fuzzy.py \ > > tests/ovsdb-monitor-sort.py \ > > tests/test-daemon.py \ > > tests/test-json.py \ > > diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py > > new file mode 100755 > > index 000000000..5b056bb48 > > --- /dev/null > > +++ b/tests/mfex_fuzzy.py > > @@ -0,0 +1,33 @@ > > +#!/usr/bin/python3 > > +try: > > + from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, > fuzz > > + from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP > > +except ModuleNotFoundError as err: > > + print(err + ": Scapy") > > +import sys > > + > > +path = str(sys.argv[1]) + "/pcap/fuzzy.pcap" > > +pktdump = PcapWriter(path, append=False, sync=True) > > + > > +for i in range(0, 2000): > > + > > + # Generate random protocol bases, use a fuzz() over the combined packet > > + # for full fuzzing. > > + eth = Ether(src=RandMAC(), dst=RandMAC()) > > + vlan = Dot1Q() > > + ipv4 = IP(src=RandIP(), dst=RandIP()) > > + ipv6 = IPv6(src=RandIP6(), dst=RandIP6()) > > + udp = UDP(dport=RandShort(), sport=RandShort()) > > + tcp = TCP(dport=RandShort(), sport=RandShort()) > > + > > + # IPv4 packets with fuzzing > > + pktdump.write(fuzz(eth / ipv4 / udp)) > > + pktdump.write(fuzz(eth / ipv4 / tcp)) > > + pktdump.write(fuzz(eth / vlan / ipv4 / udp)) > > + pktdump.write(fuzz(eth / vlan / ipv4 / tcp)) > > + > > + # IPv6 packets with fuzzing > > + pktdump.write(fuzz(eth / ipv6 / udp)) > > + pktdump.write(fuzz(eth / ipv6 / tcp)) > > + pktdump.write(fuzz(eth / vlan / ipv6 / udp)) > > + pktdump.write(fuzz(eth / vlan / ipv6 / tcp)) > > diff --git a/tests/pcap/mfex_test.pcap b/tests/pcap/mfex_test.pcap > > new file mode 100644 > > index > 0000000000000000000000000000000000000000..1aac67b8d643ecb016c758cb > a4cc32212a80f52a > > GIT binary patch > > literal 416 > > > zcmca|c+)~A1{MYw`2U}Qff2}Q<eHVR>K`M68ITRa|G@yFii5$Gfk6YL%z>@uY&}o > | > > > z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$<F=a4i;9x8Q*<ZrSM6Uf > z > > xK>KST_NTIwYriok6N4Vm)gX- > Q@<yO<!C`>c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW > > > > literal 0 > > HcmV?d00001 > > > > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at > > index 802895488..a764c69ca 100644 > > --- a/tests/system-dpdk.at > > +++ b/tests/system-dpdk.at > > @@ -232,3 +232,132 @@ OVS_VSWITCHD_STOP(["\@does not exist. The > Open vSwitch kernel module is probably > > \@EAL: No free hugepages reported in hugepages-1048576kB@d"]) > > AT_CLEANUP > > dnl > > -------------------------------------------------------------------------- > > + > > +dnl > > -------------------------------------------------------------------------- > > +dnl Add standard DPDK PHY port > > +AT_SETUP([OVS-DPDK - MFEX Autovalidator]) > > +AT_KEYWORDS([dpdk]) > > + > > +OVS_DPDK_START() > > + > > +dnl Add userspace bridge and attach it to OVS > > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) > > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk > options:dpdk- > devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], > [stdout], [stderr]) > > +AT_CHECK([ovs-vsctl show], [], [stdout]) > > + > > +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep > "True"], [], [dnl > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], > [dnl > > +Miniflow extract implementation set to autovalidator. > > +]) > > + > > +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP > 'rx_packets=\s*\K\d+'` -ge 1000]) > > + > > +dnl Clean up > > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) > > +AT_CLEANUP > > +dnl > > -------------------------------------------------------------------------- > > + > > +dnl > > -------------------------------------------------------------------------- > > +dnl Add standard DPDK PHY port > > +AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy]) > > +AT_KEYWORDS([dpdk]) > > +AT_SKIP_IF([! pip3 list | grep scapy], [], []) > > +AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout]) > > +OVS_DPDK_START() > > + > > +dnl Add userspace bridge and attach it to OVS > > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) > > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk > options:dpdk- > devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy.pcap,infinite_rx=1], [], > [stdout], > [stderr]) > > +AT_CHECK([ovs-vsctl show], [], [stdout]) > > + > > +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep > "True"], [], [dnl > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], > [dnl > > +Miniflow extract implementation set to autovalidator. > > +]) > > + > > +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP > 'rx_packets=\s*\K\d+'` -ge 100000]) > > + > > +dnl Clean up > > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) > > +AT_CLEANUP > > +dnl > > -------------------------------------------------------------------------- > > + > > +dnl > > -------------------------------------------------------------------------- > > +AT_SETUP([OVS-DPDK - MFEX Configuration]) > > +AT_KEYWORDS([dpdk]) > > +OVS_DPDK_START() > > +dnl Add userspace bridge and attach it to OVS > > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) > > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk > options:dpdk- > devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], > [stdout], [stderr]) > > +AT_CHECK([ovs-vsctl show], [], [stdout]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar 1], [2], > > +[], [dnl > > +Error: unknown argument 1. > > +ovs-appctl: ovs-vswitchd: server returned an error > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study 300 > xyz], [2], > > +[], [dnl > > +Error: Invalid study_pkt_cnt value: xyz. > > +ovs-appctl: ovs-vswitchd: server returned an error > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar abcd], [2], > > +[], [dnl > > +Error: unknown argument abcd. > > +ovs-appctl: ovs-vswitchd: server returned an error > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 0 scalar abcd], > [2], > > +[], [dnl > > +Error: unknown argument abcd. > > +ovs-appctl: ovs-vswitchd: server returned an error > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd], [2], > > +[], [dnl > > +Error: -pmd option requires a thread id argument. > > +ovs-appctl: ovs-vswitchd: server returned an error > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set tudy abcd], [2], > > +[], [dnl > > +Error: unknown argument abcd. > > +ovs-appctl: ovs-vswitchd: server returned an error > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study abcd], > [2], > > +[], [dnl > > +Error: Invalid study_pkt_cnt value: abcd. > > +ovs-appctl: ovs-vswitchd: server returned an error > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0], > [dnl > > +Miniflow extract implementation set to study, on pmd thread 21, studying > 128 packets. > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study 512], > [0], [dnl > > +Miniflow extract implementation set to study, on pmd thread 21, studying > 512 packets. > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study 512], [0], [dnl > > +Miniflow extract implementation set to study, studying 512 packets. > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study], [0], [dnl > > +Miniflow extract implementation set to study, studying 128 packets. > > +]) > > + > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 > autovalidator], [0], [dnl > > +Miniflow extract implementation set to autovalidator, on pmd thread 21. > > +]) > > + > > +dnl Clean up > > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) > > +AT_CLEANUP dnl > > +dnl > > -------------------------------------------------------------------------- > > -- > > It’s missing some of the negative tests cases, which I have added based on the > comment on the v6 patch on name changes. > Here is the diff: > > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at > index a764c69ca..929bbf0a9 100644 > --- a/tests/system-dpdk.at > +++ b/tests/system-dpdk.at > @@ -337,6 +337,30 @@ Error: Invalid study_pkt_cnt value: abcd. > ovs-appctl: ovs-vswitchd: server returned an error > ]) > > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd zero study], [2], > +[], [dnl > +Error: Miniflow extract parser not changed, PMD thread argument passed is > not valid: 'zero'. Pass a valid pmd thread ID. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1], [2], > +[], [dnl > +Error: no miniflow extract name provided. Output of miniflow-parser-get > shows implementation list. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1 superstudy], > [2], > +[], [dnl > +Error: unknown miniflow extract implementation superstudy. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set superstudy], [2], > +[], [dnl > +Error: unknown miniflow extract implementation superstudy. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1 study -pmd], > [2], > +[], [dnl > +Error: Invalid study_pkt_cnt value: -pmd. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0], > [dnl > Miniflow extract implementation set to study, on pmd thread 21, studying 128 > packets. > ]) _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
