Added some missing negative test cases, see below. On 15 Jul 2021, at 14:42, kumar Amber wrote:
> From: Kumar Amber <[email protected]> > > Tests: > 6: OVS-DPDK - MFEX Autovalidator > 7: OVS-DPDK - MFEX Autovalidator Fuzzy > 8: OVS-DPDK - MFEX Configuration > > Added a new directory to store the PCAP file used > in the tests and a script to generate the fuzzy traffic > type pcap to be used in fuzzy unit test. > > Signed-off-by: Kumar Amber <[email protected]> > Acked-by: Flavio Leitner <[email protected]> > > --- > v13: > - fix -v in the command > - added the configuration test case and supporting doc update > v12: > - change skip paramter for unit test > v11: > - fix comments from Eelco > v7: > - fix review comments(Eelco) > v5: > - fix review comments(Ian, Flavio, Eelco) > - remove sleep from first test and added minor 5 sec sleep to fuzzy > --- > --- > Documentation/topics/dpdk/bridge.rst | 56 ++++++++++++ > tests/.gitignore | 1 + > tests/automake.mk | 6 ++ > tests/mfex_fuzzy.py | 33 +++++++ > tests/pcap/mfex_test.pcap | Bin 0 -> 416 bytes > tests/system-dpdk.at | 129 +++++++++++++++++++++++++++ > 6 files changed, 225 insertions(+) > create mode 100755 tests/mfex_fuzzy.py > create mode 100644 tests/pcap/mfex_test.pcap > > diff --git a/Documentation/topics/dpdk/bridge.rst > b/Documentation/topics/dpdk/bridge.rst > index 8c500c504..913b3e6f6 100644 > --- a/Documentation/topics/dpdk/bridge.rst > +++ b/Documentation/topics/dpdk/bridge.rst > @@ -346,3 +346,59 @@ A compile time option is available in order to test it > with the OVS unit > test suite. Use the following configure option :: > > $ ./configure --enable-mfex-default-autovalidator > + > +Unit Test Miniflow Extract > +++++++++++++++++++++++++++ > + > +Unit test can also be used to test the workflow mentioned above by running > +the following test-case in tests/system-dpdk.at :: > + > + make check-dpdk TESTSUITEFLAGS='-k MFEX' > + OVS-DPDK - MFEX Autovalidator > + > +The unit test uses mulitple traffic type to test the correctness of the > +implementaions. > + > +The MFEX commands can also be tested for negative and positive cases to > +verify that the MFEX set command does not allow for incorrect parameters. > +A user can directly run the following configuration test case in > +tests/system-dpdk.at :: > + > + make check-dpdk TESTSUITEFLAGS='-k MFEX' > + OVS-DPDK - MFEX Configuration > + > +Running Fuzzy test with Autovalidator > ++++++++++++++++++++++++++++++++++++++ > + > +Fuzzy tests can also be done on miniflow extract with the help of > +auto-validator and Scapy. The steps below describes the steps to > +reproduce the setup with IP being fuzzed to generate packets. > + > +Scapy is used to create fuzzy IP packets and save them into a PCAP :: > + > + pkt = fuzz(Ether()/IP()/TCP()) > + > +Set the miniflow extract to autovalidator using :: > + > + $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator > + > +OVS is configured to receive the generated packets :: > + > + $ ovs-vsctl add-port br0 pcap0 -- \ > + set Interface pcap0 type=dpdk options:dpdk-devargs=net_pcap0 > + "rx_pcap=fuzzy.pcap" > + > +With this workflow, the autovalidator will ensure that all MFEX > +implementations are classifying each packet in exactly the same way. > +If an optimized MFEX implementation causes a different miniflow to be > +generated, the autovalidator has ovs_assert and logging statements that > +will inform about the issue. > + > +Unit Fuzzy test with Autovalidator > ++++++++++++++++++++++++++++++++++++++ > + > +Unit test can also be used to test the workflow mentioned above by running > +the following test-case in tests/system-dpdk.at :: > + > + make check-dpdk TESTSUITEFLAGS='-k MFEX' > + OVS-DPDK - MFEX Autovalidator Fuzzy > diff --git a/tests/.gitignore b/tests/.gitignore > index 45b4f67b2..a3d927e5d 100644 > --- a/tests/.gitignore > +++ b/tests/.gitignore > @@ -11,6 +11,7 @@ > /ovsdb-cluster-testsuite > /ovsdb-cluster-testsuite.dir/ > /ovsdb-cluster-testsuite.log > +/pcap/ > /pki/ > /system-afxdp-testsuite > /system-afxdp-testsuite.dir/ > diff --git a/tests/automake.mk b/tests/automake.mk > index f45f8d76c..a6c15ba55 100644 > --- a/tests/automake.mk > +++ b/tests/automake.mk > @@ -143,6 +143,11 @@ $(srcdir)/tests/fuzz-regression-list.at: > tests/automake.mk > echo "TEST_FUZZ_REGRESSION([$$basename])"; \ > done > [email protected] && mv [email protected] $@ > > +EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS) > +MFEX_AUTOVALIDATOR_TESTS = \ > + tests/pcap/mfex_test.pcap \ > + tests/mfex_fuzzy.py > + > OVSDB_CLUSTER_TESTSUITE_AT = \ > tests/ovsdb-cluster-testsuite.at \ > tests/ovsdb-execution.at \ > @@ -512,6 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type-props.c > CHECK_PYFILES = \ > tests/appctl.py \ > tests/flowgen.py \ > + tests/mfex_fuzzy.py \ > tests/ovsdb-monitor-sort.py \ > tests/test-daemon.py \ > tests/test-json.py \ > diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py > new file mode 100755 > index 000000000..5b056bb48 > --- /dev/null > +++ b/tests/mfex_fuzzy.py > @@ -0,0 +1,33 @@ > +#!/usr/bin/python3 > +try: > + from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, > fuzz > + from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP > +except ModuleNotFoundError as err: > + print(err + ": Scapy") > +import sys > + > +path = str(sys.argv[1]) + "/pcap/fuzzy.pcap" > +pktdump = PcapWriter(path, append=False, sync=True) > + > +for i in range(0, 2000): > + > + # Generate random protocol bases, use a fuzz() over the combined packet > + # for full fuzzing. > + eth = Ether(src=RandMAC(), dst=RandMAC()) > + vlan = Dot1Q() > + ipv4 = IP(src=RandIP(), dst=RandIP()) > + ipv6 = IPv6(src=RandIP6(), dst=RandIP6()) > + udp = UDP(dport=RandShort(), sport=RandShort()) > + tcp = TCP(dport=RandShort(), sport=RandShort()) > + > + # IPv4 packets with fuzzing > + pktdump.write(fuzz(eth / ipv4 / udp)) > + pktdump.write(fuzz(eth / ipv4 / tcp)) > + pktdump.write(fuzz(eth / vlan / ipv4 / udp)) > + pktdump.write(fuzz(eth / vlan / ipv4 / tcp)) > + > + # IPv6 packets with fuzzing > + pktdump.write(fuzz(eth / ipv6 / udp)) > + pktdump.write(fuzz(eth / ipv6 / tcp)) > + pktdump.write(fuzz(eth / vlan / ipv6 / udp)) > + pktdump.write(fuzz(eth / vlan / ipv6 / tcp)) > diff --git a/tests/pcap/mfex_test.pcap b/tests/pcap/mfex_test.pcap > new file mode 100644 > index > 0000000000000000000000000000000000000000..1aac67b8d643ecb016c758cba4cc32212a80f52a > GIT binary patch > literal 416 > zcmca|c+)~A1{MYw`2U}Qff2}Q<eHVR>K`M68ITRa|G@yFii5$Gfk6YL%z>@uY&}o| > z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$<F=a4i;9x8Q*<ZrSM6Ufz > xK>KST_NTIwYriok6N4Vm)gX-Q@<yO<!C`>c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW > > literal 0 > HcmV?d00001 > > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at > index 802895488..a764c69ca 100644 > --- a/tests/system-dpdk.at > +++ b/tests/system-dpdk.at > @@ -232,3 +232,132 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch > kernel module is probably > \@EAL: No free hugepages reported in hugepages-1048576kB@d"]) > AT_CLEANUP > dnl > -------------------------------------------------------------------------- > + > +dnl > -------------------------------------------------------------------------- > +dnl Add standard DPDK PHY port > +AT_SETUP([OVS-DPDK - MFEX Autovalidator]) > +AT_KEYWORDS([dpdk]) > + > +OVS_DPDK_START() > + > +dnl Add userspace bridge and attach it to OVS > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk > options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], > [], [stdout], [stderr]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > + > +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep > "True"], [], [dnl > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], > [dnl > +Miniflow extract implementation set to autovalidator. > +]) > + > +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP > 'rx_packets=\s*\K\d+'` -ge 1000]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > +dnl > -------------------------------------------------------------------------- > +dnl Add standard DPDK PHY port > +AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy]) > +AT_KEYWORDS([dpdk]) > +AT_SKIP_IF([! pip3 list | grep scapy], [], []) > +AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout]) > +OVS_DPDK_START() > + > +dnl Add userspace bridge and attach it to OVS > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk > options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy.pcap,infinite_rx=1], > [], [stdout], [stderr]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > + > +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep > "True"], [], [dnl > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], > [dnl > +Miniflow extract implementation set to autovalidator. > +]) > + > +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP > 'rx_packets=\s*\K\d+'` -ge 100000]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > +dnl > -------------------------------------------------------------------------- > +AT_SETUP([OVS-DPDK - MFEX Configuration]) > +AT_KEYWORDS([dpdk]) > +OVS_DPDK_START() > +dnl Add userspace bridge and attach it to OVS > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk > options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], > [], [stdout], [stderr]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar 1], [2], > +[], [dnl > +Error: unknown argument 1. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study 300 xyz], > [2], > +[], [dnl > +Error: Invalid study_pkt_cnt value: xyz. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar abcd], [2], > +[], [dnl > +Error: unknown argument abcd. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 0 scalar abcd], > [2], > +[], [dnl > +Error: unknown argument abcd. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd], [2], > +[], [dnl > +Error: -pmd option requires a thread id argument. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set tudy abcd], [2], > +[], [dnl > +Error: unknown argument abcd. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study abcd], [2], > +[], [dnl > +Error: Invalid study_pkt_cnt value: abcd. > +ovs-appctl: ovs-vswitchd: server returned an error > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0], > [dnl > +Miniflow extract implementation set to study, on pmd thread 21, studying 128 > packets. > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study 512], > [0], [dnl > +Miniflow extract implementation set to study, on pmd thread 21, studying 512 > packets. > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study 512], [0], [dnl > +Miniflow extract implementation set to study, studying 512 packets. > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study], [0], [dnl > +Miniflow extract implementation set to study, studying 128 packets. > +]) > + > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 autovalidator], > [0], [dnl > +Miniflow extract implementation set to autovalidator, on pmd thread 21. > +]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) > +AT_CLEANUP dnl > +dnl > -------------------------------------------------------------------------- > -- It’s missing some of the negative tests cases, which I have added based on the comment on the v6 patch on name changes. Here is the diff: diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index a764c69ca..929bbf0a9 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -337,6 +337,30 @@ Error: Invalid study_pkt_cnt value: abcd. ovs-appctl: ovs-vswitchd: server returned an error ]) +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd zero study], [2], +[], [dnl +Error: Miniflow extract parser not changed, PMD thread argument passed is not valid: 'zero'. Pass a valid pmd thread ID. +ovs-appctl: ovs-vswitchd: server returned an error +]) + +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1], [2], +[], [dnl +Error: no miniflow extract name provided. Output of miniflow-parser-get shows implementation list. +ovs-appctl: ovs-vswitchd: server returned an error +]) + +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1 superstudy], [2], +[], [dnl +Error: unknown miniflow extract implementation superstudy. +ovs-appctl: ovs-vswitchd: server returned an error +]) + +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set superstudy], [2], +[], [dnl +Error: unknown miniflow extract implementation superstudy. +ovs-appctl: ovs-vswitchd: server returned an error +]) + +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1 study -pmd], [2], +[], [dnl +Error: Invalid study_pkt_cnt value: -pmd. +ovs-appctl: ovs-vswitchd: server returned an error +]) + AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0], [dnl Miniflow extract implementation set to study, on pmd thread 21, studying 128 packets. ]) _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
