On 15 Jul 2021, at 17:06, Stokes, Ian wrote:
>> Added some missing negative test cases, see below. >> > > Thanks for these Eelco, would you like me to add you as a co-author for this > patch as you have code that is part of the next revison? Whatever is easiest for you, I’m fine either way. > Regards > Ian >> On 15 Jul 2021, at 14:42, kumar Amber wrote: >> >>> From: Kumar Amber <[email protected]> >>> >>> Tests: >>> 6: OVS-DPDK - MFEX Autovalidator >>> 7: OVS-DPDK - MFEX Autovalidator Fuzzy >>> 8: OVS-DPDK - MFEX Configuration >>> >>> Added a new directory to store the PCAP file used >>> in the tests and a script to generate the fuzzy traffic >>> type pcap to be used in fuzzy unit test. >>> >>> Signed-off-by: Kumar Amber <[email protected]> >>> Acked-by: Flavio Leitner <[email protected]> >>> >>> --- >>> v13: >>> - fix -v in the command >>> - added the configuration test case and supporting doc update >>> v12: >>> - change skip paramter for unit test >>> v11: >>> - fix comments from Eelco >>> v7: >>> - fix review comments(Eelco) >>> v5: >>> - fix review comments(Ian, Flavio, Eelco) >>> - remove sleep from first test and added minor 5 sec sleep to fuzzy >>> --- >>> --- >>> Documentation/topics/dpdk/bridge.rst | 56 ++++++++++++ >>> tests/.gitignore | 1 + >>> tests/automake.mk | 6 ++ >>> tests/mfex_fuzzy.py | 33 +++++++ >>> tests/pcap/mfex_test.pcap | Bin 0 -> 416 bytes >>> tests/system-dpdk.at | 129 +++++++++++++++++++++++++++ >>> 6 files changed, 225 insertions(+) >>> create mode 100755 tests/mfex_fuzzy.py >>> create mode 100644 tests/pcap/mfex_test.pcap >>> >>> diff --git a/Documentation/topics/dpdk/bridge.rst >> b/Documentation/topics/dpdk/bridge.rst >>> index 8c500c504..913b3e6f6 100644 >>> --- a/Documentation/topics/dpdk/bridge.rst >>> +++ b/Documentation/topics/dpdk/bridge.rst >>> @@ -346,3 +346,59 @@ A compile time option is available in order to test it >> with the OVS unit >>> test suite. Use the following configure option :: >>> >>> $ ./configure --enable-mfex-default-autovalidator >>> + >>> +Unit Test Miniflow Extract >>> +++++++++++++++++++++++++++ >>> + >>> +Unit test can also be used to test the workflow mentioned above by running >>> +the following test-case in tests/system-dpdk.at :: >>> + >>> + make check-dpdk TESTSUITEFLAGS='-k MFEX' >>> + OVS-DPDK - MFEX Autovalidator >>> + >>> +The unit test uses mulitple traffic type to test the correctness of the >>> +implementaions. >>> + >>> +The MFEX commands can also be tested for negative and positive cases to >>> +verify that the MFEX set command does not allow for incorrect parameters. >>> +A user can directly run the following configuration test case in >>> +tests/system-dpdk.at :: >>> + >>> + make check-dpdk TESTSUITEFLAGS='-k MFEX' >>> + OVS-DPDK - MFEX Configuration >>> + >>> +Running Fuzzy test with Autovalidator >>> ++++++++++++++++++++++++++++++++++++++ >>> + >>> +Fuzzy tests can also be done on miniflow extract with the help of >>> +auto-validator and Scapy. The steps below describes the steps to >>> +reproduce the setup with IP being fuzzed to generate packets. >>> + >>> +Scapy is used to create fuzzy IP packets and save them into a PCAP :: >>> + >>> + pkt = fuzz(Ether()/IP()/TCP()) >>> + >>> +Set the miniflow extract to autovalidator using :: >>> + >>> + $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator >>> + >>> +OVS is configured to receive the generated packets :: >>> + >>> + $ ovs-vsctl add-port br0 pcap0 -- \ >>> + set Interface pcap0 type=dpdk options:dpdk-devargs=net_pcap0 >>> + "rx_pcap=fuzzy.pcap" >>> + >>> +With this workflow, the autovalidator will ensure that all MFEX >>> +implementations are classifying each packet in exactly the same way. >>> +If an optimized MFEX implementation causes a different miniflow to be >>> +generated, the autovalidator has ovs_assert and logging statements that >>> +will inform about the issue. >>> + >>> +Unit Fuzzy test with Autovalidator >>> ++++++++++++++++++++++++++++++++++++++ >>> + >>> +Unit test can also be used to test the workflow mentioned above by running >>> +the following test-case in tests/system-dpdk.at :: >>> + >>> + make check-dpdk TESTSUITEFLAGS='-k MFEX' >>> + OVS-DPDK - MFEX Autovalidator Fuzzy >>> diff --git a/tests/.gitignore b/tests/.gitignore >>> index 45b4f67b2..a3d927e5d 100644 >>> --- a/tests/.gitignore >>> +++ b/tests/.gitignore >>> @@ -11,6 +11,7 @@ >>> /ovsdb-cluster-testsuite >>> /ovsdb-cluster-testsuite.dir/ >>> /ovsdb-cluster-testsuite.log >>> +/pcap/ >>> /pki/ >>> /system-afxdp-testsuite >>> /system-afxdp-testsuite.dir/ >>> diff --git a/tests/automake.mk b/tests/automake.mk >>> index f45f8d76c..a6c15ba55 100644 >>> --- a/tests/automake.mk >>> +++ b/tests/automake.mk >>> @@ -143,6 +143,11 @@ $(srcdir)/tests/fuzz-regression-list.at: >> tests/automake.mk >>> echo "TEST_FUZZ_REGRESSION([$$basename])"; \ >>> done > [email protected] && mv [email protected] $@ >>> >>> +EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS) >>> +MFEX_AUTOVALIDATOR_TESTS = \ >>> + tests/pcap/mfex_test.pcap \ >>> + tests/mfex_fuzzy.py >>> + >>> OVSDB_CLUSTER_TESTSUITE_AT = \ >>> tests/ovsdb-cluster-testsuite.at \ >>> tests/ovsdb-execution.at \ >>> @@ -512,6 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type- >> props.c >>> CHECK_PYFILES = \ >>> tests/appctl.py \ >>> tests/flowgen.py \ >>> + tests/mfex_fuzzy.py \ >>> tests/ovsdb-monitor-sort.py \ >>> tests/test-daemon.py \ >>> tests/test-json.py \ >>> diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py >>> new file mode 100755 >>> index 000000000..5b056bb48 >>> --- /dev/null >>> +++ b/tests/mfex_fuzzy.py >>> @@ -0,0 +1,33 @@ >>> +#!/usr/bin/python3 >>> +try: >>> + from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, >> fuzz >>> + from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP >>> +except ModuleNotFoundError as err: >>> + print(err + ": Scapy") >>> +import sys >>> + >>> +path = str(sys.argv[1]) + "/pcap/fuzzy.pcap" >>> +pktdump = PcapWriter(path, append=False, sync=True) >>> + >>> +for i in range(0, 2000): >>> + >>> + # Generate random protocol bases, use a fuzz() over the combined packet >>> + # for full fuzzing. >>> + eth = Ether(src=RandMAC(), dst=RandMAC()) >>> + vlan = Dot1Q() >>> + ipv4 = IP(src=RandIP(), dst=RandIP()) >>> + ipv6 = IPv6(src=RandIP6(), dst=RandIP6()) >>> + udp = UDP(dport=RandShort(), sport=RandShort()) >>> + tcp = TCP(dport=RandShort(), sport=RandShort()) >>> + >>> + # IPv4 packets with fuzzing >>> + pktdump.write(fuzz(eth / ipv4 / udp)) >>> + pktdump.write(fuzz(eth / ipv4 / tcp)) >>> + pktdump.write(fuzz(eth / vlan / ipv4 / udp)) >>> + pktdump.write(fuzz(eth / vlan / ipv4 / tcp)) >>> + >>> + # IPv6 packets with fuzzing >>> + pktdump.write(fuzz(eth / ipv6 / udp)) >>> + pktdump.write(fuzz(eth / ipv6 / tcp)) >>> + pktdump.write(fuzz(eth / vlan / ipv6 / udp)) >>> + pktdump.write(fuzz(eth / vlan / ipv6 / tcp)) >>> diff --git a/tests/pcap/mfex_test.pcap b/tests/pcap/mfex_test.pcap >>> new file mode 100644 >>> index >> 0000000000000000000000000000000000000000..1aac67b8d643ecb016c758cb >> a4cc32212a80f52a >>> GIT binary patch >>> literal 416 >>> >> zcmca|c+)~A1{MYw`2U}Qff2}Q<eHVR>K`M68ITRa|G@yFii5$Gfk6YL%z>@uY&}o >> | >>> >> z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$<F=a4i;9x8Q*<ZrSM6Uf >> z >>> xK>KST_NTIwYriok6N4Vm)gX- >> Q@<yO<!C`>c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW >>> >>> literal 0 >>> HcmV?d00001 >>> >>> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at >>> index 802895488..a764c69ca 100644 >>> --- a/tests/system-dpdk.at >>> +++ b/tests/system-dpdk.at >>> @@ -232,3 +232,132 @@ OVS_VSWITCHD_STOP(["\@does not exist. The >> Open vSwitch kernel module is probably >>> \@EAL: No free hugepages reported in hugepages-1048576kB@d"]) >>> AT_CLEANUP >>> dnl >>> -------------------------------------------------------------------------- >>> + >>> +dnl >>> -------------------------------------------------------------------------- >>> +dnl Add standard DPDK PHY port >>> +AT_SETUP([OVS-DPDK - MFEX Autovalidator]) >>> +AT_KEYWORDS([dpdk]) >>> + >>> +OVS_DPDK_START() >>> + >>> +dnl Add userspace bridge and attach it to OVS >>> +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) >>> +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk >> options:dpdk- >> devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], >> [stdout], [stderr]) >>> +AT_CHECK([ovs-vsctl show], [], [stdout]) >>> + >>> +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep >> "True"], [], [dnl >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], >> [dnl >>> +Miniflow extract implementation set to autovalidator. >>> +]) >>> + >>> +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP >> 'rx_packets=\s*\K\d+'` -ge 1000]) >>> + >>> +dnl Clean up >>> +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) >>> +AT_CLEANUP >>> +dnl >>> -------------------------------------------------------------------------- >>> + >>> +dnl >>> -------------------------------------------------------------------------- >>> +dnl Add standard DPDK PHY port >>> +AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy]) >>> +AT_KEYWORDS([dpdk]) >>> +AT_SKIP_IF([! pip3 list | grep scapy], [], []) >>> +AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout]) >>> +OVS_DPDK_START() >>> + >>> +dnl Add userspace bridge and attach it to OVS >>> +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) >>> +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk >> options:dpdk- >> devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy.pcap,infinite_rx=1], [], >> [stdout], >> [stderr]) >>> +AT_CHECK([ovs-vsctl show], [], [stdout]) >>> + >>> +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep >> "True"], [], [dnl >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], >> [dnl >>> +Miniflow extract implementation set to autovalidator. >>> +]) >>> + >>> +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP >> 'rx_packets=\s*\K\d+'` -ge 100000]) >>> + >>> +dnl Clean up >>> +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) >>> +AT_CLEANUP >>> +dnl >>> -------------------------------------------------------------------------- >>> + >>> +dnl >>> -------------------------------------------------------------------------- >>> +AT_SETUP([OVS-DPDK - MFEX Configuration]) >>> +AT_KEYWORDS([dpdk]) >>> +OVS_DPDK_START() >>> +dnl Add userspace bridge and attach it to OVS >>> +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) >>> +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk >> options:dpdk- >> devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], >> [stdout], [stderr]) >>> +AT_CHECK([ovs-vsctl show], [], [stdout]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar 1], [2], >>> +[], [dnl >>> +Error: unknown argument 1. >>> +ovs-appctl: ovs-vswitchd: server returned an error >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study 300 >> xyz], [2], >>> +[], [dnl >>> +Error: Invalid study_pkt_cnt value: xyz. >>> +ovs-appctl: ovs-vswitchd: server returned an error >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar abcd], [2], >>> +[], [dnl >>> +Error: unknown argument abcd. >>> +ovs-appctl: ovs-vswitchd: server returned an error >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 0 scalar abcd], >> [2], >>> +[], [dnl >>> +Error: unknown argument abcd. >>> +ovs-appctl: ovs-vswitchd: server returned an error >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd], [2], >>> +[], [dnl >>> +Error: -pmd option requires a thread id argument. >>> +ovs-appctl: ovs-vswitchd: server returned an error >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set tudy abcd], [2], >>> +[], [dnl >>> +Error: unknown argument abcd. >>> +ovs-appctl: ovs-vswitchd: server returned an error >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study abcd], >> [2], >>> +[], [dnl >>> +Error: Invalid study_pkt_cnt value: abcd. >>> +ovs-appctl: ovs-vswitchd: server returned an error >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0], >> [dnl >>> +Miniflow extract implementation set to study, on pmd thread 21, studying >> 128 packets. >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study 512], >> [0], [dnl >>> +Miniflow extract implementation set to study, on pmd thread 21, studying >> 512 packets. >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study 512], [0], [dnl >>> +Miniflow extract implementation set to study, studying 512 packets. >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study], [0], [dnl >>> +Miniflow extract implementation set to study, studying 128 packets. >>> +]) >>> + >>> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 >> autovalidator], [0], [dnl >>> +Miniflow extract implementation set to autovalidator, on pmd thread 21. >>> +]) >>> + >>> +dnl Clean up >>> +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) >>> +AT_CLEANUP dnl >>> +dnl >>> -------------------------------------------------------------------------- >>> -- >> >> It’s missing some of the negative tests cases, which I have added based on >> the >> comment on the v6 patch on name changes. >> Here is the diff: >> >> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at >> index a764c69ca..929bbf0a9 100644 >> --- a/tests/system-dpdk.at >> +++ b/tests/system-dpdk.at >> @@ -337,6 +337,30 @@ Error: Invalid study_pkt_cnt value: abcd. >> ovs-appctl: ovs-vswitchd: server returned an error >> ]) >> >> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd zero study], [2], >> +[], [dnl >> +Error: Miniflow extract parser not changed, PMD thread argument passed is >> not valid: 'zero'. Pass a valid pmd thread ID. >> +ovs-appctl: ovs-vswitchd: server returned an error >> +]) >> + >> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1], [2], >> +[], [dnl >> +Error: no miniflow extract name provided. Output of miniflow-parser-get >> shows implementation list. >> +ovs-appctl: ovs-vswitchd: server returned an error >> +]) >> + >> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1 superstudy], >> [2], >> +[], [dnl >> +Error: unknown miniflow extract implementation superstudy. >> +ovs-appctl: ovs-vswitchd: server returned an error >> +]) >> + >> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set superstudy], [2], >> +[], [dnl >> +Error: unknown miniflow extract implementation superstudy. >> +ovs-appctl: ovs-vswitchd: server returned an error >> +]) >> + >> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1 study -pmd], >> [2], >> +[], [dnl >> +Error: Invalid study_pkt_cnt value: -pmd. >> +ovs-appctl: ovs-vswitchd: server returned an error >> +]) >> + >> AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0], >> [dnl >> Miniflow extract implementation set to study, on pmd thread 21, studying 128 >> packets. >> ]) _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
