On 6 Jul 2021, at 15:11, Cian Ferriter wrote:
From: Kumar Amber <[email protected]>
Tests:
6: OVS-DPDK - MFEX Autovalidator
7: OVS-DPDK - MFEX Autovalidator Fuzzy
Added a new directory to store the PCAP file used
in the tests and a script to generate the fuzzy traffic
type pcap to be used in fuzzy unit test.
Signed-off-by: Kumar Amber <[email protected]>
---
v5:
- fix review comments(Ian, Flavio, Eelco)
- remove sleep from first test and added minor 5 sec sleep to fuzzy
---
---
Documentation/topics/dpdk/bridge.rst | 55
+++++++++++++++++++++++++++
tests/automake.mk | 5 +++
tests/mfex_fuzzy.py | 32 ++++++++++++++++
tests/pcap/mfex_test | Bin 0 -> 416 bytes
tests/system-dpdk.at | 46 ++++++++++++++++++++++
5 files changed, 138 insertions(+)
create mode 100755 tests/mfex_fuzzy.py
create mode 100644 tests/pcap/mfex_test
diff --git a/Documentation/topics/dpdk/bridge.rst
b/Documentation/topics/dpdk/bridge.rst
index 8495687e8..8a8ef3782 100644
--- a/Documentation/topics/dpdk/bridge.rst
+++ b/Documentation/topics/dpdk/bridge.rst
@@ -341,3 +341,58 @@ A compile time option is available in order to
test it with the OVS unit
test suite. Use the following configure option ::
$ ./configure --enable-mfex-default-autovalidator
+
+Unit Test Miniflow Extract
+++++++++++++++++++++++++++
+
+Unit test can also be used to test the workflow mentioned above by
running
+the following test-case in tests/system-dpdk.at ::
+
+ make check-dpdk TESTSUITEFLAGS='-k MFEX'
+ OVS-DPDK - MFEX Autovalidator
+
+The unit test uses mulitple traffic type to test the correctness of
the
+implementaions.
+
+Running Fuzzy test with Autovalidator
++++++++++++++++++++++++++++++++++++++
+
+Fuzzy tests can also be done on miniflow extract with the help of
+auto-validator and Scapy. The steps below describes the steps to
+reproduce the setup with IP being fuzzed to generate packets.
+
+Scapy is used to create fuzzy IP packets and save them into a PCAP ::
+
+ pkt = fuzz(Ether()/IP()/TCP())
+
+Set the miniflow extract to autovalidator using ::
+
+ $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator
+
+OVS is configured to receive the generated packets ::
+
+ $ ovs-vsctl add-port br0 pcap0 -- \
+ set Interface pcap0 type=dpdk options:dpdk-devargs=net_pcap0
+ "rx_pcap=fuzzy.pcap"
+
+With this workflow, the autovalidator will ensure that all MFEX
+implementations are classifying each packet in exactly the same way.
+If an optimized MFEX implementation causes a different miniflow to be
+generated, the autovalidator has ovs_assert and logging statements
that
+will inform about the issue.
+
+Unit Fuzzy test with Autovalidator
++++++++++++++++++++++++++++++++++++++
+
+The prerquiste before running the unit test is to run the script
provided ::
+
+ tests/mfex_fuzzy.py
+
+This script generates a pcap with mulitple type of fuzzed packets to
be used
+in the below unit test-case.
+
+Unit test can also be used to test the workflow mentioned above by
running
+the following test-case in tests/system-dpdk.at ::
+
+ make check-dpdk TESTSUITEFLAGS='-k MFEX'
+ OVS-DPDK - MFEX Autovalidator Fuzzy
diff --git a/tests/automake.mk b/tests/automake.mk
index f45f8d76c..e94ccd27c 100644
--- a/tests/automake.mk
+++ b/tests/automake.mk
@@ -143,6 +143,11 @@ $(srcdir)/tests/fuzz-regression-list.at:
tests/automake.mk
echo "TEST_FUZZ_REGRESSION([$$basename])"; \
done > [email protected] && mv [email protected] $@
+EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS)
+MFEX_AUTOVALIDATOR_TESTS = \
+ tests/pcap/mfex_test \
+ tests/mfex_fuzzy.py
+
OVSDB_CLUSTER_TESTSUITE_AT = \
tests/ovsdb-cluster-testsuite.at \
tests/ovsdb-execution.at \
diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py
new file mode 100755
index 000000000..a8051ba2b
--- /dev/null
+++ b/tests/mfex_fuzzy.py
@@ -0,0 +1,32 @@
+#!/usr/bin/python3
+try:
+ from scapy.all import *
+except ModuleNotFoundError as err:
+ print(err + ": Scapy")
+import sys
+import os
+
+path = os.environ['OVS_DIR'] + "/tests/pcap/fuzzy"
This is failing in my setup, as OVS_DIR is not defined:
Is it something you set manually? As from make check-dpdk it fails:
7. system-dpdk.at:260: testing OVS-DPDK - MFEX Autovalidator Fuzzy ...
DEPRECATION: The default format will switch to columns in the future.
You can use --format=(legacy|columns) (or define a
format=(legacy|columns) in your pip.conf
under the [list] section) to disable this warning.
scapy (2.4.4)
./system-dpdk.at:263: $PYTHON3 $srcdir/mfex_fuzzy.py
--- /dev/null 2021-07-02 08:15:52.158758028 -0400
+++
/root/Documents/Scratch/ovs_review_mfex/OVS_master_DPDK_v20.11.1/ovs_github/tests/system-dpdk-testsuite.dir/at-groups/7/stderr 2021-07-07
10:34:09.364877
754 -0400
@@ -0,0 +1,6 @@
+Traceback (most recent call last):
+ File "../.././mfex_fuzzy.py", line 10, in <module>
+ path = os.environ['OVS_DIR'] + "/tests/pcap/fuzzy"
+ File "/usr/lib64/python3.6/os.py", line 669, in __getitem__
+ raise KeyError(key) from None
+KeyError: 'OVS_DIR'
stdout:
./system-dpdk.at:263: exit code was 1, expected 0
7. system-dpdk.at:260: 7. OVS-DPDK - MFEX Autovalidator Fuzzy
(system-dpdk.at:260): FAILED (system-dpdk.at:263)
If I set the environment variable it works fine, but should not be
needed.
+pktdump = PcapWriter(path, append=False, sync=True)
+
+for i in range(0, 2000):
+
+ # Generate random protocol bases, use a fuzz() over the combined
packet for full fuzzing.
+ eth = Ether(src=RandMAC(), dst=RandMAC())
+ vlan = Dot1Q()
+ ipv4 = IP(src=RandIP(), dst=RandIP())
+ ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
+ udp = UDP()
+ tcp = TCP()
+
I think we should also randomize the UDP/TCP ports as they get
extracted.
+ # IPv4 packets with fuzzing
+ pktdump.write(fuzz(eth/ipv4/udp))
+ pktdump.write(fuzz(eth/ipv4/tcp))
+ pktdump.write(fuzz(eth/vlan/ipv4/udp))
+ pktdump.write(fuzz(eth/vlan/ipv4/tcp))
+
+ # IPv6 packets with fuzzing
+ pktdump.write(fuzz(eth/ipv6/udp))
+ pktdump.write(fuzz(eth/ipv6/tcp))
+ pktdump.write(fuzz(eth/vlan/ipv6/udp))
+ pktdump.write(fuzz(eth/vlan/ipv6/tcp))
The generated pcap file does not have an extension, might be nice to at
it?
Also add the generated pcap file to the .git_ignore file.
\ No newline at end of file
diff --git a/tests/pcap/mfex_test b/tests/pcap/mfex_test
I would give the pcap file the .pcap extension just to be clear!
new file mode 100644
index
0000000000000000000000000000000000000000..1aac67b8d643ecb016c758cba4cc32212a80f52a
GIT binary patch
literal 416
zcmca|c+)~A1{MYw`2U}Qff2}Q<eHVR>K`M68ITRa|G@yFii5$Gfk6YL%z>@uY&}o|
z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$<F=a4i;9x8Q*<ZrSM6Ufz
xK>KST_NTIwYriok6N4Vm)gX-Q@<yO<!C`>c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW
literal 0
HcmV?d00001
diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
index 802895488..fcab92729 100644
--- a/tests/system-dpdk.at
+++ b/tests/system-dpdk.at
@@ -232,3 +232,49 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open
vSwitch kernel module is probably
\@EAL: No free hugepages reported in hugepages-1048576kB@d"])
AT_CLEANUP
dnl
--------------------------------------------------------------------------
+
+dnl
--------------------------------------------------------------------------
+dnl Add standard DPDK PHY port
I think we should also skip these tests if we do not have a machine that
has AVX512. Just to make sure we do not generate an OK where we are not
even testing the AVX512 functions.
+AT_SETUP([OVS-DPDK - MFEX Autovalidator])
+AT_KEYWORDS([dpdk])
+
+OVS_DPDK_START()
+
+dnl Add userspace bridge and attach it to OVS
+AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0
datapath_type=netdev])
+AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk
options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test,infinite_rx=1],
[], [stdout], [stderr])
+AT_CHECK([ovs-vsctl show], [], [stdout])
+
+
+AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator],
[0], [dnl
+Miniflow implementation set to autovalidator
+])
Here we set the autovalidator, and we stop it right after how do we know
enough packets were running through it? I thought we discussed looking
at frame count to make sure at least the pcap file was run through it
once?
+dnl Clean up
+AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
+AT_CLEANUP
+dnl
--------------------------------------------------------------------------
+
+dnl
--------------------------------------------------------------------------
+dnl Add standard DPDK PHY port
+AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy])
+AT_KEYWORDS([dpdk])
+AT_SKIP_IF([! pip3 list | grep scapy], [], [])
+AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py], [], [stdout])
+OVS_DPDK_START()
+
+dnl Add userspace bridge and attach it to OVS
+AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0
datapath_type=netdev])
+AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk
options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy,infinite_rx=1],
[], [stdout], [stderr])
+AT_CHECK([ovs-vsctl show], [], [stdout])
+
+
+AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator],
[0], [dnl
+Miniflow implementation set to autovalidator
+])
+sleep 5
See above, I do not feel like a sleep will work overtime, we might miss
packets on a slower machine.
+
+dnl Clean up
+AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
+AT_CLEANUP
+dnl
--------------------------------------------------------------------------
--
2.32.0
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
