Re: [ovs-dev] [PATCH v2 2/3] system-dpdk: Use dummy-pmd port for packet injection.
On Wed, Nov 24, 2021 at 5:08 PM Eelco Chaudron wrote:
>
> On 23 Nov 2021, at 15:15, David Marchand wrote:
>
> > net_pcap is not always available in DPDK (like, in a dev
> > environment when you forgot to install the libpcap-devel).
> > On the other hand, OVS already has its own way to inject packets into a
> > bridge. Let's make use of it.
> >
> > This solution is slower than net/pcap DPDK, so lower the number of
> > expected packets so that it runs in OVS_CTL_TIMEOUT seconds.
> >
> > While at it, convert "known" packets from pcap to scapy so that the
> > injected packets can be updated without having to read/write a pcap file.
> >
> > Note: this change also (avoids/) fixes a python exception in PcapWriter
> > with scapy 2.4.3 that comes from EPEL.
> >
> > Suggested-by: Ilya Maximets
> > Signed-off-by: David Marchand
>
> One small nit below, rest looks good!
Thanks for the review.
I did another pass.
I see issues in the documentation that still describes use of pcap pmd.
And some more nits below.
>
> > ---
> > Changes since v1:
> > - renamed generator script,
> > - decreased packet count for fuzzy test,
> > - simplified wait expression for packet count,
> >
> > ---
> > tests/automake.mk | 5 ++--
> > tests/genpkts.py| 56
> > tests/mfex_fuzzy.py | 32 -
> > tests/pcap/mfex_test.pcap | Bin 416 -> 0 bytes
> > tests/system-dpdk-macros.at | 4 +--
> > tests/system-dpdk.at| 30 +++
> > 6 files changed, 84 insertions(+), 43 deletions(-)
> > create mode 100755 tests/genpkts.py
> > delete mode 100755 tests/mfex_fuzzy.py
> > delete mode 100644 tests/pcap/mfex_test.pcap
> >
> > diff --git a/tests/automake.mk b/tests/automake.mk
> > index 43731d0973..c519a5cb36 100644
> > --- a/tests/automake.mk
> > +++ b/tests/automake.mk
> > @@ -145,8 +145,7 @@ $(srcdir)/tests/fuzz-regression-list.at:
> > tests/automake.mk
> >
> > EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS)
> > MFEX_AUTOVALIDATOR_TESTS = \
> > - tests/pcap/mfex_test.pcap \
> > - tests/mfex_fuzzy.py
> > + tests/genpkts.py
With the pcap file now dropped, this block is useless and can be dropped.
genpkts.py is already added to EXTRA_DIST via CHECK_PYFILES.
> >
> > OVSDB_CLUSTER_TESTSUITE_AT = \
> > tests/ovsdb-cluster-testsuite.at \
> > @@ -518,7 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type-props.c
> > CHECK_PYFILES = \
> > tests/appctl.py \
> > tests/flowgen.py \
> > - tests/mfex_fuzzy.py \
> > + tests/genpkts.py \
> > tests/ovsdb-monitor-sort.py \
> > tests/test-daemon.py \
> > tests/test-json.py \
> > diff --git a/tests/genpkts.py b/tests/genpkts.py
> > new file mode 100755
> > index 00..e243bb960f
> > --- /dev/null
> > +++ b/tests/genpkts.py
> > @@ -0,0 +1,56 @@
> > +#!/usr/bin/python3
>
> Maybe change this to “#!/usr/bin/env python3” as other scripts have.
"Not my fault", but ok, I can change it :-).
I think this file is also missing a copyright banner from Intel.
Not sure it is important, but this is not my place to touch this.
>
> > +
> > +import sys
> > +
> > +from scapy.all import RandMAC, RandIP, RandIP6, RandShort, fuzz
> > +from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP
> > +
> > +if len(sys.argv) < 2:
> > +print('usage: {} packets_count [fuzz]'.format(sys.argv[0]))
> > +sys.exit(1)
> > +
> > +tmpl = []
> > +
> > +if len(sys.argv) == 2:
> > +eth = Ether(dst='ff:ff:ff:ff:ff:ff')
> > +vlan = eth / Dot1Q(vlan=1)
> > +p = eth / IP() / TCP(sport=20, dport=80, flags='SA', window=8192)
> > +tmpl += [p.build().hex()]
> > +p = eth / IP() / UDP(sport=53, dport=53)
> > +tmpl += [p.build().hex()]
> > +p = eth / IP() / TCP(sport=20, dport=80, flags='S', window=8192)
> > +tmpl += [p.build().hex()]
> > +p = eth / IP() / UDP(sport=53, dport=53)
> > +tmpl += [p.build().hex()]
> > +p = vlan / IP() / UDP(sport=53, dport=53)
> > +tmpl += [p.build().hex()]
> > +p = vlan / IP() / TCP(sport=20, dport=80, flags='S', window=8192)
> > +tmpl += [p.build().hex()]
> > +elif sys.argv[2] == 'fuzz':
> > +# Generate random protocol bases, use a fuzz() over the combined packet
> > +# for full fuzzing.
> > +eth = Ether(src=RandMAC(), dst=RandMAC())
> > +vlan = Dot1Q()
> > +ipv4 = IP(src=RandIP(), dst=RandIP())
> > +ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
> > +udp = UDP(dport=RandShort(), sport=RandShort())
> > +tcp = TCP(dport=RandShort(), sport=RandShort())
> > +
> > +# IPv4 packets with fuzzing
> > +tmpl += [fuzz(eth / ipv4 / udp).build().hex()]
> > +tmpl += [fuzz(eth / ipv4 / tcp).build().hex()]
> > +tmpl += [fuzz(eth / vlan / ipv4 / udp).build().hex()]
> > +tmpl += [fuzz(eth / vlan / ipv4 / tcp).build().hex()]
> > +
> > +# IPv6 packets with fuzzing
> > +tmpl += [fuzz(eth / ipv6 / udp).build().hex()]
> > +tmpl += [fuzz(eth / ipv6 /
Re: [ovs-dev] [PATCH v2 2/3] system-dpdk: Use dummy-pmd port for packet injection.
On 23 Nov 2021, at 15:15, David Marchand wrote:
> net_pcap is not always available in DPDK (like, in a dev
> environment when you forgot to install the libpcap-devel).
> On the other hand, OVS already has its own way to inject packets into a
> bridge. Let's make use of it.
>
> This solution is slower than net/pcap DPDK, so lower the number of
> expected packets so that it runs in OVS_CTL_TIMEOUT seconds.
>
> While at it, convert "known" packets from pcap to scapy so that the
> injected packets can be updated without having to read/write a pcap file.
>
> Note: this change also (avoids/) fixes a python exception in PcapWriter
> with scapy 2.4.3 that comes from EPEL.
>
> Suggested-by: Ilya Maximets
> Signed-off-by: David Marchand
One small nit below, rest looks good!
> ---
> Changes since v1:
> - renamed generator script,
> - decreased packet count for fuzzy test,
> - simplified wait expression for packet count,
>
> ---
> tests/automake.mk | 5 ++--
> tests/genpkts.py| 56
> tests/mfex_fuzzy.py | 32 -
> tests/pcap/mfex_test.pcap | Bin 416 -> 0 bytes
> tests/system-dpdk-macros.at | 4 +--
> tests/system-dpdk.at| 30 +++
> 6 files changed, 84 insertions(+), 43 deletions(-)
> create mode 100755 tests/genpkts.py
> delete mode 100755 tests/mfex_fuzzy.py
> delete mode 100644 tests/pcap/mfex_test.pcap
>
> diff --git a/tests/automake.mk b/tests/automake.mk
> index 43731d0973..c519a5cb36 100644
> --- a/tests/automake.mk
> +++ b/tests/automake.mk
> @@ -145,8 +145,7 @@ $(srcdir)/tests/fuzz-regression-list.at: tests/automake.mk
>
> EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS)
> MFEX_AUTOVALIDATOR_TESTS = \
> - tests/pcap/mfex_test.pcap \
> - tests/mfex_fuzzy.py
> + tests/genpkts.py
>
> OVSDB_CLUSTER_TESTSUITE_AT = \
> tests/ovsdb-cluster-testsuite.at \
> @@ -518,7 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type-props.c
> CHECK_PYFILES = \
> tests/appctl.py \
> tests/flowgen.py \
> - tests/mfex_fuzzy.py \
> + tests/genpkts.py \
> tests/ovsdb-monitor-sort.py \
> tests/test-daemon.py \
> tests/test-json.py \
> diff --git a/tests/genpkts.py b/tests/genpkts.py
> new file mode 100755
> index 00..e243bb960f
> --- /dev/null
> +++ b/tests/genpkts.py
> @@ -0,0 +1,56 @@
> +#!/usr/bin/python3
Maybe change this to “#!/usr/bin/env python3” as other scripts have.
> +
> +import sys
> +
> +from scapy.all import RandMAC, RandIP, RandIP6, RandShort, fuzz
> +from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP
> +
> +if len(sys.argv) < 2:
> +print('usage: {} packets_count [fuzz]'.format(sys.argv[0]))
> +sys.exit(1)
> +
> +tmpl = []
> +
> +if len(sys.argv) == 2:
> +eth = Ether(dst='ff:ff:ff:ff:ff:ff')
> +vlan = eth / Dot1Q(vlan=1)
> +p = eth / IP() / TCP(sport=20, dport=80, flags='SA', window=8192)
> +tmpl += [p.build().hex()]
> +p = eth / IP() / UDP(sport=53, dport=53)
> +tmpl += [p.build().hex()]
> +p = eth / IP() / TCP(sport=20, dport=80, flags='S', window=8192)
> +tmpl += [p.build().hex()]
> +p = eth / IP() / UDP(sport=53, dport=53)
> +tmpl += [p.build().hex()]
> +p = vlan / IP() / UDP(sport=53, dport=53)
> +tmpl += [p.build().hex()]
> +p = vlan / IP() / TCP(sport=20, dport=80, flags='S', window=8192)
> +tmpl += [p.build().hex()]
> +elif sys.argv[2] == 'fuzz':
> +# Generate random protocol bases, use a fuzz() over the combined packet
> +# for full fuzzing.
> +eth = Ether(src=RandMAC(), dst=RandMAC())
> +vlan = Dot1Q()
> +ipv4 = IP(src=RandIP(), dst=RandIP())
> +ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
> +udp = UDP(dport=RandShort(), sport=RandShort())
> +tcp = TCP(dport=RandShort(), sport=RandShort())
> +
> +# IPv4 packets with fuzzing
> +tmpl += [fuzz(eth / ipv4 / udp).build().hex()]
> +tmpl += [fuzz(eth / ipv4 / tcp).build().hex()]
> +tmpl += [fuzz(eth / vlan / ipv4 / udp).build().hex()]
> +tmpl += [fuzz(eth / vlan / ipv4 / tcp).build().hex()]
> +
> +# IPv6 packets with fuzzing
> +tmpl += [fuzz(eth / ipv6 / udp).build().hex()]
> +tmpl += [fuzz(eth / ipv6 / tcp).build().hex()]
> +tmpl += [fuzz(eth / vlan / ipv6 / udp).build().hex()]
> +tmpl += [fuzz(eth / vlan / ipv6 / tcp).build().hex()]
> +
> +i = 0
> +count = int(sys.argv[1])
> +while count != 0:
> +print(tmpl[i % len(tmpl)])
> +i += 1
> +count -= 1
> diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py
> deleted file mode 100755
> index 3efe1152da..00
> --- a/tests/mfex_fuzzy.py
> +++ /dev/null
> @@ -1,32 +0,0 @@
> -#!/usr/bin/python3
> -
> -import sys
> -
> -from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, fuzz
> -from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP
> -
> -path = str(sys.argv[1]) + "/pcap/fuzzy.pcap"
> -pktdump = PcapWriter(path, append=False, sync=True)
> -
> -for i in range(0,
[ovs-dev] [PATCH v2 2/3] system-dpdk: Use dummy-pmd port for packet injection.
net_pcap is not always available in DPDK (like, in a dev
environment when you forgot to install the libpcap-devel).
On the other hand, OVS already has its own way to inject packets into a
bridge. Let's make use of it.
This solution is slower than net/pcap DPDK, so lower the number of
expected packets so that it runs in OVS_CTL_TIMEOUT seconds.
While at it, convert "known" packets from pcap to scapy so that the
injected packets can be updated without having to read/write a pcap file.
Note: this change also (avoids/) fixes a python exception in PcapWriter
with scapy 2.4.3 that comes from EPEL.
Suggested-by: Ilya Maximets
Signed-off-by: David Marchand
---
Changes since v1:
- renamed generator script,
- decreased packet count for fuzzy test,
- simplified wait expression for packet count,
---
tests/automake.mk | 5 ++--
tests/genpkts.py| 56
tests/mfex_fuzzy.py | 32 -
tests/pcap/mfex_test.pcap | Bin 416 -> 0 bytes
tests/system-dpdk-macros.at | 4 +--
tests/system-dpdk.at| 30 +++
6 files changed, 84 insertions(+), 43 deletions(-)
create mode 100755 tests/genpkts.py
delete mode 100755 tests/mfex_fuzzy.py
delete mode 100644 tests/pcap/mfex_test.pcap
diff --git a/tests/automake.mk b/tests/automake.mk
index 43731d0973..c519a5cb36 100644
--- a/tests/automake.mk
+++ b/tests/automake.mk
@@ -145,8 +145,7 @@ $(srcdir)/tests/fuzz-regression-list.at: tests/automake.mk
EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS)
MFEX_AUTOVALIDATOR_TESTS = \
- tests/pcap/mfex_test.pcap \
- tests/mfex_fuzzy.py
+ tests/genpkts.py
OVSDB_CLUSTER_TESTSUITE_AT = \
tests/ovsdb-cluster-testsuite.at \
@@ -518,7 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type-props.c
CHECK_PYFILES = \
tests/appctl.py \
tests/flowgen.py \
- tests/mfex_fuzzy.py \
+ tests/genpkts.py \
tests/ovsdb-monitor-sort.py \
tests/test-daemon.py \
tests/test-json.py \
diff --git a/tests/genpkts.py b/tests/genpkts.py
new file mode 100755
index 00..e243bb960f
--- /dev/null
+++ b/tests/genpkts.py
@@ -0,0 +1,56 @@
+#!/usr/bin/python3
+
+import sys
+
+from scapy.all import RandMAC, RandIP, RandIP6, RandShort, fuzz
+from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP
+
+if len(sys.argv) < 2:
+print('usage: {} packets_count [fuzz]'.format(sys.argv[0]))
+sys.exit(1)
+
+tmpl = []
+
+if len(sys.argv) == 2:
+eth = Ether(dst='ff:ff:ff:ff:ff:ff')
+vlan = eth / Dot1Q(vlan=1)
+p = eth / IP() / TCP(sport=20, dport=80, flags='SA', window=8192)
+tmpl += [p.build().hex()]
+p = eth / IP() / UDP(sport=53, dport=53)
+tmpl += [p.build().hex()]
+p = eth / IP() / TCP(sport=20, dport=80, flags='S', window=8192)
+tmpl += [p.build().hex()]
+p = eth / IP() / UDP(sport=53, dport=53)
+tmpl += [p.build().hex()]
+p = vlan / IP() / UDP(sport=53, dport=53)
+tmpl += [p.build().hex()]
+p = vlan / IP() / TCP(sport=20, dport=80, flags='S', window=8192)
+tmpl += [p.build().hex()]
+elif sys.argv[2] == 'fuzz':
+# Generate random protocol bases, use a fuzz() over the combined packet
+# for full fuzzing.
+eth = Ether(src=RandMAC(), dst=RandMAC())
+vlan = Dot1Q()
+ipv4 = IP(src=RandIP(), dst=RandIP())
+ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
+udp = UDP(dport=RandShort(), sport=RandShort())
+tcp = TCP(dport=RandShort(), sport=RandShort())
+
+# IPv4 packets with fuzzing
+tmpl += [fuzz(eth / ipv4 / udp).build().hex()]
+tmpl += [fuzz(eth / ipv4 / tcp).build().hex()]
+tmpl += [fuzz(eth / vlan / ipv4 / udp).build().hex()]
+tmpl += [fuzz(eth / vlan / ipv4 / tcp).build().hex()]
+
+# IPv6 packets with fuzzing
+tmpl += [fuzz(eth / ipv6 / udp).build().hex()]
+tmpl += [fuzz(eth / ipv6 / tcp).build().hex()]
+tmpl += [fuzz(eth / vlan / ipv6 / udp).build().hex()]
+tmpl += [fuzz(eth / vlan / ipv6 / tcp).build().hex()]
+
+i = 0
+count = int(sys.argv[1])
+while count != 0:
+print(tmpl[i % len(tmpl)])
+i += 1
+count -= 1
diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py
deleted file mode 100755
index 3efe1152da..00
--- a/tests/mfex_fuzzy.py
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/usr/bin/python3
-
-import sys
-
-from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, fuzz
-from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP
-
-path = str(sys.argv[1]) + "/pcap/fuzzy.pcap"
-pktdump = PcapWriter(path, append=False, sync=True)
-
-for i in range(0, 2000):
-
-# Generate random protocol bases, use a fuzz() over the combined packet
-# for full fuzzing.
-eth = Ether(src=RandMAC(), dst=RandMAC())
-vlan = Dot1Q()
-ipv4 = IP(src=RandIP(), dst=RandIP())
-ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
-udp = UDP(dport=RandShort(), sport=RandShort())
-tcp = TCP(dport=RandShort(), sport=RandShort())
-
-# IPv4 packets
