Thank you Ihar and Ales.

I pushed the change to main and all branches back to 22.03.

The speedup is amazing even with the daemon restarting for each test. Out of curiosity, I tried to determine how we could run the scapy daemon globally for the entire testsuite run. In my (very brief) look at autotest and our makefiles, the only thing that made sense to me was to update the "check-local" Makefile target in tests/automake.mk to invoke something that can start/stop the scapy daemon. I have no idea if this is the "proper" way to do it though.

On 9/15/23 03:36, Ales Musil wrote:
On Thu, Sep 14, 2023 at 9:56 PM Ihar Hrachyshka <[email protected]> wrote:

The daemon life cycle spans over the whole test case life time, which
significantly speeds up test cases that rely on fmt_pkt for packet byte
representations.

The speed-up comes from the fact that we no longer start a python
environment with all scapy modules imported on any fmt_pkt invocation;
but only on the first call to fmt_pkt.

The daemon is not started for test cases that don't trigger fmt_pkt.
(The server is started lazily as part of fmt_pkt call.)

For example, without the daemon, all tests that use fmt_pkt took the
following on my vagrant box:

real    17m23.092s
user    26m27.935s
sys     5m25.486s

With the daemon, the same set of tests run in:

real    2m16.741s
user    2m40.155s
sys     0m47.514s

We may want to make the daemon global, so that it's invoked once per
test suite run. But I haven't figured out, yet, how to run a trap to
clean up the deamon and its socket and pid files on suite exit (and not
on test case exit.)

Signed-off-by: Ihar Hrachyshka <[email protected]>

unixctl impl
---
  tests/automake.mk   |  3 +-
  tests/ovn-macros.at | 16 ++++++++---
  tests/scapy-server  | 69 +++++++++++++++++++++++++++++++++++++++++++++
  3 files changed, 83 insertions(+), 5 deletions(-)
  create mode 100755 tests/scapy-server

diff --git a/tests/automake.mk b/tests/automake.mk
index eea0d00f4..f23ec353e 100644
--- a/tests/automake.mk
+++ b/tests/automake.mk
@@ -310,7 +310,8 @@ CHECK_PYFILES = \
         tests/test-l7.py \
         tests/uuidfilt.py \
         tests/test-tcp-rst.py \
-       tests/check_acl_log.py
+       tests/check_acl_log.py \
+       tests/scapy-server

  EXTRA_DIST += $(CHECK_PYFILES)
  PYCOV_CLEAN_FILES += $(CHECK_PYFILES:.py=.py,cover) .coverage
diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at
index 13d5dc3d4..b1b2a8156 100644
--- a/tests/ovn-macros.at
+++ b/tests/ovn-macros.at
@@ -833,10 +833,18 @@ ovn_trace_client() {
  # ovs-appctl netdev-dummy/receive $vif $packet
  #
  fmt_pkt() {
-    echo "from scapy.all import *; \
-          import binascii; \
-          out = binascii.hexlify(raw($1)); \
-          print(out.decode())" | $PYTHON3
+    ctlfile=$ovs_base/scapy.ctl
+    if [[ ! -e $ctlfile ]]; then
+        start_scapy_server
+    fi
+    echo $(ovs-appctl -t $ctlfile payload "$1")
+}
+
+start_scapy_server() {
+    pidfile=$ovs_base/scapy.pid
+    ctlfile=$ovs_base/scapy.ctl
+    "$top_srcdir"/tests/scapy-server --pidfile=$pidfile
--unixctl=$ctlfile --detach
+    on_exit "test -e \"$pidfile\" && ovs-appctl -t $ctlfile exit"
  }

  sleep_sb() {
diff --git a/tests/scapy-server b/tests/scapy-server
new file mode 100755
index 000000000..a7255c84d
--- /dev/null
+++ b/tests/scapy-server
@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+
+import argparse
+
+import ovs.daemon
+import ovs.unixctl
+import ovs.unixctl.server
+
+import binascii
+from scapy.all import *  # noqa: F401,F403
+from scapy.all import raw
+
+
+vlog = ovs.vlog.Vlog("scapy-server")
+exiting = False
+
+
+def exit(conn, argv, aux):
+    global exiting
+
+    exiting = True
+    conn.reply(None)
+
+
+def process(data):
+    try:
+        data = data.replace('\n', '')
+        return binascii.hexlify(raw(eval(data))).decode()
+    except Exception:
+        return ""
+
+
+def payload(conn, argv, aux):
+    conn.reply(process(argv[0]))
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Scapy-based Frame Payload Generator")
+    parser.add_argument("--unixctl", help="UNIXCTL socket location or
'none'.")
+
+    ovs.daemon.add_args(parser)
+    ovs.vlog.add_args(parser)
+    args = parser.parse_args()
+    ovs.daemon.handle_args(args)
+    ovs.vlog.handle_args(args)
+
+    ovs.daemon.daemonize_start()
+    error, server = ovs.unixctl.server.UnixctlServer.create(args.unixctl)
+    if error:
+        ovs.util.ovs_fatal(error, "could not create unixctl server at %s"
+                           % args.unixctl, vlog)
+
+    ovs.unixctl.command_register("exit", "", 0, 0, exit, None)
+    ovs.unixctl.command_register("payload", "", 1, 1, payload, None)
+    ovs.daemon.daemonize_complete()
+
+    poller = ovs.poller.Poller()
+    while not exiting:
+        server.run()
+        server.wait(poller)
+        if exiting:
+            poller.immediate_wake()
+        poller.block()
+    server.close()
+
+
+if __name__ == '__main__':
+    main()
--
2.38.1

_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev



Looks good to me, thanks!

Acked-by: Ales Musil <[email protected]>


_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Reply via email to