On Wed, Aug 27, 2025 at 4:40 PM Xavier Simonart <xsimo...@redhat.com> wrote:
> Hi > > Thanks for adding this documentation! > Looks good to me, thanks. > > Acked-by: Xavier Simonart <xsimo...@redhat.com> > > Thanks > Xavier > > On Wed, Aug 13, 2025 at 10:38 PM Mark Michelson via dev < > ovs-dev@openvswitch.org> wrote: > >> Thanks Nick! >> >> Acked-by: Mark Michelson <mmich...@redhat.com> >> >> On 8/1/25 11:59 AM, Nicholas Hubbard wrote: >> > OVN is currently lacking documentation for how to develop tests. This >> commit >> > adds a file "Documentation/topics/test-development.rst" that provides >> both a >> > high-level overview of the OVN test-suite, along with documentation for >> the >> > most commonly used macros and functions. >> > >> > Reported-at: https://issues.redhat.com/browse/FDP-1548 >> > Signed-off-by: Nicholas Hubbard <nhubb...@redhat.com> >> > --- >> > v2: Implemented changes suggested by Mark and Xavier >> > --- >> > Documentation/automake.mk | 1 + >> > Documentation/topics/index.rst | 1 + >> > Documentation/topics/test-development.rst | 359 ++++++++++++++++++++++ >> > 3 files changed, 361 insertions(+) >> > create mode 100644 Documentation/topics/test-development.rst >> > >> > diff --git a/Documentation/automake.mk b/Documentation/automake.mk >> > index a70c294b9..6bc88c24b 100644 >> > --- a/Documentation/automake.mk >> > +++ b/Documentation/automake.mk >> > @@ -25,6 +25,7 @@ DOC_SOURCE = \ >> > Documentation/tutorials/ovn-interconnection.rst \ >> > Documentation/topics/index.rst \ >> > Documentation/topics/testing.rst \ >> > + Documentation/topics/test-development.rst \ >> > Documentation/topics/high-availability.rst \ >> > Documentation/topics/integration.rst \ >> > Documentation/topics/ovn-news-2.8.rst \ >> > diff --git a/Documentation/topics/index.rst >> b/Documentation/topics/index.rst >> > index 55bb919c0..896014dc3 100644 >> > --- a/Documentation/topics/index.rst >> > +++ b/Documentation/topics/index.rst >> > @@ -42,6 +42,7 @@ OVN >> > ovn-news-2.8 >> > vif-plug-providers/index >> > testing >> > + test-development >> > >> > .. list-table:: >> > >> > diff --git a/Documentation/topics/test-development.rst >> b/Documentation/topics/test-development.rst >> > new file mode 100644 >> > index 000000000..823957983 >> > --- /dev/null >> > +++ b/Documentation/topics/test-development.rst >> > @@ -0,0 +1,359 @@ >> > +.. >> > + Licensed under the Apache License, Version 2.0 (the "License"); >> you may >> > + not use this file except in compliance with the License. You may >> obtain >> > + a copy of the License at >> > + >> > + http://www.apache.org/licenses/LICENSE-2.0 >> > + >> > + Unless required by applicable law or agreed to in writing, >> software >> > + distributed under the License is distributed on an "AS IS" >> BASIS, WITHOUT >> > + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. >> See the >> > + License for the specific language governing permissions and >> limitations >> > + under the License. >> > + >> > + Convention for heading levels in OVN documentation: >> > + >> > + ======= Heading 0 (reserved for the title in a document) >> > + ------- Heading 1 >> > + ~~~~~~~ Heading 2 >> > + +++++++ Heading 3 >> > + ''''''' Heading 4 >> > + >> > + Avoid deeper levels because they do not render well. >> > + >> > +================ >> > +Test Development >> > +================ >> > + >> > +This document provides information relevant to writing tests for OVN. >> The >> > +documentation for executing tests exists in :doc:`/topics/testing`. >> > + >> > +OVN uses `Autotest < >> https://www.gnu.org/software/autoconf/manual/autoconf-2.68/html_node/Using-Autotest.html#Using-Autotest >> >`_ >> > +for generating its tests. The top-level of the test code is located in >> the >> > +file ``tests/testsuite.at``. This file is expanded into a shell >> script that >> > +runs all of the OVN tests. Please refer to the Autotest documentation >> linked >> > +above for more information regarding Autotest, as the rest of this >> document >> > +assumes a general understanding of it. >> > + >> > +OVN Test Suite Overview >> > +----------------------- >> > + >> > +All test code is located in the ``tests/`` directory at the root of >> the OVN >> > +repository. Generally, files suffixed with ``-macros.at`` contain >> macros and >> > +shell functions to aid in writing tests. Files suffixed with just >> ``.at`` >> > +contain the code for running actual tests. >> > + >> > +By convention macros are denoted with all uppercase letters, while >> functions >> > +use lowercase letters. >> > + >> > +The most used and important macros/functions are documented here. All >> of >> > +these macros/functions are implicitly available from all test files >> when >> > +running the testsuite via ``make check``. >> > + >> > +Macros and Functions >> > +-------------------- >> > + >> > +Verification >> > +~~~~~~~~~~~~ >> > + >> > +check COMMAND... >> > +++++++++++++++++ >> > + >> > +Function to run COMMAND and check that it succeeds without any output. >> Also >> > +logs the COMMAND. Note that most ``ovn-nbctl`` and ``ovn-sbctl`` must >> be run >> > +withing ``check`` so that the return status is checked. >> > + >> > +OVN_CHECK_PACKETS([PCAP], [EXPECTED]) >> > ++++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro to compare packets read from PCAP, in pcap format, to those read >> from >> > +EXPECTED, which is a text file containing packets as hex strings, one >> per line. >> > +If PCAP contains fewer packets than EXPECTED, it waits up to 30 >> seconds by >> > +default for more packets to appear. The wait time can be customized >> via the >> > +``OVS_CTL_TIMEOUT`` environment variable. >> > + >> > +The reception of any extra or duplicate packets causes the test to >> fail. >> > + >> > +OVN_CHECK_PACKETS_CONTAIN([PCAP], [EXPECTED]) >> > ++++++++++++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro to check packets read from PCAP contain data from EXPECTED. >> Waits until >> > +all expected packets are received, and ignores extra packets. >> > + >> > +OVN_CHECK_PACKETS_UNIQ([PCAP], [EXPECTED]) >> > +++++++++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro to check that all EXPECTED packets are received. Duplicate >> packets are >> > +ignored, however extra packets cause the test to fail. >> > + >> > +OVN_CHECK_PACKETS_REMOVE_BROADCAST([PCAP], [EXPECTED]) >> > +++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro that waits until the expected number of packets (excluding >> broadcasts) >> > +are received. Then it compares EXPECTED and received packets. Any (non >> > +broadcast), extra or duplicate packets cause the test to fail. >> > + >> > +check_uuid COMMAND >> > +++++++++++++++++++ >> > + >> > +Function to run COMMAND and check that it does not print anything else >> than >> > +uuid as output. It also fails if the output is empty. >> > + >> > +Daemon/Sandbox Management >> > +~~~~~~~~~~~~~~~~~~~~~~~~~ >> > + >> > +ovn_start [--backup-northd=none|paused] [AZ] >> > +++++++++++++++++++++++++++++++++++++++++++++ >> > + >> > +Creates and initializes ovn-sb and ovn-nb databases and starts their >> > +ovsdb-server instance, sets appropriate environment variables so that >> ovn-sbctl >> > +and ovn-nbctl use them by default, and starts ovn-northd running >> against them. >> > + >> > +Normally this starts only an active northd and no backup northd. The >> following >> > +options are accepted to adjust that: >> > + >> > +``--backup-northd`` Start a backup northd. >> > +``--backup-northd=paused`` Start the backup northd in the paused >> state. >> > +``--use-tcp-to-sb`` Use TCP instead of SSL to connect to sb >> database. >> > + >> > +The AZ parameter may name an arbitrary availability zone. If AZ is >> specified >> > +then the ovn-ic daemon is also started. >> > + >> > +net_add NETWORK >> > ++++++++++++++++ >> > + >> > +Function to create a new interconnection network named NETWORK. >> > + >> > +ovn_attach NETWORK BRIDGE IP [MASKLEN] [ENCAP] >> > +++++++++++++++++++++++++++++++++++++++++++++++ >> > + >> > +First, this function attaches BRIDGE to interconnection network >> NETWORK. >> > +Second, it configures (simulated) address IP (with network mask length >> MASKLEN, >> > +which defaults to 24) on BRIDGE. Finally, it configures the Open >> vSwitch >> > +database to work with OVN and starts ovn-controller. >> > + >> > +sim_add SANDBOX >> > ++++++++++++++++ >> > + >> > +Function to start a new simulated Open vSwitch instance named SANDBOX. >> Files >> > +related to the instance, such as logs, databases, sockets, and >> pidfiles, are >> > +created in a subdirectory of the main test directory also named >> > +SANDBOX. Afterward, the ``as`` command (see below) can be used to run >> Open >> > +vSwitch commands in the context of the new sandbox. >> > + >> > +The new sandbox starts out without any bridges. Use ovs-vsctl in the >> context of >> > +the new sandbox to create a bridge, e.g.:: >> > + >> > + sim_add hv0 # Create sandbox hv0. >> > + as hv0 # Set hv0 as default sandbox. >> > + ovs-vsctl add-br br0 # Add bridge br0 inside hv0. >> > + >> > +or:: >> > + >> > + sim_add hv0 >> > + as hv0 ovs-vsctl add-br br0 >> > + >> > +as [OVS_DIR] COMMAND >> > +++++++++++++++++++++ >> > + >> > +``as $1`` sets the ``OVS_*DIR`` and ``OVN_*DIR*`` environment >> variables to >> > +point to $ovs_base/$1. >> > + >> > +``as $1 COMMAND...`` sets those variables in a subshell and invokes >> COMMAND >> > +there. >> > + >> > +wait_for_ports_up [PORT...] >> > ++++++++++++++++++++++++++++ >> > + >> > +With arguments, this function waits for specified Logical_Switch_Ports >> to come >> > +up. Without arguments, waits for all "plain" and router >> Logical_Switch_Ports to >> > +come up. >> > + >> > +PARSE_LISTENING_PORT(LOGFILE, VARIABLE) >> > ++++++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro that parses the TCP or SSL/TLS port on which a server is >> listening from >> > +LOGFILE, given that the server was told to listen on a kernel-chosen >> port, and >> > +assigns the port number to shell VARIABLE. You should specify the >> listening >> > +remote as ptcp:0:127.0.0.1 or pssl:0:127.0.0.1, or the equivalent with >> [::1] >> > +instead of 127.0.0.1. Here's an example of how to use this with >> ovsdb-server:: >> > + >> > + ovsdb-server --log-file --remote=ptcp:0:127.0.0.1 ... >> > + PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT]) >> > + >> > +Now $TCP_PORT holds the listening port. >> > + >> > +OVN_POPULATE_ARP() >> > +++++++++++++++++++ >> > + >> > +Macro to pre-populate the ARP tables of all of the OVN instances that >> have been >> > +started with ```ovn_attach()``. That means that packets sent from one >> > +hypervisor to another never get dropped or delayed by ARP resolution, >> which >> > +makes testing easier. >> > + >> > +OVS_TRAFFIC_VSWITCHD_START([vsctl-args], [vsctl-output], [=override]) >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro to create a database and starts ovsdb-server, starts ovs-vswitchd >> > +connected to that database, calls ovs-vsctl to create a bridge named >> br0 with >> > +predictable settings, passing 'vsctl-args' as additional commands to >> > +ovs-vsctl. If 'vsctl-args' causes ovs-vsctl to provide output (e.g. >> because it >> > +includes "create" commands) then 'vsctl-output' specifies the expected >> output >> > +after filtering through uuidfilt. >> > + >> > +If a test needs to use "system" devices (as dummies), then specify >> > +``=override`` (literally) as the third argument. Otherwise, system >> devices >> > +won't work at all (which makes sense because tests should not access a >> system's >> > +real Ethernet devices). >> > + >> > +OVS_TRAFFIC_VSWITCHD_STOP([WHITELIST], [extra_cmds]) >> > +++++++++++++++++++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro to gracefully stops ovs-vswitchd and ovsdb-server, checking >> their log >> > +files for messages with severity WARN or higher and signaling an error >> if any >> > +is present. The optional WHITELIST may contain shell-quoted "sed" >> commands to >> > +delete any warnings that are actually expected, e.g.:: >> > + >> > + OVS_TRAFFIC_VSWITCHD_STOP(["/expected error/d"]) >> > + >> > +'extra_cmds' are shell commands to be executed afte >> OVS_VSWITCHD_STOP() is >> > +invoked. They can be used to perform additional cleanups such as name >> space >> > +removal. >> > + >> > +OVS_APP_EXIT_AND_WAIT(DAEMON) >> > ++++++++++++++++++++++++++++++ >> > + >> > +Ask the daemon named DAEMON to exit, via ``ovs-appctl``, and then wait >> for it >> > +to exit. >> > + >> > +OVN_CLEANUP(sim [, sim ...]) >> > +++++++++++++++++++++++++++++ >> > + >> > +Macro to gracefully terminate all OVN daemons, including those in >> specified >> > +sandbox instances. Also checks the log file for messages with severity >> WARN or >> > +higher and signals an error if any is present. Optional arguments may >> contain >> > +"acceptable" error messages. >> > + >> > +Before terminating the daemons, it also issues recomputes on >> ovn-controllers in >> > +listed sandboxes, and checks whether the related ports and the >> openflows before >> > +and after recompute are the same. Optional arguments may also contain >> > +acceptable ``related_ports`` differences, datapaths and tables on >> which flow >> > +differences are considered as acceptable. >> > + >> > +OVN_CLEANUP_SBOX(sbox) >> > +++++++++++++++++++++++ >> > + >> > +Macro to gracefully terminate OVN daemons in the specified sandbox >> instance. >> > +The sandbox name ``vtep`` is treated as a special case, and is assumed >> to have >> > +ovn-controller-vtep and ovs-vtep daemons running instead of >> ovn-controller. >> > + >> > +Also checks the log file for messages with severity WARN or >> > +higher and signals an error if any is present. Optional arguments may >> contain >> > +"acceptable" error messages. >> > + >> > +Before terminating the daemons, it also issues recomputes on >> ovn-controllers in >> > +listed sandboxes, and checks whether the related ports and the >> openflows before >> > +and after recompute are the same. Optional arguments may also contain >> > +acceptable ``related_ports`` differences, datapaths and tables on >> which flow >> > +differences are considered as acceptable. >> > + >> > +OVN_CLEANUP_CONTROLLER(sbox) >> > +++++++++++++++++++++++++++++ >> > + >> > +Macro to gracefully terminate ovn-controller in the specified sandbox >> > +instance. The sandbox name ``vtep`` is treated as a special case, and >> is >> > +assumed to have ovn-controller-vtep and ovs-vtep daemons running >> instead of >> > +ovn-controller. >> > + >> > +Issues recomputes on ovn-controllers in listed sandbox, and checks >> whether the >> > +related ports and the openflows before and after recompute are the >> same. >> > +Optional arguments may also contain acceptable ``related_ports`` >> differences, >> > +datapaths and tables on which flow differences are considered as >> acceptable. >> > + >> > +OVN_CLEANUP_IC([az ...]) >> > +++++++++++++++++++++++++ >> > + >> > +Macro to gracefully terminate all interconnection DBs and daemons in >> the >> > +specified AZs, if any. >> > + >> > +Test Management >> > +~~~~~~~~~~~~~~~ >> > + >> > +OVN_FOR_EACH_NORTHD(TEST) >> > ++++++++++++++++++++++++++ >> > + >> > +Macro to wrap an arbitrary TEST. Defines versions of the TEST with all >> > +combinations of northd, parallelization enabled and conditional >> monitoring >> > +on/off. Normally the first statement in TEST is a call to ``AT_SETUP``. >> > + >> > +OVN_FOR_EACH_NORTHD_NO_HV(TEST) >> > ++++++++++++++++++++++++++++++++ >> > + >> > +Macro to wrap an arbitrary TEST. Defines versions of the TEST with all >> > +combinations of northd and parallelization enabled. To be used when the >> > +ovn-controller configuration is not relevant. Normally the first >> statement in >> > +TEST is a call to ``AT_SETUP``. >> > + >> > +on_exit COMMAND >> > ++++++++++++++++ >> > + >> > +Function to add the shell COMMAND to a collection that is executed >> when the >> > +current test completes, as a cleanup action. The most common use is to >> kill a >> > +daemon started by the test. This is important to prevent tests that >> start >> > +daemons from hanging at exit. >> > + >> > +Cleanup commands are executed in the reverse order of calls to this >> function. >> > + >> > +OVN_NBCTL(NBCTL_COMMAND) >> > +++++++++++++++++++++++++ >> > + >> > +Macro to add NBCTL_COMMAND to list of commands to be run by the >> > +``RUN_OVN_NBCTL`` macro. >> > + >> > +RUN_OVN_NBCTL() >> > ++++++++++++++++ >> > + >> > +Macro to execute a list of commands built by the ``OVN_NBCTL`` macro. >> The list >> > +of commands is executed in a single invocation of ``ovn-nbctl`` >> > + >> > +OVS_VSCTL(VSCTL_COMMAND) >> > +++++++++++++++++++++++++ >> > + >> > +Macro to add VSCTL_COMMAND to list of commands to be run by >> ``RUN_OVS_VSCTL``. >> > + >> > +RUN_OVS_VSCTL() >> > ++++++++++++++++ >> > + >> > +Macro to execute the list of commands built by the ``OVS_VSCTL`` >> macro. The >> > +list of commands is executed in a single invocation of ``ovs-vsctl``. >> > + >> > +OVS_WAIT_FOR_OUTPUT(COMMAND, EXIT-STATUS, STDOUT, STDERR) >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> > + >> > +Executes shell COMMAND in a loop until it exits with status >> EXIT-STATUS, prints >> > +STDOUT on stdout, and prints STDERR on stderr. If this doesn't happen >> within a >> > +reasonable time limit, then the test fails. >> > + >> > +There is an ``OVS_WAIT_FOR_OUTPUT_UNQUOTED`` version of this macro >> that expands >> > +shell ``$variables``, ``$(command)``, and so on. The plain version >> does not >> > + >> > +OVS_WAIT_UNTIL(COMMAND[, IF-FAILED]) >> > +++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro that executes shell COMMAND in a loop until it returns zero >> return code. >> > +If COMMAND does not return zero code within reasonable time limit, >> then the >> > +test fails. In that case, runs IF-FAILED before exiting. >> > + >> > +OVS_WAIT_WHILE(COMMAND[, IF-FAILED]) >> > +++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro that executes shell COMMAND in a loop until it returns non-zero >> return >> > +code. If COMMAND does not return non-zero code within reasonable time >> limit, >> > +then the test fails. In that case, runs IF-FAILED before exiting. >> > + >> > +OVS_WAIT_UNTIL_EQUAL(COMMAND, OUTPUT) >> > ++++++++++++++++++++++++++++++++++++++ >> > + >> > +Macro that executes shell COMMAND in a loop until it returns zero and >> the >> > +output equals OUTPUT. If COMMAND does not return zero or a desired >> output >> > +within a reasonable time limit, fails the test. >> >> _______________________________________________ >> dev mailing list >> d...@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> >> Thank you Nick, Mark and Xavier, I went ahead and merged this into main. Regards, Ales _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
