The test failure is only on one of the tests and not related to this patch, so looks like a fluke / flaky test.
Other than that, I accidentally excluded calls to `update-alternatives` required to run the DPDK test as I mistakenly thought that was not relevant for the upstream package source, but they indeed are. (This is what happens when returning to code > 2 weeks later ;) ) New iteration on the way soon. -- Frode Nordahl On Mon, Jun 19, 2023 at 8:33 AM Frode Nordahl <[email protected]> wrote: > > The autopkgtests [0][1] are relevant in an upstream context > because an Open vSwitch contributor may want to have a quick > way of running the upstream system testsuites on recent > Debian/Ubuntu releases in an automated and contained manner. > > During the Debian/Ubuntu/upstream package source sync work [2], a > relatively naive autopkgtest was added. It had been around since > Open vSwitch was initially packaged many years ago. > > Replace the autopkgtest with a test that runs all the upstream > system testsuites instead. > > To run the autopkgtest, take a look at [1] for prerequisites then: > > ./boot.sh && ./configure && make debian-source > autopkgtest \ > --test-name kernel \ > --env DEB_BUILD_OPTIONS="nocheck parallel=32 nodpdk" \ > ../openvswitch_3.1.90-1.dsc \ > -- qemu \ > --cpus 32 \ > --ram-size 8129 \ > autopkgtest-mantic-amd64.img > > 0: https://wiki.debian.org/ContinuousIntegration/autopkgtest > 1: https://packaging.ubuntu.com/html/auto-pkg-test.html > 2: https://mail.openvswitch.org/pipermail/ovs-dev/2022-July/396219.html > > Signed-off-by: Frode Nordahl <[email protected]> > --- > debian/automake.mk | 26 ++++- > debian/rules | 8 ++ > debian/tests/afxdp | 1 + > debian/tests/afxdp-skip-tests.txt | 2 + > debian/tests/control | 38 +++++-- > debian/tests/dpdk | 46 +------- > debian/tests/kernel | 1 + > debian/tests/offloads | 1 + > debian/tests/offloads-skip-tests.txt | 2 + > debian/tests/openflow.py | 66 ------------ > debian/tests/run-tests.sh | 151 +++++++++++++++++++++++++++ > debian/tests/system-userspace | 1 + > debian/tests/testlist.py | 74 +++++++++++++ > debian/tests/vanilla | 29 ----- > 14 files changed, 296 insertions(+), 150 deletions(-) > create mode 120000 debian/tests/afxdp > create mode 100644 debian/tests/afxdp-skip-tests.txt > mode change 100755 => 120000 debian/tests/dpdk > create mode 120000 debian/tests/kernel > create mode 120000 debian/tests/offloads > create mode 100644 debian/tests/offloads-skip-tests.txt > delete mode 100755 debian/tests/openflow.py > create mode 100755 debian/tests/run-tests.sh > create mode 120000 debian/tests/system-userspace > create mode 100755 debian/tests/testlist.py > delete mode 100755 debian/tests/vanilla > > diff --git a/debian/automake.mk b/debian/automake.mk > index 7b2afafae..89341fccc 100644 > --- a/debian/automake.mk > +++ b/debian/automake.mk > @@ -61,10 +61,16 @@ EXTRA_DIST += \ > debian/rules \ > debian/source/format \ > debian/source/lintian-overrides \ > + debian/tests/afxdp \ > + debian/tests/afxdp-skip-tests.txt \ > debian/tests/control \ > debian/tests/dpdk \ > - debian/tests/openflow.py \ > - debian/tests/vanilla \ > + debian/tests/kernel \ > + debian/tests/offloads \ > + debian/tests/offloads-skip-tests.txt \ > + debian/tests/run-tests.sh \ > + debian/tests/system-userspace \ > + debian/tests/testlist.py \ > debian/watch > > check-debian-changelog-version: > @@ -113,7 +119,6 @@ CLEANFILES += debian/control > debian: debian/copyright debian/control > .PHONY: debian > > - > debian-deb: debian > @if test X"$(srcdir)" != X"$(top_builddir)"; then > \ > echo "Debian packages should be built from $(abs_srcdir)/"; > \ > @@ -130,3 +135,18 @@ else > $(AM_V_GEN) DEB_BUILD_OPTIONS="nocheck parallel=`nproc` nodpdk" \ > fakeroot debian/rules binary > endif > + > +debian-source: debian > + @if test X"$(srcdir)" != X"$(top_builddir)"; then > \ > + echo "Debian packages should be built from $(abs_srcdir)/"; > \ > + exit 1; > \ > + fi > + $(MAKE) distclean > + $(update_deb_copyright) > + $(update_deb_control) > + $(AM_V_GEN) fakeroot debian/rules clean > + tar -cvzf ../openvswitch_$(PACKAGE_VERSION).orig.tar.gz \ > + --exclude .git \ > + --transform 's,^\.,openvswitch-$(PACKAGE_VERSION),' \ > + . > + dpkg-source -b . > diff --git a/debian/rules b/debian/rules > index 28c249d07..f4c69651e 100755 > --- a/debian/rules > +++ b/debian/rules > @@ -91,6 +91,14 @@ endif > > execute_before_dh_auto_clean: > find . -name "*.pyc" -delete > + if [ -f debian/control ]; then \ > + cp debian/control debian/control.save; \ > + fi > + > +execute_after_dh_auto_clean: > + if [ -f debian/control.save ]; then \ > + mv debian/control.save debian/control; \ > + fi > > override_dh_auto_install: > dh_auto_install --sourcedirectory=_debian > diff --git a/debian/tests/afxdp b/debian/tests/afxdp > new file mode 120000 > index 000000000..7080a3249 > --- /dev/null > +++ b/debian/tests/afxdp > @@ -0,0 +1 @@ > +run-tests.sh > \ No newline at end of file > diff --git a/debian/tests/afxdp-skip-tests.txt > b/debian/tests/afxdp-skip-tests.txt > new file mode 100644 > index 000000000..9140b2d52 > --- /dev/null > +++ b/debian/tests/afxdp-skip-tests.txt > @@ -0,0 +1,2 @@ > +# LP: #2020679 > +ICMP from different source related with NAT > diff --git a/debian/tests/control b/debian/tests/control > index b481ed53c..29a65fb12 100644 > --- a/debian/tests/control > +++ b/debian/tests/control > @@ -1,8 +1,32 @@ > -Tests: vanilla dpdk > +Tests: afxdp, kernel, offloads, system-userspace > Depends: > - iperf, > - mininet (>= 2.2.0~), > - openvswitch-switch, > - openvswitch-switch-dpdk [amd64 i386], > - openvswitch-testcontroller, > -Restrictions: needs-root rw-build-tree isolation-machine > + @, > + @builddeps@, > + conntrack, > + net-tools, > + netcat-openbsd, > + tcpdump, > + python3-pyftpdlib, > +Restrictions: > + allow-stderr, > + isolation-machine, > + needs-root, > + rw-build-tree, > + > +# The dpdk tests are marked `skippable`, because they have environment > specific > +# requirements which may or may not be available at runtime (hugepages > setup). > +Tests: dpdk > +Depends: > + @, > + @builddeps@, > + conntrack, > + dpdk-dev [amd64 i386 ppc64el arm64 riscv64], > + net-tools, > + netcat-openbsd, > + tcpdump, > + python3-pyftpdlib, > +Restrictions: > + allow-stderr, > + isolation-machine, > + needs-root, > + skippable, > diff --git a/debian/tests/dpdk b/debian/tests/dpdk > deleted file mode 100755 > index 2aabae050..000000000 > --- a/debian/tests/dpdk > +++ /dev/null > @@ -1,45 +0,0 @@ > -#!/bin/sh > - > -set -e > - > -if [ ! -x /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk ]; then > - echo "DPDK enabled binary not detected - skipping" > - exit 0 > -fi > - > -sse3flag=$(sed -n "/^flags.*sse3/p" < /proc/cpuinfo | wc -l) > -if [ "${sse3flag}" -eq 0 ]; then > - echo "sse3 not available in test environment" > - echo "for adt-virt-qemu please consider adding --qemu-options='-cpu > qemu64,+ssse3'" > - echo "SKIPPING" > - exit 0 > -fi > - > -update-alternatives --set ovs-vswitchd \ > - /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk > -service openvswitch-switch restart > - > -modprobe openvswitch || true > - > -echo "kernel modules loaded: " > -# Check that ovs loaded > -lsmod | grep "openvswitch" > -echo "OK" > - > -echo "Checking daemons: " > -pgrep ovs-vswitchd > -pgrep ovsdb-server > -echo "OK" > - > -echo "stop conflicting openvswitch testcontroller" > -systemctl stop openvswitch-testcontroller || true > - > -if dpkg --compare-versions "$(dpkg-query --showformat '${Version}\n' --show > mininet)" ge "2.3.0-1"; then > - PYCMD="python3" > -else > - PYCMD="python2" > -fi > - > -printf "running openflow tests using mininet" > -${PYCMD} `dirname $0`/openflow.py 2>&1 > -echo "OK" > diff --git a/debian/tests/dpdk b/debian/tests/dpdk > new file mode 120000 > index 000000000..7080a3249 > --- /dev/null > +++ b/debian/tests/dpdk > @@ -0,0 +1 @@ > +run-tests.sh > \ No newline at end of file > diff --git a/debian/tests/kernel b/debian/tests/kernel > new file mode 120000 > index 000000000..7080a3249 > --- /dev/null > +++ b/debian/tests/kernel > @@ -0,0 +1 @@ > +run-tests.sh > \ No newline at end of file > diff --git a/debian/tests/offloads b/debian/tests/offloads > new file mode 120000 > index 000000000..7080a3249 > --- /dev/null > +++ b/debian/tests/offloads > @@ -0,0 +1 @@ > +run-tests.sh > \ No newline at end of file > diff --git a/debian/tests/offloads-skip-tests.txt > b/debian/tests/offloads-skip-tests.txt > new file mode 100644 > index 000000000..91a4a5591 > --- /dev/null > +++ b/debian/tests/offloads-skip-tests.txt > @@ -0,0 +1,2 @@ > +# LP: #2020677 > +check interface meter offloading > diff --git a/debian/tests/openflow.py b/debian/tests/openflow.py > deleted file mode 100755 > index 216b57cf4..000000000 > --- a/debian/tests/openflow.py > +++ /dev/null > @@ -1,66 +0,0 @@ > -import unittest > -import logging > -from mininet.net import Mininet > -import mininet.log as log > -from mininet.node import OVSController, OVSKernelSwitch > - > -Switch = OVSKernelSwitch > -Controller = OVSController > -logging.basicConfig(level=logging.DEBUG) > -log.setLogLevel('info') > - > - > -class BasicOpenflowTest(unittest.TestCase): > - > - def addHost(self, N): > - logging.debug("Creating host h%s and add to net.", N) > - name = 'h%d' % N > - ip = '10.0.0.%d' % N > - return self.net.addHost(name, ip=ip) > - > - def setUp(self): > - self.net = Mininet(controller=Controller, switch=Switch) > - > - logging.info("Creating controllers") > - self.net.addController('c1', command='ovs-testcontroller') > - > - logging.info("Creating switches") > - s1 = self.net.addSwitch('s1', protocols="OpenFlow10") > - s2 = self.net.addSwitch('s2', protocols="OpenFlow10") > - > - logging.info("Creating hosts (7 on each switch)") > - hosts1 = [self.addHost(n) for n in (1, 2, 3, 4, 5, 6, 7)] > - hosts2 = [self.addHost(n) for n in (8, 9, 10, 11, 12, 13, 14)] > - > - logging.info("Creating links") > - for h in hosts1: > - self.net.addLink(s1, h) > - for h in hosts2: > - self.net.addLink(s2, h) > - self.net.addLink(s1, s2) > - > - logging.info("Starting network") > - self.net.start() > - > - def testPingAll(self): > - logging.info("Testing network") > - packetLoss = self.net.pingAll() > - self.assertTrue( > - packetLoss == 0, > - "Packet loss during ping test %s" % > - packetLoss) > - > - def testIPerfTCP(self): > - logging.info("Running TCP performance test") > - self.net.iperf() > - > - def testIPerfUDP(self): > - logging.info("Running UDP performance test") > - self.net.iperf(l4Type='UDP') > - > - def tearDown(self): > - logging.info("Stopping network") > - self.net.stop() > - > -if __name__ == '__main__': > - unittest.main() > diff --git a/debian/tests/run-tests.sh b/debian/tests/run-tests.sh > new file mode 100755 > index 000000000..abc23694f > --- /dev/null > +++ b/debian/tests/run-tests.sh > @@ -0,0 +1,151 @@ > +#!/bin/bash > + > +set -ex > + > +PROGRAM=`basename $0` > +TARGET=check-${PROGRAM} > + > +# The autopkgtests are run in throwaway environments, let's be good citizens > +# regardless, and attempt to clean up any environment modifications. > +function cleanup { > + rc=$? > + > + set +e > + > + # Dump the log to console on error > + if [ $rc -ne 0 ]; then > + case "${PROGRAM}" in > + kernel) > + # For historical reasons the log for the system kernel > + # datapath testsuite has a deviant name. > + logname="kmod" > + ;; > + *) > + logname="${PROGRAM}" > + ;; > + esac > + if [ -f _debian/tests/system-${logname}-testsuite.log ]; then > + cat _debian/tests/system-${logname}-testsuite.log > + fi > + fi > + > + # The DPDK test requires post-test cleanup steps. > + if [ "$PROGRAM" = "dpdk" ]; then > + mv /etc/dpdk/dpdk.conf.bak /etc/dpdk/dpdk.conf > + systemctl restart dpdk > + > + if dirs +1 > /dev/null 2>&1; then > + popd > + umount ${BIND_MOUNT_DIR} > + rmdir ${BIND_MOUNT_DIR} > + fi > + fi > + > + exit $rc > +} > +trap cleanup EXIT > + > +# The DPDK test requires preparing steps. > +if [ "$PROGRAM" = "dpdk" ]; then > + ARCH=$(dpkg --print-architecture) > + echo "Check required features on arch: ${ARCH}" > + case "${ARCH}" in > + amd64) > + # For amd64 the OVS DPDK support works with ssse3 > + # > https://github.com/openvswitch/ovs/blob/8045c0f8de5192355ca438ed7eef77457c3c1625/acinclude.m4#LL441C52-L441C52 > + if ! grep -q '^flags.*sse3' /proc/cpuinfo; then > + echo "Missing ssse3 on ${ARCH} - not supported, SKIP test" > + exit 77 > + fi > + ;; > + arm64) > + if ! grep -q '^Features.*crc32' /proc/cpuinfo; then > + echo "Missing crc32 on ${ARCH} - not supported, SKIP test" > + exit 77 > + fi > + ;; > + esac > + echo "no known missing feature on ${ARCH}, continue test" > + > + # Allocate hugepages, use 2M pages when possible because of higher > + # probability of successful allocation at runtime and smaller test > + # footprint in CI virtual machines. > + # > + # If the tests are to be run on real physical hardware, you may need > + # to adjust these variables depending on CPU architecture and topology. > + numa_node=$(lscpu | awk '/NUMA node\(s\)/{print$3}') > + if [ -z "$numa_node" -o "$numa_node" -eq 0 ]; then > + numa_node=1 > + fi > + DPDK_NR_1G_PAGES=${DPDK_NR_1G_PAGES:-0} > + DPDK_NR_2M_PAGES=${DPDK_NR_2M_PAGES:-$((${numa_node} * (2667 + 512) / > 2))} > + > + printf "Determine hugepage allocation for %s NUMA Node(s) on arch: %s" \ > + ${numa_node} ${ARCH} > + echo "DPDK_NR_2M_PAGES=${DPDK_NR_2M_PAGES}" > + echo "DPDK_NR_1G_PAGES=${DPDK_NR_1G_PAGES}" > + > + mv /etc/dpdk/dpdk.conf /etc/dpdk/dpdk.conf.bak > + cat << EOF > /etc/dpdk/dpdk.conf > +NR_1G_PAGES=${DPDK_NR_1G_PAGES} > +NR_2M_PAGES=${DPDK_NR_2M_PAGES} > +DROPCACHE_BEFORE_HP_ALLOC=1 > +EOF > + systemctl restart dpdk > + realhp_2m=$(cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages) > + realhp_1g=$(cat > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages) > + if [ "$realhp_2m" != "$DPDK_NR_2M_PAGES" -o \ > + "$realhp_1g" != "$DPDK_NR_1G_PAGES" ]; then > + echo "Unable to allocate huge pages required for the test, SKIP test" > + exit 77 > + fi > + > + # Long log messages from DPDK library overflow and is written as multiple > + # lines. This does not play well with the OVS testsuite assertions. > Even > + # a tmp directory in /tmp will make the paths too long. > + # > + # Realpaths from build will be embedded in testsuite artifacts, so we do > + # this before the build, and use a bind mount to avoid copying data > around > + # (using a symlink would not be sufficient). > + # > + # Ensure we use a short path for running the testsuite (LP:# 2019069). > + BIND_MOUNT_DIR=$(mktemp -d /XXX) > + mount --bind . ${BIND_MOUNT_DIR} > + pushd ${BIND_MOUNT_DIR} > +fi > + > +# A built source tree is required in order to make use of the system level > +# testsuites. > +# > +# We build it here instead of using the `build-needed` Restriction field, > +# because we need to pass in additional environment variables in order to > +# avoid running the build time checks yet another time (they would have just > +# run as part of the package under test build process anyway). > +export DEB_BUILD_OPTIONS="nocheck $DEB_BUILD_OPTIONS" > +debian/rules build > + > +# Ensure none of the Open vSwitch daemons are running. > +systemctl stop \ > + openvswitch-ipsec \ > + openvswitch-testcontroller \ > + ovs-vswitchd \ > + ovsdb-server > + > +# Optionally build list of tests to run, an empty list means run all tests. > +TEST_LIST="" > +if [ -f debian/tests/${PROGRAM}-skip-tests.txt ]; then > + TEST_LIST=$(cat debian/tests/${PROGRAM}-skip-tests.txt | \ > + debian/tests/testlist.py - tests/system-${PROGRAM}-testsuite) > +fi > + > +# Run the testsuite. > +# > +# By not having paths from build directory in AUTOTEST_PATH, apart from > +# `tests`, will ensure binaries are executed from system PATH, i.e. from the > +# binary package under test, and not the built source tree. > +make \ > + -C _debian \ > + ${TARGET} \ > + AUTOTEST_PATH=tests \ > + TESTSUITEFLAGS="-j1 ${TEST_LIST}" \ > + RECHECK=yes > diff --git a/debian/tests/system-userspace b/debian/tests/system-userspace > new file mode 120000 > index 000000000..7080a3249 > --- /dev/null > +++ b/debian/tests/system-userspace > @@ -0,0 +1 @@ > +run-tests.sh > \ No newline at end of file > diff --git a/debian/tests/testlist.py b/debian/tests/testlist.py > new file mode 100755 > index 000000000..bc351261a > --- /dev/null > +++ b/debian/tests/testlist.py > @@ -0,0 +1,74 @@ > +#!/usr/bin/env python3 > +import enum > +import itertools > +import os > +import sys > + > + > +if len(sys.argv) < 3: > + print( > + "usage: {} skipdescriptionlist|- testsuite\n" > + "\n" > + "This program reads two files, a skiplist containing the \n" > + "description of tests to skip separated by newline, and a \n" > + "generated testsuite script.\n" > + "\n" > + "From this it produces string with range of tests to execute \n" > + "which can be provided to the testsuite > script.\n".format(sys.argv[0]), > + file=sys.stderr, > + ) > + sys.exit(os.EX_USAGE) > + > + > +SKIP_TEST_STRINGS = [] > +with open(sys.argv[1]) if sys.argv[1] != "-" else sys.stdin as fin: > + SKIP_TEST_STRINGS = [ > + line.rstrip() for line in fin.readlines() if not line.startswith("#") > + ] > + > + > [email protected] > +class State(enum.Enum): > + INIT = enum.auto() > + AT_HELP_ALL = enum.auto() > + > + > +SKIP_TESTS = set() > +TESTS = set() > +with open(sys.argv[2]) as fin: > + state = State.INIT > + last_test = 0 > + for line in fin.readlines(): > + if state == State.INIT: > + if not line.startswith('at_help_all="'): > + continue > + else: > + state = State.AT_HELP_ALL > + data = line.split('"')[1].rstrip().split(";") > + elif state == State.AT_HELP_ALL: > + if line.startswith('"'): > + break > + data = line.rstrip().split(";") > + test_nr = int(data[0]) > + if last_test < test_nr: > + last_test = test_nr > + for skip_string in SKIP_TEST_STRINGS: > + if skip_string in data[2]: > + SKIP_TESTS.add(test_nr) > + else: > + TESTS.add(test_nr) > + > + > +def ranges(testlist): > + for a, b in itertools.groupby( > + enumerate(list(testlist)), lambda pair: pair[1] - pair[0] > + ): > + b = list(b) > + yield b[0][1], b[-1][1] > + > + > +testranges = [ > + "{}-{}".format(testrange[0], testrange[1]) > + for testrange in ranges(TESTS - SKIP_TESTS) > +] > +print(" ".join(testranges)) > diff --git a/debian/tests/vanilla b/debian/tests/vanilla > deleted file mode 100755 > index 80304f4df..000000000 > --- a/debian/tests/vanilla > +++ /dev/null > @@ -1,29 +0,0 @@ > -#!/bin/sh > - > -set -e > - > -echo "Checking service status right after install: " > -# for transparency we want to see all status and then fail if one is inactive > -systemctl status ovsdb-server.service || true > -systemctl status ovs-vswitchd.service || true > -systemctl status openvswitch-switch.service || true > -systemctl is-active ovs-vswitchd.service ovsdb-server.service > openvswitch-switch.service > -echo "OK" > - > -echo "Checking daemon pids to exist: " > -pgrep ovs-vswitchd > -pgrep ovsdb-server > -echo "OK" > - > -echo "stop conflicting openvswitch testcontroller" > -systemctl stop openvswitch-testcontroller || true > - > -if dpkg --compare-versions "$(dpkg-query --showformat '${Version}\n' --show > mininet)" ge "2.3.0-1"; then > - PYCMD="python3" > -else > - PYCMD="python2" > -fi > - > -printf "running openflow tests using mininet" > -${PYCMD} `dirname $0`/openflow.py 2>&1 > -echo "OK" > -- > 2.40.1 > > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
