Run "make check-offloads" as part of the GitHub actions tests. This test was run 25 times using GitHub actions, and the failing rerun test cases where excluded. There are quite some first-run failures, but unfortunately, there is no other more stable kernel available as a GitHub-hosted runner.
Did not yet include sanitizers in the run, as it's causing the test to run too long >30min and there seems to be (timing) issues with some of the tests. Signed-off-by: Eelco Chaudron <[email protected]> --- v2: Added a new test keyword to exclude the failing tests. Added some documentation around the keyword usage. v3: Fixed documentation and log copy for make distcheck. .ci/linux-build.sh | 6 +++++- .github/workflows/build-and-test.yml | 9 ++++++++- Documentation/topics/testing.rst | 17 +++++++++++++++++ tests/system-offloads-traffic.at | 3 +++ tests/system-traffic.at | 2 ++ 5 files changed, 35 insertions(+), 2 deletions(-) diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 6394a8137..19ed9796d 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -159,7 +159,7 @@ fi OPTS="${EXTRA_OPTS} ${OPTS} $*" -if [ "$TESTSUITE" ]; then +if [ "$TESTSUITE" = 'test' ]; then # 'distcheck' will reconfigure with required options. # Now we only need to prepare the Makefile without sparse-wrapped CC. configure_ovs @@ -169,6 +169,10 @@ if [ "$TESTSUITE" ]; then TESTSUITEFLAGS=-j4 RECHECK=yes else build_ovs + if [ -n "$TESTSUITE" ]; then + sudo -E PATH="$PATH" make "$TESTSUITE" TESTSUITEFLAGS="$TEST_OPTS" \ + RECHECK=yes + fi fi exit 0 diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml index 9f518ff01..ffcb1cd4f 100644 --- a/.github/workflows/build-and-test.yml +++ b/.github/workflows/build-and-test.yml @@ -17,9 +17,10 @@ jobs: OPTS: ${{ matrix.opts }} SANITIZERS: ${{ matrix.sanitizers }} TESTSUITE: ${{ matrix.testsuite }} + TEST_OPTS: ${{ matrix.test_opts }} name: linux ${{ join(matrix.*, ' ') }} - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 timeout-minutes: 30 strategy: @@ -86,6 +87,10 @@ jobs: m32: m32 opts: --disable-ssl + - compiler: gcc + testsuite: check-offloads + test_opts: "-k !github_offloads_skip" + steps: - name: checkout uses: actions/checkout@v3 @@ -146,8 +151,10 @@ jobs: # Also, upload-artifact@v2 doesn't work well enough with wildcards. # So, we're just archiving everything here to avoid any issues. mkdir logs + sudo chown -R $USER ./tests/system-offloads-testsuite.* || true cp config.log ./logs/ cp -r ./*/_build/sub/tests/testsuite.* ./logs/ || true + cp -r ./tests/system-offloads-testsuite.* ./logs/ || true tar -czvf logs.tgz logs/ - name: upload logs on failure diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index 5f6940b84..c2d680e71 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -114,6 +114,23 @@ And from another window, one can execute ovs-xxx commands like:: Once done with investigation, press ENTER to perform cleanup operation. +GitHub actions +++++++++++++++ + +The OVS GitHub repository also runs some of these unit tests through GitHub +actions. These tests are defined in the +``ovs/.github/workflows/build-and-test.yml`` file. + +Based on the GitHub runners available, not all tests will work. In these cases, +the AT_KEYWORDS() macro can be used. For example, to skip a +``make check-offloads`` test, use the ``github_offloads_skip`` keyword. + +Only use these keywords if no other way to fix or skip the test is available. + +To see a list of currently skipped tests, you can do something like:: + + $ make check-offloads TESTSUITEFLAGS="-l" | grep -B 1 github_offloads_skip + .. _testing-coverage: Coverage diff --git a/tests/system-offloads-traffic.at b/tests/system-offloads-traffic.at index eb331d6ce..5cde26663 100644 --- a/tests/system-offloads-traffic.at +++ b/tests/system-offloads-traffic.at @@ -191,6 +191,7 @@ AT_CLEANUP AT_SETUP([offloads - check interface meter offloading - offloads disabled]) AT_KEYWORDS([dp-meter]) +AT_KEYWORDS([github_offloads_skip]) AT_SKIP_IF([test $HAVE_NC = "no"]) OVS_TRAFFIC_VSWITCHD_START() @@ -240,6 +241,7 @@ AT_CLEANUP AT_SETUP([offloads - check interface meter offloading - offloads enabled]) AT_KEYWORDS([offload-meter]) +AT_KEYWORDS([github_offloads_skip]) AT_SKIP_IF([test $SUPPORT_TC_INGRESS_PPS = "no"]) AT_SKIP_IF([test $HAVE_NC = "no"]) OVS_TRAFFIC_VSWITCHD_START([], [], [-- set Open_vSwitch . other_config:hw-offload=true]) @@ -348,6 +350,7 @@ AT_CLEANUP AT_SETUP([offloads - check_pkt_len action - offloads enabled]) +AT_KEYWORDS([github_offloads_skip]) OVS_TRAFFIC_VSWITCHD_START([], [], [-- set Open_vSwitch . other_config:hw-offload=true]) ADD_NAMESPACES(at_ns1, at_ns2, at_ns3, at_ns4) diff --git a/tests/system-traffic.at b/tests/system-traffic.at index 380372430..e69eafdc3 100644 --- a/tests/system-traffic.at +++ b/tests/system-traffic.at @@ -2279,6 +2279,7 @@ OVS_TRAFFIC_VSWITCHD_STOP AT_CLEANUP AT_SETUP([conntrack - force commit]) +AT_KEYWORDS([github_offloads_skip]) CHECK_CONNTRACK() OVS_TRAFFIC_VSWITCHD_START() AT_CHECK([ovs-appctl vlog/set dpif:dbg dpif_netdev:dbg ofproto_dpif_upcall:dbg]) @@ -7186,6 +7187,7 @@ OVS_TRAFFIC_VSWITCHD_STOP AT_CLEANUP AT_SETUP([conntrack - Multiple ICMP traverse]) +AT_KEYWORDS([github_offloads_skip]) dnl This tracks sending ICMP packets via conntrack multiple times for the dnl same packet CHECK_CONNTRACK() _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
