On 2/28/23 16:52, Eelco Chaudron wrote: > Run "make check-offloads" as part of the GitHub actions tests. > > This test was run 25 times using GitHub actions, and the > failing rerun test cases where excluded. There are quite some > first-run failures, but unfortunately, there is no other > more stable kernel available as a GitHub-hosted runner. > > Did not yet include sanitizers in the run, as it's causing > the test to run too long >30min and there seems to be (timing) > issues with some of the tests. > > Signed-off-by: Eelco Chaudron <[email protected]> > --- > v2: Added a new test keyword to exclude the failing tests. > Added some documentation around the keyword usage. > > .ci/linux-build.sh | 6 +++++- > .github/workflows/build-and-test.yml | 10 +++++++++- > Documentation/topics/testing.rst | 16 ++++++++++++++++ > tests/system-offloads-traffic.at | 3 +++ > tests/system-traffic.at | 2 ++ > 5 files changed, 35 insertions(+), 2 deletions(-) > > diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh > index 6394a8137..19ed9796d 100755 > --- a/.ci/linux-build.sh > +++ b/.ci/linux-build.sh > @@ -159,7 +159,7 @@ fi > > OPTS="${EXTRA_OPTS} ${OPTS} $*" > > -if [ "$TESTSUITE" ]; then > +if [ "$TESTSUITE" = 'test' ]; then > # 'distcheck' will reconfigure with required options. > # Now we only need to prepare the Makefile without sparse-wrapped CC. > configure_ovs > @@ -169,6 +169,10 @@ if [ "$TESTSUITE" ]; then > TESTSUITEFLAGS=-j4 RECHECK=yes > else > build_ovs > + if [ -n "$TESTSUITE" ]; then > + sudo -E PATH="$PATH" make "$TESTSUITE" TESTSUITEFLAGS="$TEST_OPTS" \ > + RECHECK=yes > + fi > fi > > exit 0 > diff --git a/.github/workflows/build-and-test.yml > b/.github/workflows/build-and-test.yml > index 86e594bf3..45326b659 100644 > --- a/.github/workflows/build-and-test.yml > +++ b/.github/workflows/build-and-test.yml > @@ -17,6 +17,7 @@ jobs: > OPTS: ${{ matrix.opts }} > SANITIZERS: ${{ matrix.sanitizers }} > TESTSUITE: ${{ matrix.testsuite }} > + TEST_OPTS: ${{ matrix.test_opts }} > > name: linux ${{ join(matrix.*, ' ') }} > runs-on: ubuntu-22.04 > @@ -86,6 +87,10 @@ jobs: > m32: m32 > opts: --disable-ssl > > + - compiler: gcc > + testsuite: check-offloads > + test_opts: "-k !github_offloads_skip" > + > steps: > - name: checkout > uses: actions/checkout@v3 > @@ -147,7 +152,10 @@ jobs: > # So, we're just archiving everything here to avoid any issues. > mkdir logs > cp config.log ./logs/ > - cp -r ./*/_build/sub/tests/testsuite.* ./logs/ || true > + sudo chown -R $USER ./tests/testsuite.* \ > + ./tests/system-offloads-testsuite.* || true > + cp -r ./tests/testsuite.* ./logs/ || true
This seems suspicious because distcheck builds and runs tests in a different directory. > + cp -r ./tests/system-offloads-testsuite.* ./logs/ || true > tar -czvf logs.tgz logs/ > > - name: upload logs on failure > diff --git a/Documentation/topics/testing.rst > b/Documentation/topics/testing.rst > index 5f6940b84..d06167ee3 100644 > --- a/Documentation/topics/testing.rst > +++ b/Documentation/topics/testing.rst > @@ -114,6 +114,22 @@ And from another window, one can execute ovs-xxx > commands like:: > > Once done with investigation, press ENTER to perform cleanup operation. > > +GitHub actions > +++++++++++++++ Empty line should be here. > +The OVS GitHub repository also runs some of these unit tests through GitHub > +actions. These tests are defined in the > +``ovs/.github/workflows/build-and-test.yml`` file. > + > +Based on the GitHub runners available, not all tests will work. In these > cases, > +the AT_KEYWORDS() macro can be used. For example, to skip a > +``make check-offloads`` test, use the ``github_offloads_skip`` keyword. > + > +Only use these keywords if no other way to skip the test is available. s/skip/fix or skip/ maybe? > + > +To see a list of currently skipped tests, you can do something like:: > + > + $ make check-offloads TESTSUITEFLAGS="-l" | grep -B 1 > github_offloads_skip > + > .. _testing-coverage: > > Coverage > diff --git a/tests/system-offloads-traffic.at > b/tests/system-offloads-traffic.at > index 7558812eb..6ee92746b 100644 > --- a/tests/system-offloads-traffic.at > +++ b/tests/system-offloads-traffic.at > @@ -191,6 +191,7 @@ AT_CLEANUP > > AT_SETUP([offloads - check interface meter offloading - offloads disabled]) > AT_KEYWORDS([dp-meter]) > +AT_KEYWORDS([github_offloads_skip]) > AT_SKIP_IF([test $HAVE_NC = "no"]) > OVS_TRAFFIC_VSWITCHD_START() > > @@ -240,6 +241,7 @@ AT_CLEANUP > > AT_SETUP([offloads - check interface meter offloading - offloads enabled]) > AT_KEYWORDS([offload-meter]) > +AT_KEYWORDS([github_offloads_skip]) > AT_SKIP_IF([test $SUPPORT_TC_INGRESS_PPS = "no"]) > AT_SKIP_IF([test $HAVE_NC = "no"]) > OVS_TRAFFIC_VSWITCHD_START([], [], [-- set Open_vSwitch . > other_config:hw-offload=true]) > @@ -348,6 +350,7 @@ AT_CLEANUP > > > AT_SETUP([offloads - check_pkt_len action - offloads enabled]) > +AT_KEYWORDS([github_offloads_skip]) > OVS_TRAFFIC_VSWITCHD_START([], [], [-- set Open_vSwitch . > other_config:hw-offload=true]) > > ADD_NAMESPACES(at_ns1, at_ns2, at_ns3, at_ns4) > diff --git a/tests/system-traffic.at b/tests/system-traffic.at > index 3a15b88a2..e8e4ced08 100644 > --- a/tests/system-traffic.at > +++ b/tests/system-traffic.at > @@ -2279,6 +2279,7 @@ OVS_TRAFFIC_VSWITCHD_STOP > AT_CLEANUP > > AT_SETUP([conntrack - force commit]) > +AT_KEYWORDS([github_offloads_skip]) > CHECK_CONNTRACK() > OVS_TRAFFIC_VSWITCHD_START() > AT_CHECK([ovs-appctl vlog/set dpif:dbg dpif_netdev:dbg > ofproto_dpif_upcall:dbg]) > @@ -7112,6 +7113,7 @@ OVS_TRAFFIC_VSWITCHD_STOP > AT_CLEANUP > > AT_SETUP([conntrack - Multiple ICMP traverse]) > +AT_KEYWORDS([github_offloads_skip]) > dnl This tracks sending ICMP packets via conntrack multiple times for the > dnl same packet > CHECK_CONNTRACK() > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
