On 7/11/23 10:30, Ales Musil wrote: > On Tue, Jul 11, 2023 at 7:14 AM Ales Musil <[email protected]> wrote: > >> >> >> On Mon, Jul 10, 2023 at 5:26 PM Dumitru Ceara <[email protected]> wrote: >> >>> If we want to catch new failures faster we have a better chance if CI >>> doesn't auto-retry (once). >>> >>> There are some tests that are still "unstable" and fail every now and >>> then. In order to reduce the number of false negatives keep the >>> --recheck for them. To achieve that we use a new macro, TAG_UNSTABLE, >>> to tag all these tests. The list of "unstable" tests is compiled based >>> on the following discussion: >>> https://mail.openvswitch.org/pipermail/ovs-dev/2023-June/405465.html >>> >>> In order to avoid new GitHub actions jobs, we re-purpose the last job of >>> each target type to also run the unstable tests. These jobs were >>> already running less tests than others so the additional run time should >>> not be an issue. >>> >>> Signed-off-by: Dumitru Ceara <[email protected]> >>> --- >>> V2: >>> - Addressed Ales' comments: >>> - always run stable and unstable tests before declaring pass/fail >>> Changes in v1 (since RFC): >>> - kept recheck for unstable tests >>> - introduced TAG_UNSTABLE >>> - changed test.yml to run unstable tests in the last batch of every >>> test target type. >>> --- >>> .ci/ci.sh | 2 +- >>> .ci/linux-build.sh | 76 ++++++++++++++++++++++++++++++-------- >>> .github/workflows/test.yml | 15 ++++---- >>> tests/ovn-ic.at | 1 + >>> tests/ovn-ipsec.at | 1 + >>> tests/ovn-macros.at | 5 +++ >>> tests/ovn-northd.at | 1 + >>> tests/ovn-performance.at | 1 + >>> tests/ovn.at | 13 +++++++ >>> 9 files changed, 92 insertions(+), 23 deletions(-) >>> >>> diff --git a/.ci/ci.sh b/.ci/ci.sh >>> index 10f11939c5..a500aba764 100755 >>> --- a/.ci/ci.sh >>> +++ b/.ci/ci.sh >>> @@ -101,7 +101,7 @@ function run_tests() { >>> && \ >>> ARCH=$ARCH CC=$CC LIBS=$LIBS OPTS=$OPTS TESTSUITE=$TESTSUITE \ >>> TEST_RANGE=$TEST_RANGE SANITIZERS=$SANITIZERS DPDK=$DPDK \ >>> - ./.ci/linux-build.sh >>> + UNSTABLE=$UNSTABLE ./.ci/linux-build.sh >>> >> > I've missed one thing, please add the recheck here as well. > >
Good catch! I'll send a v3. I think I also need to add at least one more test to the list of unstable ones: "system-ovn.at:11332 Tiered ACLs -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes" Regards, Dumitru _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
