On 7/11/23 11:59, Dumitru Ceara wrote: > On 7/11/23 10:30, Ales Musil wrote: >> On Tue, Jul 11, 2023 at 7:14 AM Ales Musil <[email protected]> wrote: >> >>> >>> >>> On Mon, Jul 10, 2023 at 5:26 PM Dumitru Ceara <[email protected]> wrote: >>> >>>> If we want to catch new failures faster we have a better chance if CI >>>> doesn't auto-retry (once). >>>> >>>> There are some tests that are still "unstable" and fail every now and >>>> then. In order to reduce the number of false negatives keep the >>>> --recheck for them. To achieve that we use a new macro, TAG_UNSTABLE, >>>> to tag all these tests. The list of "unstable" tests is compiled based >>>> on the following discussion: >>>> https://mail.openvswitch.org/pipermail/ovs-dev/2023-June/405465.html >>>> >>>> In order to avoid new GitHub actions jobs, we re-purpose the last job of >>>> each target type to also run the unstable tests. These jobs were >>>> already running less tests than others so the additional run time should >>>> not be an issue. >>>> >>>> Signed-off-by: Dumitru Ceara <[email protected]> >>>> --- >>>> V2: >>>> - Addressed Ales' comments: >>>> - always run stable and unstable tests before declaring pass/fail >>>> Changes in v1 (since RFC): >>>> - kept recheck for unstable tests >>>> - introduced TAG_UNSTABLE >>>> - changed test.yml to run unstable tests in the last batch of every >>>> test target type. >>>> --- >>>> .ci/ci.sh | 2 +- >>>> .ci/linux-build.sh | 76 ++++++++++++++++++++++++++++++-------- >>>> .github/workflows/test.yml | 15 ++++---- >>>> tests/ovn-ic.at | 1 + >>>> tests/ovn-ipsec.at | 1 + >>>> tests/ovn-macros.at | 5 +++ >>>> tests/ovn-northd.at | 1 + >>>> tests/ovn-performance.at | 1 + >>>> tests/ovn.at | 13 +++++++ >>>> 9 files changed, 92 insertions(+), 23 deletions(-) >>>> >>>> diff --git a/.ci/ci.sh b/.ci/ci.sh >>>> index 10f11939c5..a500aba764 100755 >>>> --- a/.ci/ci.sh >>>> +++ b/.ci/ci.sh >>>> @@ -101,7 +101,7 @@ function run_tests() { >>>> && \ >>>> ARCH=$ARCH CC=$CC LIBS=$LIBS OPTS=$OPTS TESTSUITE=$TESTSUITE \ >>>> TEST_RANGE=$TEST_RANGE SANITIZERS=$SANITIZERS DPDK=$DPDK \ >>>> - ./.ci/linux-build.sh >>>> + UNSTABLE=$UNSTABLE ./.ci/linux-build.sh >>>> >>> >> I've missed one thing, please add the recheck here as well. >> >> > > Good catch! I'll send a v3. I think I also need to add at least one > more test to the list of unstable ones: "system-ovn.at:11332 Tiered ACLs > -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes" >
I posted v3: https://patchwork.ozlabs.org/project/ovn/list/?series=363690 In the end I didn't add the "Tiered ACLs" test case to the list of unstable ones because Xavier informed me privately that he's working on a fix. Regards, Dumitru _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
