On 7/7/23 14:27, Ales Musil wrote:
> On Wed, Jul 5, 2023 at 7:08 PM Dumitru Ceara <dce...@redhat.com> wrote:
>
>> If we want to catch new failures faster we have a better chance if CI
>> doesn't auto-retry (once).
>>
>> There are some tests that are still "unstable" and fail every now and
>> then. In order to reduce the number of false negatives keep the
>> --recheck for them. To achieve that we use a new macro, TAG_UNSTABLE,
>> to tag all these tests. The list of "unstable" tests is compiled based
>> on the following discussion:
>> https://mail.openvswitch.org/pipermail/ovs-dev/2023-June/405465.html
>>
>> In order to avoid new GitHub actions jobs, we re-purpose the last job of
>> each target type to also run the unstable tests. These jobs were
>> already running less tests than others so the additional run time should
>> not be an issue.
>>
>> Signed-off-by: Dumitru Ceara <dce...@redhat.com>
>>
>
> Hi,
>
> I have one question below.
>
Hi Ales,
Thanks for the review!
>
>> ---
>> Changes in v1 (since RFC):
>> - kept recheck for unstable tests
>> - introduced TAG_UNSTABLE
>> - changed test.yml to run unstable tests in the last batch of every
>> test target type.
>> ---
>> .ci/ci.sh | 2 +-
>> .ci/linux-build.sh | 54 +++++++++++++++++++++++++++-----------
>> .github/workflows/test.yml | 15 ++++++-----
>> tests/ovn-ic.at | 1 +
>> tests/ovn-ipsec.at | 1 +
>> tests/ovn-macros.at | 5 ++++
>> tests/ovn-northd.at | 1 +
>> tests/ovn-performance.at | 1 +
>> tests/ovn.at | 13 +++++++++
>> 9 files changed, 70 insertions(+), 23 deletions(-)
>>
>> diff --git a/.ci/ci.sh b/.ci/ci.sh
>> index 10f11939c5..a500aba764 100755
>> --- a/.ci/ci.sh
>> +++ b/.ci/ci.sh
>> @@ -101,7 +101,7 @@ function run_tests() {
>> && \
>> ARCH=$ARCH CC=$CC LIBS=$LIBS OPTS=$OPTS TESTSUITE=$TESTSUITE \
>> TEST_RANGE=$TEST_RANGE SANITIZERS=$SANITIZERS DPDK=$DPDK \
>> - ./.ci/linux-build.sh
>> + UNSTABLE=$UNSTABLE ./.ci/linux-build.sh
>> "
>> }
>>
>> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
>> index 5a79a52daf..58e41a7b68 100755
>> --- a/.ci/linux-build.sh
>> +++ b/.ci/linux-build.sh
>> @@ -9,6 +9,7 @@ COMMON_CFLAGS=""
>> OVN_CFLAGS=""
>> OPTS="$OPTS --enable-Werror"
>> JOBS=${JOBS:-"-j4"}
>> +RECHECK=${RECHECK:-"no"}
>>
>> function install_dpdk()
>> {
>> @@ -99,6 +100,17 @@ function configure_clang()
>> COMMON_CFLAGS="${COMMON_CFLAGS}
>> -Wno-error=unused-command-line-argument"
>> }
>>
>> +function run_tests()
>> +{
>> + if ! make distcheck CFLAGS="${COMMON_CFLAGS} ${OVN_CFLAGS}" $JOBS \
>> + TESTSUITEFLAGS="$JOBS $TEST_RANGE" RECHECK=$RECHECK
>> + then
>> + # testsuite.log is necessary for debugging.
>> + cat */_build/sub/tests/testsuite.log
>> + exit 1
>>
>
> This essentially mean that if we fail in the stable part we won't run the
> unstable,
> which might be ok. But IMO it would be better to run both and fail
> depending on results
> from both WDYT? The same applies for system tests.
>
Makes sense, I did that and posted v2 here:
https://patchwork.ozlabs.org/project/ovn/patch/20230710152543.95100-1-dce...@redhat.com/
Regards,
Dumitru
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
