On Thu, Mar 09, 2023 at 05:22:43PM +0100, Eelco Chaudron wrote: > > > On 9 Mar 2023, at 15:42, Simon Horman wrote: > > > On Wed, Mar 08, 2023 at 04:18:47PM +0100, Eelco Chaudron wrote: > >> Run "make check-offloads" as part of the GitHub actions tests. > >> > >> This test was run 25 times using GitHub actions, and the > >> failing rerun test cases where excluded. There are quite some > >> first-run failures, but unfortunately, there is no other > >> more stable kernel available as a GitHub-hosted runner. > >> > >> Did not yet include sanitizers in the run, as it's causing > >> the test to run too long >30min and there seems to be (timing) > >> issues with some of the tests. > >> > >> Signed-off-by: Eelco Chaudron <[email protected]> > > > > Hi Eelco, > > > > I like this patch a lot. > > But I am observing reliability problems when executing the new job. > > > > For 5 runs, on each occasion some tests failed the first time. > > And on 3 of those runs at least one test failed on the recheck, > > so the job failed. > > Damn :)
Yes, it pained me to report this. > I did 25 runs (I did not check for re-runs), and they were fine. I also > cleaned up my jobs recently, so I no longer have them. > > I can do this again and figure out wich tests are failing. Then analyze the > failures to see if we need to exclude them or can fine-tune them. I will see if I can spend some cycles on reproducing this (outside of GHA). I'll likely start with the tests that show up in the summary below. > Ilya any other approach you can think off? > > > > The job is here: > > https://github.com/horms/ovs/actions/runs/4365805718/jobs/7654575559 > > > > And a summary of the checks run for recheck for each job run is as follows. > > > > Attempt #1 > > 85: failed > > 166: ok > > > > Attempt #2 > > 50: ok > > 64: failed > > 86: ok > > 135: failed > > > > Attempt #3 > > 50: ok > > > > Attempt #4 > > 50: failed > > > > Attempt #5 > > 50: ok > > 163: ok > > 169: ok > > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
