On 3/4/25 14:26, Mark Brown wrote:
On Thu, Apr 03, 2025 at 02:19:40PM +0800, David Gow wrote:
That being said, there are a _lot_ of Cirrus FW tests (over 5000 by my
count when all of the parameterised ones are taken into account), so
this can still trigger the default 5-minute kunit.py timeout. And
because its split up over a lot of smaller suites, it doesn't really
make sense to mark all of those tests/suites as slow (though some of
them might merit it). If that's causing issues for people, maybe that
could be a good reason to remove them from KUNIT_ALL_TESTS, though
personally, I suspect the better answer might be to move it behind a
config option which can then be excluded from the default .kunitconfig
if necessary, as in [2]. (Or just increase the default timeut.)
I've not heard anyone mention hitting the timeouts, though now I run
interactively I do see that the Cirrus stuff is a good proportion of the
runtime for --alltests. We could potentially add a config option,
though it'd need to end up in the config file so it gets turned on and
we'd still run into any timeout issues. There's a tension with people
expecting --alltests to actually run all the tests as well...
I don't want to get into the situation where nobody outside of Cirrus
is running the tests. One of the main points of converting our tests to
kunit was the hope that more people would run them on more
configurations.
Ideally if they have any failures they would report those back with
enough information for us to investigate. Disabling tests that fail
and ignoring them isn't helpful.
