On Mon, Sep 09, 2024 at 08:31:52AM +0000, Stefano Rivera wrote: > > Hi Julian (2024.09.08_21:33:49_+0000) > > > I built 6440 packages build-depending on dh-python in one way or > > > another. 1483 failed, and 1124 of them say "NO TESTS RAN" in the logs. > > > > My guess is that most of these 1124 have no tests at all, rather than > > having a misconfigured setup. A unittest is the pybuild default test > > framework, unittest is used and fails to find any tests, hence all of > > these failures. > > Yes, almost certainly. > [...] > > I would leave unittest as the default runner, but without missing test > detection. > > That's a slightly unexpected behaviour, but it makes the default case > work. > > Downside is that you have to opt-in to missing test detection. Maybe we > can have a lintian tag for that?
That seems a bit heavy to ask for. Is there any way of identifying those packages that do genuinely use unittest? If there are not that many of them, then implementing a --test-unittest option would be a good way to go. I would imagine the following timeline: (1) --test-unittest is introduced as an option to explicitly select unittest as the test framework. When --test-unittest is specified, the test will fail if no tests are found. unittest is still used as a fallback test framework; in this case, the dh_auto_test call will succeed if no tests are run. (2) Add some sort of warning for pybuild-using packages that run dh_auto_test but haven't specified a test framework and for which autodetection of the test framework fails. If there aren't any tests to run, an empty override_dh_auto_test target should be specified. (3) Stop using unittest as the default test framework, and fail if no test framework has been specified or autodetected. But that might be overkill for something which may not actually be much of a problem. Best wishes, Julian
