Hi Jörg, All our tests are tested anyway with GitHub Actions. The idea is to run a subset of tests locally to catch 90% of the problems before I commit and wait 40 minutes for all the tests to run. It works most of the time. Of course the whole tests should be run before deploying to production, but running a subset of tests improves productivity in locating errors without having to wait for the full test suit to run.
(by the way, running all our tests take 90 minutes, but we skip many tests and run them in random anyway - we have 11 languages and we always test 3 specific languages + another language selected by random. This is how we reduce the time from 90 minutes to 40 minutes. And if we make changes in languages, we can wait 90 minutes and run all the tests) I also can run specific tests if I work on a specific module. For example if I work on a specific view - I can run only tests of this view. But again, of course we run all the tests before we deploy to production. Thanks, Uri. אורי u...@speedy.net On Mon, Feb 12, 2024 at 3:36 PM Jörg Breitbart <j.breitb...@netzkolchose.de> wrote: > @Uri > > May I ask, whats the idea behind running a random subset of tests only? > Wouldn't a Monte Carlo approach be highly unreliable, e.g. lure ppl into > thinking everything is ok, but in reality the random test selection did > not catch affected code paths? I mean for tests - its all about > reliability, isn't it? And 200 out of 6k tests sounds like running often > into false positive test results, esp. if your test base is skewed > towards features not being affected by current changes. > > I think this could still work with a better reliability / changed code > coverage, if the abstraction is a bit more complicated, e.g.: > - introduce grouping flags on tests - on module, class or even single > method scope > - on test run, declare what flags should be tested (runs all test with > given flags) > - alternatively use appropriate flags reflecting your code changes + > your test counter on top, but now it selects from the flagged tests with > higher probability to run affected tests > > Ah well, just some quick thoughts on that... > > Cheers, > Jörg > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/0138232b-4b8f-4a7a-99b0-67c0702f9ce8%40netzkolchose.de > . > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CABD5YeFeq_Y66GgjUD-OXP0%3D_v5LZLzC2bg4h6qaASzri0zSyA%40mail.gmail.com.
