> > I guess this is true in a technical sense, but is it true in a noticeable > sense? Yes, having lots of processes vying for a lock isn't great, but I'd > be surprised if it adds up to very much. Am I wrong about this?
Running the tests serially in a single process doesn't have the setup cost of multiple processes and databases, so yes the overhead is quite high. Well, not *silently*, right? Isn't the whole point of SerialMixin precisely > to degrade tests to serial performance? If you're using it, that's exactly > what you're after right? My thought is that by default it should fully > serialize tests it's mixed into, and that by configuring a different file > (or "__file__", say), you can create sub-groups of tests that must run > serially (while others can continue in other processes). Maybe I'm missing > a piece of the puzzle here? > I'm worried that someone will discover that SerializeMixin "fixes their problems" and then proceed to use it on every test case in their suite, even where it is unnecessary. If SerializeMixin had a default file it would make this a bit easier and implicit. Requiring a file declaration introduces at least a bit of thought about what will be locked. If you want a default file for your project, it takes just a two line subclass... On Fri, 21 May 2021 at 22:41, 'Mike Lissner' via Django developers (Contributions to Django itself) <django-developers@googlegroups.com> wrote: > > you will degrade to *worse than single process* speed. > > I guess this is true in a technical sense, but is it true in a noticeable > sense? Yes, having lots of processes vying for a lock isn't great, but I'd > be surprised if it adds up to very much. Am I wrong about this? > > > This would silently degrade parallel tests to serial performance if > SerialMixin was incorrectly configured. > > Well, not *silently*, right? Isn't the whole point of SerialMixin > precisely *to* degrade tests to serial performance? If you're using it, > that's exactly what you're after right? My thought is that by default it > should fully serialize tests it's mixed into, and that by configuring a > different file (or "__file__", say), you can create sub-groups of tests > that must run serially (while others can continue in other processes). > Maybe I'm missing a piece of the puzzle here? > > (Nice fix in the PR, thanks.) > On Friday, May 21, 2021 at 2:43:12 AM UTC-7 Adam Johnson wrote: > >> 1. Should we tweak the docs to show something like the above, that would >>> make the solution there easier to just drop in? >>> >> >> I don't think so. Locking is a last resort and ruins performance - if >> used across your whole test suite you will degrade to *worse than single >> process* speed. You should instead try to isolate your resources per >> process, e.g. by mixing os.getpid() in resource names. >> >> I don't think Django should provide much more than it already does - if I >> was to make a default for SerializeMixin it would be __file__ as the docs >> suggest, but that's a bit hard since __file__ is dynamic per module. >> >> 2. Would it be crazy to upgrade the SerailzeMixin so that if lockfile is >>> left out, it finds some standard file in every django project and uses that >>> for the locking? If it worked that way, switching to parallel tests would >>> just mean using the `--parallel` flag, identifying tests that failed, and >>> mixing in the SerializeMixin into each of those tests. It'd be much easier. >>> Currently if the property is missing, you get a ValueError. >>> >> >> I don't think it would be sensible to use a "standard file". This would >> silently degrade parallel tests to serial performance if SerialMixin was >> incorrectly configured. >> >> Instead, we can shift the check for the lockfile attribute to import >> time. Then you don't need to go through a whole test run to find >> misconfigured tests. I've made a PR for that here: >> https://github.com/django/django/pull/14427 >> >> Thanks, >> >> Adam >> >> On Fri, 21 May 2021 at 01:36, 'Mike Lissner' via Django developers >> (Contributions to Django itself) <django-d...@googlegroups.com> wrote: >> >>> I spent the day today making the tests for my project run in parallel. >>> As part of that, I had my first encounter with the SerializeMixin, which >>> you can use in conjunction with the `--parallel` flag to ensure that >>> certain tests don't run at the same time. It's neat, in theory: >>> >>> >>> https://docs.djangoproject.com/en/3.2/topics/testing/advanced/#enforce-running-test-classes-sequentially >>> >>> One thing I found kind of lacking about it is that it didn't easily >>> solve my main problem. What I wanted to do was to just mark meddlesome >>> tests to run serially. To do that though is kind of tricky. The way I >>> eventually did so was to set lockfile as: >>> >>> lockfile = Path(__file__).parents[1] / "settings.py" >>> >>> I didn't find many others using SerializeMixin online (searching Github >>> and Google), which surprised me, and I didn't see any other solutions like >>> mine when I looked either. It sort of felt like an under-used tool. >>> >>> The way it works now, you use the SerializeMixin in your test class, and >>> then you set up a property on the class called `lockfile`, that points to a >>> file that is used to lock the tests so none others can run at the same >>> time. >>> >>> The docs show an example that has `lockfile = __file__`, which I thought >>> I could just sprinkle into my meddlesome tests to fix them. Unfortunately, >>> that just locks different classes within a single tests.py file, so I had >>> to do the solution above. >>> >>> A couple thougths: >>> >>> 1. Should we tweak the docs to show something like the above, that would >>> make the solution there easier to just drop in? >>> >>> 2. Would it be crazy to upgrade the SerailzeMixin so that if lockfile is >>> left out, it finds some standard file in every django project and uses that >>> for the locking? If it worked that way, switching to parallel tests would >>> just mean using the `--parallel` flag, identifying tests that failed, and >>> mixing in the SerializeMixin into each of those tests. It'd be much easier. >>> Currently if the property is missing, you get a ValueError. >>> >>> Thoughts? >>> >>> Mike >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django developers (Contributions to Django itself)" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to django-develop...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-developers/70ac0aec-f1a5-4bdb-876d-582e75168820n%40googlegroups.com >>> <https://groups.google.com/d/msgid/django-developers/70ac0aec-f1a5-4bdb-876d-582e75168820n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/d5261461-9d32-4069-bd7e-2f4ab7e04d02n%40googlegroups.com > <https://groups.google.com/d/msgid/django-developers/d5261461-9d32-4069-bd7e-2f4ab7e04d02n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM0QAoWQVSRQQjNoM-evL0uoEBuucCdSYsDoV7UjoHNONg%40mail.gmail.com.
