Is this opt-{in,out} considered to be a global flag, meant to be
toggled on or off depending on whether it is an "expert" working on
the project or not?I don't think that would be a good idea, almost all of our projects have a mix of skill levels, and people move from team to team on a regular basis. I don't think we have a project that has only been worked on by senior developers or only worked on by junior developers. What I think would be exceptionally useful would be to emit loud warnings when developing (DEBUG=True or runserver) whenever this code can detect that a prefetch MAY be applicable. A lot of our junior developers are not as aware about DB structures and SQL performance (particularly now that they don't design schemas and write DB queries), so warnings as part of their normal development help trigger teaching moments. Magically (sometimes) making things faster doesn't teach them anything. Turning this feature on/off per app is scary. If we're dealing with models from AppA, we will get auto pre-fetching, but if we work with models from AppB, we do not? If we're dealing with those models in the same module, we will have different behaviour depending on which model is being used? Please no. I also think that it might be handy to specify "qs.prefetch_related(auto=True)", or "with qs.auto_prefetch():", which would then trigger the newly proposed behaviour. It's like "I want you to prefetch all the things I use, but I don't know what just yet". Having said that, I'd also like that to spit out a warning (again, dev only) that specified what was actually prefetched, because why waste a DB query in production to determine what we know whilst developing? Cheers Tom On Wed, Aug 16, 2017 at 5:28 PM, Brice Parent <cont...@brice.xyz> wrote: > Le 16/08/17 à 14:07, Adam Johnson a écrit : >> >> I wouldn't want is to give free optimisations to non-ORM pros > > > Something like 99% of django projects are by non-ORM pros, it can be easy to > forget that on a mailing list of experts. > > I don't count myself as an ORM or SQL expert at all, but I've worked on at > least 2 projects for which they had contracted such experts to optimise > their projects once they reached some limits. Those experts only worked for > them for a limited period of time, and if we changed this behaviour by > default, I'm 100% sure they wouldn't understanding why their website was > getting slower with the new release. And they would probably only see the > performance issue on staging servers if they have some, or in production > with a full database. > I admit this can be a great thing for new comers and even small sites. But > remember that, with or without this functionality, their website will need > to be optimised in many ways when they start to have more users, and > removing this 1+N common pitfall (or hiding the problem, as it is not > completely solved as it's not the more efficient fix) will not be the only > point they should look at. > I think the solution for this would simply be to have it as opt-out, not to > harm any already-working and optimised websites and allow to maintain a > stable behaviour for 3rd party apps, but make it opt-in to new users just by > pre-setting some setting in the settings.py file generated by "django-admin > startproject". It could be a simple boolean setting or a list of apps for > which we want the auto-fetch feature to be activated on. With this, new > projects would be optimised unless they want to optimise manually, existing > projects would still work without decreasing performance or creating memory > problems in any case. Best of both worlds. > And for existing non-optimised but existing websites, I prefer the > status-quo than risking to create problems that might occur on production > website. > We could also, as proposed elsewhere in this thread, warn the developers (so > only when settings.debug is set) when this kind of 1+N queries are executed > that it could be way more efficient, either by activating the auto-fetch > functionality or by manually prefetch related data. > > - Brice > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at https://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/dc483976-f7d0-3aee-4f9e-76f515e5c761%40brice.xyz. > > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFHbX1%2B3qp3O1R2d4x4rUtYwe01Dc5PWfHd6gHC9BV%2BWkVp3Mw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
