What you are describing sounds like denormalisation, but the first step would be to ensure that the indexes on the child tables are being created.
The query time to find rows if the lookup field is indexed is close to linear, without an index it will be close to exponential. You should only denormalise after you have investigated the root cause of your performance issues. Most times, denormalization is a form of premature optimization [ https://stackify.com/premature-optimization-evil/ ] If you are talking about virtual denormalisation (ie a view or table generated by the manager you mention) then this relies again on indexing of the underlying data to work, so you get no performance gain. Correct me if I'm wrong, but AFAIK this is true of most managers - they help you with your logical view of the data, not the performance of retrieving it. There are many django deployments where the performance is fine with orders of magnitude more data than you are referring to, it just takes careful planning. D On Tue, 27 Oct 2020 at 13:40, Matthew Amstutz <silverstrings...@gmail.com> wrote: > > Hello, I was wondering about instance based management. If I'm wrong, > please tell me. > > When we have users and user generated content in a large database, query > times are increased significantly. Why is there no instance based manager > (like the models.Manager()) that basically generates a table for each user > and queries ONLY that table? Would that not just flatten the database > instead of increasing it's size? For example, if we have 1,000,000 users > all of which generate at least 10 posts per day and one of the users only > generates 5 in the span of 10 days, unless we have a many to many field or > something to hold those five posts, the query time to find their posts > would be ridiculous. > > So if we have a table generated for each user that holds arbitrary > connections to anything they generate, it would in theory cut query times > significantly. Why is there no feature like this? Again, if I'm wrong > please tell me but the amount of tables doesn't matter and instead the data > they hold does so, in my understanding, 1,000,000,000 posts will always be > the size of 1,000,000,000 posts no matter their organization. > > I've got ideas on implementation and even asyncronous supports as well as > customization but I have no idea how to bring this up to the django > developers and I'm not even sure it would work (though, no matter how hard > I try, I can't see anything wrong with it). > > Let me know your input and if there's a way I can ask the django devs > about this and possibly even suggest a few things pertaining to it. I'd > like to help make django the best it can be and if this works and we can > implement it, django will be very fast with user generated content. > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/7e36ded7-2f3d-43c2-881c-cbc75c80b5c2n%40googlegroups.com > <https://groups.google.com/d/msgid/django-developers/7e36ded7-2f3d-43c2-881c-cbc75c80b5c2n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- -- ====================== Daryl Egarr, Director Kawhai Consultants Ltd Cell 021 521 353 da...@kawhai.net ====================== -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALzH9quw5tSNNuAe%2Bspt9edTcJWV_ckUqdiyEtW5Qybx9kFepQ%40mail.gmail.com.
