I've been struggling with getting Django's defaults to play nicely with our rather large data-sets. For small data-sets, Django works like a charm. However, our company manages around tens of thousands of phones for other companies (and then there's the call-detail for many of those accounts/providers, with about 600k rows there for merely one account).
I'm looking for tips on how to better manage this data. On the default admin, yes one can search for a given phone number rather than wade through page-after-page-after-page of phones. But there's also a hierarchy...clients have providers and providers have phones and statements, so it would be nice to limit the phones to a given client (with several hundred clients, drilldown would be nice). Are there any "best practices" or war stories of things to do (or things *not* to do) when using Django with large volumes of data? Or am I writing that chapter as I encounter it? Some of my lessons learned so far: -be careful with tracebacks if, for some reason, you have a query-set that brings back bajillions of records and you error out before filtering them, they'll show up in your debugging page (45 minutes after you made the page-request, and pumping your browser's memory usage through the roof) -the default admin tool is good for blogs, bad for bajillions of records. This might be mitigated a bit with some judicious use of limit_choices_to but I haven't mastered such yet -PostgreSQL can do dumb things with your queries, which a little jiggering/subclassing of the query object can mung into something a little smarter (and much faster-running) Any further additions to this list from other veterans of large data-sets would be most welcome... Thanks! -tkc --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
