Are you using MySQL? It's been implicated in these sorts of issues before: http://nedbatchelder.com/blog/200809/a_server_memory_leak.html
--Ned. http://nedbatchelder.com 7timesTom wrote: > Some time ago Webfaction notified me "Memory usage over your > limit" (80MB) It's taken me ages to find this darn memory leak! It was > one short line that didn't even add functionality to my users. > > I have 2000 items (cars) in a database table, which my users can > search through. After the webfaction memory warning, I optimised the > code so that my pagination was in place so that in theory only 10 > items were loaded into memory per page view. But still my memory usage > was massive. > > Anyway today I notice the line: > msg =cgi.escape(str(len(cars)))+'<br>\n' # which stores a debugging > message for me > > That line was between these two lines (spread out by other stuff and > comments): > cars = Car.objects.filter(...) #upto 2000 items > msg =... len(cars) # can you see the massive memory use here? > paginator = Paginator(cars, per_page=10) > > As you all know Django has been well designed to be lazy at executing > database queries, this is how the Paginator can take a queryset > representing all 2000 items, and return just 10 having loaded only > those 10 into memory. > > So what's wrong with len(cars)? Well, it forces the db query to be > executed in full (before the Paginator line) and loaded into memory. > This caused massive memory use and made me go over my limit. > > If I'd wanted the total number of cars, or number of results, I > should've done Car.objects.filter(...).count() which actually executes > a SELECT COUNT(*) which returns a single number, rather than, in > essence, a huge list. > > Maybe this post will help someone else also one thing I don't > understand: > > Why wasn't the memory returned to me after each page view? Why was an > apache off/on necessary to clear memory? And is there anything I can > do to help clear memory that been used, but not returned to the system? > > > > -- Ned Batchelder, http://nedbatchelder.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
