On 08/12/2010 03:32 PM, Jani Tiainen wrote:
>> Hello all,
>>
>> I have a django application for analysis of accounting information in
>> torque batch system. The job is an object connected to several accounting
>> event objects:
>>
>> class Job(models.Model):
>> jobid = models.CharField(max_length=16, db_index=True, editable=False)
>> server = models.ForeignKey('TorqueServer', editable=False)
>> job_owner = models.ForeignKey('User', null=True)
>> ....
>>
>> class AccountingEvent(models.Model):
>> timestamp = models.DateTimeField(verbose_name='time stamp')
>> type = models.CharField(max_length=1, choices=EVENT_CHOICES)
>> job = models.ForeignKey('Job')
>>
>>
>> I have a method that goes through all jobs and tests their accounting
>> events. The problem is that my function that does the iteration over all
>> jobs eats all memory eventually. It is quite simple function:
>>
>> maxjobid =
>> Job.objects.filter(job_state__shortname='C').aggregate(Max("id"))['id__max
>> '] for n in range(1,maxjobid+1):
>> j = Job.objects.get(pk=n)
>> aes = AccountingEvent.objects.filter(job=j).order_by("-timestamp")
>> ae = aes[0]
>> if ae.type=='D':
>> j.job_state = getJobState('D')
>> j.save()
>>
>> I tried to inspect this with guppy and I see that the number of str and
>> unicode objects get really high. I don't understand why python garbage
>> collector does not free them. I see this with development server, debug on
>> and mysql as a DB backend.
>
> That is your problem. when DEBUG = True Django ORM records every SQL query
> made in a request - or in case of script whole lifetime of a script which can
> grow quite huge.
>
> http://docs.djangoproject.com/en/1.2/faq/models/#why-is-django-leaking-memory
>
Thank you very much. Shame on me for not finding this in django faq.
--
Tomas Kouba
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
