Re: Help interpreting profiler results (or: why is my app so slow?)

[Matt Andrews]

I wouldn't say huge (see the page/view in question at 
http://www.scenepointblank.com) -- I grab a bunch of things and then 
display them, mostly truncated (with HTML stripped out, usually for the 
homepage). The custom template tags take a bunch of arguments (say for a 
news post, it takes arguments about where to show the photo, how large to 
make the headline, etc) so there's lots of loading of views inside loops 
for each news story etc.

I've tried experimenting before with the template fragment caching and saw 
little in the way of results.


On Thursday, 18 April 2013 13:40:10 UTC+1, Shawn Milochik wrote:
>
> Hit send too soon:
>
>
> https://docs.djangoproject.com/en/dev/topics/cache/#template-fragment-caching
>
>
> On Thu, Apr 18, 2013 at 8:39 AM, Shawn Milochik 
>  > wrote:
>
>> Yes, it does look like template tags are taking some time. Is the page 
>> huge? Are you doing a ton of formatting? Is there something you could maybe 
>> move to server-side?
>>
>> Also, this might help with caching bits of your output: 
>>
>>
>> On Thu, Apr 18, 2013 at 6:17 AM, Matt Andrews 
>> > > wrote:
>>
>>>
>>> On Thursday, 18 April 2013 10:45:40 UTC+1, Tom Evans wrote:
>>>
>>> On Wed, Apr 17, 2013 at 11:18 PM, Matt Andrews  
 wrote: 
 > Hi all. 
 > 
 > Having performance problems with my Django app. I've posted here 
 before 
 > talking about this: one theory for my slowness woes was that I'm 
 using raw 
 > SQL for everything after getting sick of Django doing things weirdly 
 > (duplicating queries, adding bizarre things like "LIMIT 3453453" to 
 queries, 
 > not being able to JOIN things like I wanted etc). I'm not opposed to 
 going 
 > back to the ORM but need to know if this is where my bottleneck is. 
 > 
 > I've run a profiler against my code and the results are here: 
 > http://pastebin.com/raw.php?i=**HQf9bqGp
 >  
 > 
 > On my local machine (a not very powerful laptop) I see Django Debug 
 Toolbar 
 > load times of ~1900ms for my site homepage. This includes 168ms of db 
 calls 
 > (11 queries, which I think are fairly well-tuned, indexed, etc). I 
 cache 
 > pretty well on production but load times are still slow -- some of 
 this may 
 > be down to my cheap webhost, though. In my settings I enabled 
 > django.template.loaders.**cached.Loader but this doesn't seem to 
 make much 
 > difference. 
 > 
 > I'm having trouble seeing what the profiler results above are telling 
 me: 
 > can anyone shed any light? 

 Most of your time is spent in pprint, which was called over 14,000 
 times to generate your page. Over 2 seconds spent printing out debug. 
 This should be telling you "don't use pprint when you want to see how 
 fast your code is". 

 Cheers 

 Tom 

>>>
>>> Good point Tom, apologies. Here's the profiler results with DebugToolbar 
>>> switched off (and ordered by cumulative time, thanks Shawn!): 
>>> http://pastebin.com/raw.php?i=y3iP0cLn
>>>
>>> The top one is obviously my "home" method inside my views, but I'm 
>>> struggling to get more from it than that. Lots of template rendering, but 
>>> caching not helping here...?
>>>  
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-users...@googlegroups.com .
>>> To post to this group, send email to django...@googlegroups.com
>>> .
>>> Visit this group at http://groups.google.com/group/django-users?hl=en.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>  
>>>  
>>>
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Help interpreting profiler results (or: why is my app so slow?)

[Shawn Milochik]

Hit send too soon:

https://docs.djangoproject.com/en/dev/topics/cache/#template-fragment-caching


On Thu, Apr 18, 2013 at 8:39 AM, Shawn Milochik  wrote:

> Yes, it does look like template tags are taking some time. Is the page
> huge? Are you doing a ton of formatting? Is there something you could maybe
> move to server-side?
>
> Also, this might help with caching bits of your output:
>
>
> On Thu, Apr 18, 2013 at 6:17 AM, Matt Andrews wrote:
>
>>
>> On Thursday, 18 April 2013 10:45:40 UTC+1, Tom Evans wrote:
>>
>> On Wed, Apr 17, 2013 at 11:18 PM, Matt Andrews 
>>> wrote:
>>> > Hi all.
>>> >
>>> > Having performance problems with my Django app. I've posted here
>>> before
>>> > talking about this: one theory for my slowness woes was that I'm using
>>> raw
>>> > SQL for everything after getting sick of Django doing things weirdly
>>> > (duplicating queries, adding bizarre things like "LIMIT 3453453" to
>>> queries,
>>> > not being able to JOIN things like I wanted etc). I'm not opposed to
>>> going
>>> > back to the ORM but need to know if this is where my bottleneck is.
>>> >
>>> > I've run a profiler against my code and the results are here:
>>> > http://pastebin.com/raw.php?i=**HQf9bqGp
>>> >
>>> > On my local machine (a not very powerful laptop) I see Django Debug
>>> Toolbar
>>> > load times of ~1900ms for my site homepage. This includes 168ms of db
>>> calls
>>> > (11 queries, which I think are fairly well-tuned, indexed, etc). I
>>> cache
>>> > pretty well on production but load times are still slow -- some of
>>> this may
>>> > be down to my cheap webhost, though. In my settings I enabled
>>> > django.template.loaders.**cached.Loader but this doesn't seem to make
>>> much
>>> > difference.
>>> >
>>> > I'm having trouble seeing what the profiler results above are telling
>>> me:
>>> > can anyone shed any light?
>>>
>>> Most of your time is spent in pprint, which was called over 14,000
>>> times to generate your page. Over 2 seconds spent printing out debug.
>>> This should be telling you "don't use pprint when you want to see how
>>> fast your code is".
>>>
>>> Cheers
>>>
>>> Tom
>>>
>>
>> Good point Tom, apologies. Here's the profiler results with DebugToolbar
>> switched off (and ordered by cumulative time, thanks Shawn!):
>> http://pastebin.com/raw.php?i=y3iP0cLn
>>
>> The top one is obviously my "home" method inside my views, but I'm
>> struggling to get more from it than that. Lots of template rendering, but
>> caching not helping here...?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Help interpreting profiler results (or: why is my app so slow?)

[Shawn Milochik]

Yes, it does look like template tags are taking some time. Is the page
huge? Are you doing a ton of formatting? Is there something you could maybe
move to server-side?

Also, this might help with caching bits of your output:


On Thu, Apr 18, 2013 at 6:17 AM, Matt Andrews  wrote:

>
> On Thursday, 18 April 2013 10:45:40 UTC+1, Tom Evans wrote:
>
> On Wed, Apr 17, 2013 at 11:18 PM, Matt Andrews 
>> wrote:
>> > Hi all.
>> >
>> > Having performance problems with my Django app. I've posted here before
>> > talking about this: one theory for my slowness woes was that I'm using
>> raw
>> > SQL for everything after getting sick of Django doing things weirdly
>> > (duplicating queries, adding bizarre things like "LIMIT 3453453" to
>> queries,
>> > not being able to JOIN things like I wanted etc). I'm not opposed to
>> going
>> > back to the ORM but need to know if this is where my bottleneck is.
>> >
>> > I've run a profiler against my code and the results are here:
>> > http://pastebin.com/raw.php?i=**HQf9bqGp
>> >
>> > On my local machine (a not very powerful laptop) I see Django Debug
>> Toolbar
>> > load times of ~1900ms for my site homepage. This includes 168ms of db
>> calls
>> > (11 queries, which I think are fairly well-tuned, indexed, etc). I
>> cache
>> > pretty well on production but load times are still slow -- some of this
>> may
>> > be down to my cheap webhost, though. In my settings I enabled
>> > django.template.loaders.**cached.Loader but this doesn't seem to make
>> much
>> > difference.
>> >
>> > I'm having trouble seeing what the profiler results above are telling
>> me:
>> > can anyone shed any light?
>>
>> Most of your time is spent in pprint, which was called over 14,000
>> times to generate your page. Over 2 seconds spent printing out debug.
>> This should be telling you "don't use pprint when you want to see how
>> fast your code is".
>>
>> Cheers
>>
>> Tom
>>
>
> Good point Tom, apologies. Here's the profiler results with DebugToolbar
> switched off (and ordered by cumulative time, thanks Shawn!):
> http://pastebin.com/raw.php?i=y3iP0cLn
>
> The top one is obviously my "home" method inside my views, but I'm
> struggling to get more from it than that. Lots of template rendering, but
> caching not helping here...?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Help interpreting profiler results (or: why is my app so slow?)

[Matt Andrews]

On Thursday, 18 April 2013 10:45:40 UTC+1, Tom Evans wrote:

> On Wed, Apr 17, 2013 at 11:18 PM, Matt Andrews 
>  
> wrote: 
> > Hi all. 
> > 
> > Having performance problems with my Django app. I've posted here before 
> > talking about this: one theory for my slowness woes was that I'm using 
> raw 
> > SQL for everything after getting sick of Django doing things weirdly 
> > (duplicating queries, adding bizarre things like "LIMIT 3453453" to 
> queries, 
> > not being able to JOIN things like I wanted etc). I'm not opposed to 
> going 
> > back to the ORM but need to know if this is where my bottleneck is. 
> > 
> > I've run a profiler against my code and the results are here: 
> > http://pastebin.com/raw.php?i=HQf9bqGp 
> > 
> > On my local machine (a not very powerful laptop) I see Django Debug 
> Toolbar 
> > load times of ~1900ms for my site homepage. This includes 168ms of db 
> calls 
> > (11 queries, which I think are fairly well-tuned, indexed, etc). I cache 
> > pretty well on production but load times are still slow -- some of this 
> may 
> > be down to my cheap webhost, though. In my settings I enabled 
> > django.template.loaders.cached.Loader but this doesn't seem to make much 
> > difference. 
> > 
> > I'm having trouble seeing what the profiler results above are telling 
> me: 
> > can anyone shed any light? 
>
> Most of your time is spent in pprint, which was called over 14,000 
> times to generate your page. Over 2 seconds spent printing out debug. 
> This should be telling you "don't use pprint when you want to see how 
> fast your code is". 
>
> Cheers 
>
> Tom 
>

Good point Tom, apologies. Here's the profiler results with DebugToolbar 
switched off (and ordered by cumulative time, thanks Shawn!): 
http://pastebin.com/raw.php?i=y3iP0cLn

The top one is obviously my "home" method inside my views, but I'm 
struggling to get more from it than that. Lots of template rendering, but 
caching not helping here...?

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Help interpreting profiler results (or: why is my app so slow?)

[Tom Evans]

On Wed, Apr 17, 2013 at 11:18 PM, Matt Andrews  wrote:
> Hi all.
>
> Having performance problems with my Django app. I've posted here before
> talking about this: one theory for my slowness woes was that I'm using raw
> SQL for everything after getting sick of Django doing things weirdly
> (duplicating queries, adding bizarre things like "LIMIT 3453453" to queries,
> not being able to JOIN things like I wanted etc). I'm not opposed to going
> back to the ORM but need to know if this is where my bottleneck is.
>
> I've run a profiler against my code and the results are here:
> http://pastebin.com/raw.php?i=HQf9bqGp
>
> On my local machine (a not very powerful laptop) I see Django Debug Toolbar
> load times of ~1900ms for my site homepage. This includes 168ms of db calls
> (11 queries, which I think are fairly well-tuned, indexed, etc). I cache
> pretty well on production but load times are still slow -- some of this may
> be down to my cheap webhost, though. In my settings I enabled
> django.template.loaders.cached.Loader but this doesn't seem to make much
> difference.
>
> I'm having trouble seeing what the profiler results above are telling me:
> can anyone shed any light?

Most of your time is spent in pprint, which was called over 14,000
times to generate your page. Over 2 seconds spent printing out debug.
This should be telling you "don't use pprint when you want to see how
fast your code is".

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Help interpreting profiler results (or: why is my app so slow?)

[Shawn Milochik]

When you print those out with pstats, sort by cumtime.

cumtime = cumulative time

That will tell you almost exactly (and maybe really exactly) which code is
slow.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Help interpreting profiler results (or: why is my app so slow?)

[Matt Andrews]

Hi all.

Having performance problems with my Django app. I've posted here before 
talking about this: one theory for my slowness woes was that I'm using raw 
SQL for everything after getting sick of Django doing things weirdly 
(duplicating queries, adding bizarre things like "LIMIT 3453453" to 
queries, not being able to JOIN things like I wanted etc). I'm not opposed 
to going back to the ORM but need to know if this is where my bottleneck is.

I've run a profiler against my code and the results are here: 
http://pastebin.com/raw.php?i=HQf9bqGp

On my local machine (a not very powerful laptop) I see Django Debug Toolbar 
load times of ~1900ms for my site homepage. This includes 168ms of db calls 
(11 queries, which I think are fairly well-tuned, indexed, etc). I cache 
pretty well on production but load times are still slow -- some of this may 
be down to my cheap webhost, though. In my settings I 
enabled django.template.loaders.cached.Loader but this doesn't seem to make 
much difference.

I'm having trouble seeing what the profiler results above are telling me: 
can anyone shed any light?

If it helps at all, this is the homepage of my app: 
http://www.scenepointblank.com -- there's lot of HTML processing (stripping 
tags or truncating strings and parsing HTML etc). I use a couple of 
template tags quite frequently to re-use partial views (a news post, for 
example) so I can just pass it an object to work with. I don't know if this 
means a huge template overhead. I've explored replacing Django's templates 
with Jinja2 but fell at the first hurdle when none of the aforementioned 
template tags worked and I couldn't figure out how to port them to Jinja.

Any tips would be hugely appreciated.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.