Re: Performance Docs

[Daniele Procida]

On Fri, Sep 20, 2013, Alex Gaynor  wrote:

>I didn't follow this PR as it was in progress, but I just sat down and read
>them. I have to say I'm extremely saddened by
>https://docs.djangoproject.com/en/dev/topics/performance/#alternative-
>software-implementations.

See if  is less 
saddening then.

If not, can you suggest something that would work in the context of a document 
that's really intended to be an overview for users not already experienced in 
these matters (and in particular, ones who may be tempted to think "Python's 
slow! I need something faster").

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Performance Docs

[Tim Graham]

Aymeric's edits look good to me. Thanks for raising the issue, Alex.

On Saturday, September 21, 2013 4:31:33 AM UTC-4, Marc Tamlyn wrote:
>
> That looks better to me. Still conveys the same message with all the 
> caveats.
> On 21 sept. 2013, at 10:04, Marc Tamlyn > 
> wrote:
>
> Not sure where the compatibility comment came from as I think that's 
> clearly not right.
>
> The paragraph talks about both Pypy and C libraries. Historically, the 
> Python and C versions of simplejson have seriously diverged in unicode 
> handling; the compatibility concern is valid here. I can't comment on Pypy 
> as I haven't had the opportunity to use it yet.
>
> So yes it needs rewording as it is kinda misleading, but I don't want to 
> see an explicit recommendation of pypy in the Django docs (until such a 
> time as we do an 'advanced performance' guide!)
>
> This is really a discussion about the intended audience. Still, I prefer 
> to mention Pypy and explain when it's the appropriate answer (eg. after 
> you've activated the cached template loader) rather than ignore it entirely.
>
> Alex, Marc, do you think the wording here is a step in the right direction?
> https://github.com/django/django/pull/1652
>
> Daniele, Tim, I'd like to know your thoughts too, since you have spent 
> lots of time on the document already.
>
> Of course, readers come from widely different backgrounds, with different 
> definitions of "Django expertise". Some are always going to misunderstand 
> the document. Let's aim at getting the right message across for careful 
> readers, it'll be better than nothing!
>
> -- 
> Aymeric.
>
>  -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com .
> To post to this group, send email to django-d...@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Performance Docs

[Marc Tamlyn]

That looks better to me. Still conveys the same message with all the
caveats.
On 21 sept. 2013, at 10:04, Marc Tamlyn  wrote:

Not sure where the compatibility comment came from as I think that's
clearly not right.

The paragraph talks about both Pypy and C libraries. Historically, the
Python and C versions of simplejson have seriously diverged in unicode
handling; the compatibility concern is valid here. I can't comment on Pypy
as I haven't had the opportunity to use it yet.

So yes it needs rewording as it is kinda misleading, but I don't want to
see an explicit recommendation of pypy in the Django docs (until such a
time as we do an 'advanced performance' guide!)

This is really a discussion about the intended audience. Still, I prefer to
mention Pypy and explain when it's the appropriate answer (eg. after you've
activated the cached template loader) rather than ignore it entirely.

Alex, Marc, do you think the wording here is a step in the right direction?
https://github.com/django/django/pull/1652

Daniele, Tim, I'd like to know your thoughts too, since you have spent lots
of time on the document already.

Of course, readers come from widely different backgrounds, with different
definitions of "Django expertise". Some are always going to misunderstand
the document. Let's aim at getting the right message across for careful
readers, it'll be better than nothing!

-- 
Aymeric.

 --
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Performance Docs

[Aymeric Augustin]

On 21 sept. 2013, at 10:04, Marc Tamlyn  wrote:
> Not sure where the compatibility comment came from as I think that's clearly 
> not right.
> 
The paragraph talks about both Pypy and C libraries. Historically, the Python 
and C versions of simplejson have seriously diverged in unicode handling; the 
compatibility concern is valid here. I can't comment on Pypy as I haven't had 
the opportunity to use it yet.
> So yes it needs rewording as it is kinda misleading, but I don't want to see 
> an explicit recommendation of pypy in the Django docs (until such a time as 
> we do an 'advanced performance' guide!)
> 
This is really a discussion about the intended audience. Still, I prefer to 
mention Pypy and explain when it's the appropriate answer (eg. after you've 
activated the cached template loader) rather than ignore it entirely.

Alex, Marc, do you think the wording here is a step in the right direction?
https://github.com/django/django/pull/1652

Daniele, Tim, I'd like to know your thoughts too, since you have spent lots of 
time on the document already.

Of course, readers come from widely different backgrounds, with different 
definitions of "Django expertise". Some are always going to misunderstand the 
document. Let's aim at getting the right message across for careful readers, 
it'll be better than nothing!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Performance Docs

[Marc Tamlyn]

Not sure where the compatibility comment came from as I think that's
clearly not right.

As for the performance gain, it's true pypy won't give you much of a help
until you've fixed your code. Once you have a passably performant site you
wish to make very fast, it's a major player in that equation. But it can't
fix doing 3000 SQL queries on a page. We were trying to avoid 'advanced'
performance questions in thus document, focusing more in simple settings
and obvious code smells. Everything else you have to benchmark.

I don't see the argument that 'cpython is slow therefore use pypy' as any
different from 'Django templates are slow therefore use jinja2', which is
not an approach to performance we should be taking.

So yes it needs rewording as it is kinda misleading, but I don't want to
see an explicit recommendation of pypy in the Django docs (until such a
time as we do an 'advanced performance' guide!)

Marc
On 21 Sep 2013 05:36, "Russell Keith-Magee"  wrote:

> Hi Alex,
>
> I'll had a look at one of Daniele's initial drafts of this document -- I
> don't remember if this point was in the draft I saw, but if it was, I'll
> certainly wear some of the blame for this getting committed if it was.
>
> Regardless - having had my attention drawn to it specifically, I'll agree
> that it's not exactly well phrased advice.
>
> I completely agree that "Any gains are likely to be small, and
> compatibility issues are common." is unsubstantiated FUD.
>
> However, I think there's a good sentiment buried under the actual text --
> i.e., make sure you have your house in order, because it's much more likely
> the problem lies with bad database queries et al, rather than with Python
> itself being slow. The people who are likely to need this document aren't
> he seasoned web veterans -- they're the people coming to web programming
> (or programming at all) for the first time. Your conversations with people
> talking about "IO bound" applications indicates you're already at the
> experienced end of the  pool. They're not going to making mistakes that can
> be fixed with a select_related() or using memcache, which is the target
> audience for this document in my mind.
>
> So - I agree this last section needs work. The focus needs to be on "don't
> view PyPy as a silver bullet", not "PyPy is unstable and won't give you a
> performance gain".
>
> Yours,
> Russ Magee %-)
>
>
>
> On Sat, Sep 21, 2013 at 12:04 PM, Alex Gaynor wrote:
>
>> Hi all,
>>
>> I didn't follow this PR as it was in progress, but I just sat down and
>> read them. I have to say I'm extremely saddened by
>> https://docs.djangoproject.com/en/dev/topics/performance/#alternative-software-implementations.
>>  This seems like extremely bad advice to me.
>>
>> In my work I've had the opportunity to do performance work across a huge
>> number of applications, and there is an amazing corrolation between people
>> who insist their application is strictly IO bound, and applications which I
>> find 30% wins in just by switching to PyPy.
>>
>> PyPy is known to provide substantial performance gains on Django's
>> template engine (
>> http://speed.pypy.org/comparison/?exe=1%2BL%2Bdefault%2C2%2B472&ben=3&env=1&hor=true&bas=none&chart=normal+bars),
>> at the time of writing it's something like 10x.
>>
>> Further, the claim that "Any gains are likely to be small, and
>> compatibility issues are common." is pure FUD, and unsubstantiated.
>>
>> I'd strongly prefer we remove this from our docs, and if anything we
>> should be encouraging the opposite, people with performance issues should
>> be encouraged to consider things like PyPy, not hide from them.
>>
>> Alex
>>
>> --
>> "I disapprove of what you say, but I will defend to the death your right
>> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>> "The people's good is the highest law." -- Cicero
>> GPG Key fingerprint: 125F 5C67 DFE9 4084
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving 

Re: Performance Docs

[Russell Keith-Magee]

Hi Alex,

I'll had a look at one of Daniele's initial drafts of this document -- I
don't remember if this point was in the draft I saw, but if it was, I'll
certainly wear some of the blame for this getting committed if it was.

Regardless - having had my attention drawn to it specifically, I'll agree
that it's not exactly well phrased advice.

I completely agree that "Any gains are likely to be small, and
compatibility issues are common." is unsubstantiated FUD.

However, I think there's a good sentiment buried under the actual text --
i.e., make sure you have your house in order, because it's much more likely
the problem lies with bad database queries et al, rather than with Python
itself being slow. The people who are likely to need this document aren't
he seasoned web veterans -- they're the people coming to web programming
(or programming at all) for the first time. Your conversations with people
talking about "IO bound" applications indicates you're already at the
experienced end of the  pool. They're not going to making mistakes that can
be fixed with a select_related() or using memcache, which is the target
audience for this document in my mind.

So - I agree this last section needs work. The focus needs to be on "don't
view PyPy as a silver bullet", not "PyPy is unstable and won't give you a
performance gain".

Yours,
Russ Magee %-)



On Sat, Sep 21, 2013 at 12:04 PM, Alex Gaynor  wrote:

> Hi all,
>
> I didn't follow this PR as it was in progress, but I just sat down and
> read them. I have to say I'm extremely saddened by
> https://docs.djangoproject.com/en/dev/topics/performance/#alternative-software-implementations.
>  This seems like extremely bad advice to me.
>
> In my work I've had the opportunity to do performance work across a huge
> number of applications, and there is an amazing corrolation between people
> who insist their application is strictly IO bound, and applications which I
> find 30% wins in just by switching to PyPy.
>
> PyPy is known to provide substantial performance gains on Django's
> template engine (
> http://speed.pypy.org/comparison/?exe=1%2BL%2Bdefault%2C2%2B472&ben=3&env=1&hor=true&bas=none&chart=normal+bars),
> at the time of writing it's something like 10x.
>
> Further, the claim that "Any gains are likely to be small, and
> compatibility issues are common." is pure FUD, and unsubstantiated.
>
> I'd strongly prefer we remove this from our docs, and if anything we
> should be encouraging the opposite, people with performance issues should
> be encouraged to consider things like PyPy, not hide from them.
>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Performance Docs

[Donald Stufft]

On Sep 21, 2013, at 12:04 AM, Alex Gaynor  wrote:

> Hi all,
> 
> I didn't follow this PR as it was in progress, but I just sat down and read 
> them. I have to say I'm extremely saddened by 
> https://docs.djangoproject.com/en/dev/topics/performance/#alternative-software-implementations
>  . This seems like extremely bad advice to me.
> 
> In my work I've had the opportunity to do performance work across a huge 
> number of applications, and there is an amazing corrolation between people 
> who insist their application is strictly IO bound, and applications which I 
> find 30% wins in just by switching to PyPy.
> 
> PyPy is known to provide substantial performance gains on Django's template 
> engine 
> (http://speed.pypy.org/comparison/?exe=1%2BL%2Bdefault%2C2%2B472&ben=3&env=1&hor=true&bas=none&chart=normal+bars),
>  at the time of writing it's something like 10x.
> 
> Further, the claim that "Any gains are likely to be small, and compatibility 
> issues are common." is pure FUD, and unsubstantiated.
> 
> I'd strongly prefer we remove this from our docs, and if anything we should 
> be encouraging the opposite, people with performance issues should be 
> encouraged to consider things like PyPy, not hide from them.
> 
> Alex
> 
> -- 
> "I disapprove of what you say, but I will defend to the death your right to 
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.

I agree.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail

Performance Docs

[Alex Gaynor]

Hi all,

I didn't follow this PR as it was in progress, but I just sat down and read
them. I have to say I'm extremely saddened by
https://docs.djangoproject.com/en/dev/topics/performance/#alternative-software-implementations.
This seems like extremely bad advice to me.

In my work I've had the opportunity to do performance work across a huge
number of applications, and there is an amazing corrolation between people
who insist their application is strictly IO bound, and applications which I
find 30% wins in just by switching to PyPy.

PyPy is known to provide substantial performance gains on Django's template
engine (
http://speed.pypy.org/comparison/?exe=1%2BL%2Bdefault%2C2%2B472&ben=3&env=1&hor=true&bas=none&chart=normal+bars),
at the time of writing it's something like 10x.

Further, the claim that "Any gains are likely to be small, and
compatibility issues are common." is pure FUD, and unsubstantiated.

I'd strongly prefer we remove this from our docs, and if anything we should
be encouraging the opposite, people with performance issues should be
encouraged to consider things like PyPy, not hide from them.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.