[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Richard Damon]

On 12/13/19 12:27 AM, Steven D'Aprano wrote:
> On Wed, Dec 11, 2019 at 10:30:15PM -0500, Richard Damon wrote:
>
>> I will way that in my many years of
>> programming experience I can't think of any great cases where a language
>> as part of the language definition limited to 'size' of a program to
>> good effect,
> Good thing that's not what the PEP proposes then :-)
>
> The PEP talks about CPython implementation, not hard limits on all 
> Python compilers/interpreters.

Not the way I read the PEP.

It talks about changing the language.

Under 'Enforcement' it admits it can't 'force' other implementations but
says other implementations should generate the same errors unless doing
so hurts performance, so it is a language change, not just limits for a
given implementation.

As I said originally, I have no problem with the idea of creating a
variant implementation of CPython with this sort or documented limits to
demonstrate that it does provide a real benefit. We would then be in a
good place to determine the real costs and benefits, and one of the
branches might just wither and die because it isn't useful enough anymore.

>
>> and generally such limits relegate a language into being
>> seen as a 'toy language'. 
> The designers of C recognised that, in practice, compilers will have 
> limits. Rather than demanding "NO ARBITRARY LIMITS!!!" they specified 
> *minimum* levels for compliant compilers. Often quite low limits.
>
> Actual compilers often impose their own limits:
>
> https://gcc.gnu.org/onlinedocs/cpp/Implementation-limits.html
>
> https://www.cs.auckland.ac.nz/references/unix/digital/AQTLTBTE/DOCU_012.HTM
Yes, as I said above, there is a big definition between an
implementation of a language documenting some of its limits and the
language definition itself limiting what the language can do.
>
> So if Mark's proposal relegates Python to a "toy language", we'll be in 
> good company with other "toys" that have implementation, or even 
> language, limits:
>
>
> https://stackoverflow.com/questions/5689798/why-does-java-limit-the-size-of-a-method-to-65535-byte

There is a big difference between limiting the size of a single method
and limiting the total number of classes allowed in a program. The first
can be largely gotten around by refactoring the method to be implemented
in multiple methods, the latter can't be.

> https://web.archive.org/web/20160304023522/http://programmers.stackexchange.com/questions/108699/why-does-javas-collection-size-return-an-int
Yes, Java made a decision early in its life cycle to lock itself into
fixed sized types, and
> https://www.sqlite.org/limits.html
These are a very different type of limits. These are defines that the
programmer can change to establish what the various limits are. They can
be increased or decreased as desired by the programmer (with natural
upper limits based on the size of certain fundamental types).
>
>
> (If you read only one of those five links, please read the last.)
>
>
>> The biggest issue is that computers are
>> growing more powerful every day, and programs follow in getting bigger,
>> so any limit that we think of as more than sufficient soon becomes too
>> small (No one will need more than 640k of RAM).
> The beauty of this proposal is that since its an implementation limit, 
> not a law of nature, if and when computers get more powerful and 
> machines routinely have multiple zettabyte memories *wink* we can always 
> update the implementation.
>
> I'm not entirely being facetious here. There's a serious point. Unlike 
> languages like C and Java, where changes have to go through a long, 
> slow, difficult process, we have a much more agile process. If the PEP 
> is accepted, that doesn't mean we're locked into that decision for life. 
> Relaxing limits in the future doesn't break backwards compatibility.
>
> "What if computers get more powerful? Our limits will be obsolete!!!" 
> Naturally. Do you still expect to be using Python 3.9 in ten years time 
> with your fancy new uber-hyper-quantum ten thousand gigabyte computer? 
> Probably not. As hardware grows, and our needs grow, so can the 
> hypothetical limits.
>
> What is valuable are *concrete*, actual (not theoretical) examples of 
> where Mark's proposed limits are too low, so that we can get a realistic 
> view of where the potential tradeoffs lie:
>
> * lose this much functionality (code that did run, or might have run, 
>   but that won't run under the PEP)
>
> * in order to gain this much in safety, maintainability, efficiency.
>
>
> And before people jump down my throat again, I've already said -- on 
> multiple occassions -- that the onus is on Mark to demonstrate the 
> plausibility of any such gains.
>
>
> Thank you for reading :-)
>
As I said, the proposal as listed on Python.org is a language change,
not a proposal for an implementation that has a set of limits.

To do the former, really needs much more evidence to make it a
reasonable course of action, the latter, if 

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Steven D'Aprano]

On Wed, Dec 11, 2019 at 10:30:15PM -0500, Richard Damon wrote:

> I will way that in my many years of
> programming experience I can't think of any great cases where a language
> as part of the language definition limited to 'size' of a program to
> good effect,

Good thing that's not what the PEP proposes then :-)

The PEP talks about CPython implementation, not hard limits on all 
Python compilers/interpreters.


> and generally such limits relegate a language into being
> seen as a 'toy language'. 

The designers of C recognised that, in practice, compilers will have 
limits. Rather than demanding "NO ARBITRARY LIMITS!!!" they specified 
*minimum* levels for compliant compilers. Often quite low limits.

Actual compilers often impose their own limits:

https://gcc.gnu.org/onlinedocs/cpp/Implementation-limits.html

https://www.cs.auckland.ac.nz/references/unix/digital/AQTLTBTE/DOCU_012.HTM

So if Mark's proposal relegates Python to a "toy language", we'll be in 
good company with other "toys" that have implementation, or even 
language, limits:


https://stackoverflow.com/questions/5689798/why-does-java-limit-the-size-of-a-method-to-65535-byte

https://web.archive.org/web/20160304023522/http://programmers.stackexchange.com/questions/108699/why-does-javas-collection-size-return-an-int

https://www.sqlite.org/limits.html


(If you read only one of those five links, please read the last.)


> The biggest issue is that computers are
> growing more powerful every day, and programs follow in getting bigger,
> so any limit that we think of as more than sufficient soon becomes too
> small (No one will need more than 640k of RAM).

The beauty of this proposal is that since its an implementation limit, 
not a law of nature, if and when computers get more powerful and 
machines routinely have multiple zettabyte memories *wink* we can always 
update the implementation.

I'm not entirely being facetious here. There's a serious point. Unlike 
languages like C and Java, where changes have to go through a long, 
slow, difficult process, we have a much more agile process. If the PEP 
is accepted, that doesn't mean we're locked into that decision for life. 
Relaxing limits in the future doesn't break backwards compatibility.

"What if computers get more powerful? Our limits will be obsolete!!!" 
Naturally. Do you still expect to be using Python 3.9 in ten years time 
with your fancy new uber-hyper-quantum ten thousand gigabyte computer? 
Probably not. As hardware grows, and our needs grow, so can the 
hypothetical limits.

What is valuable are *concrete*, actual (not theoretical) examples of 
where Mark's proposed limits are too low, so that we can get a realistic 
view of where the potential tradeoffs lie:

* lose this much functionality (code that did run, or might have run, 
  but that won't run under the PEP)

* in order to gain this much in safety, maintainability, efficiency.


And before people jump down my throat again, I've already said -- on 
multiple occassions -- that the onus is on Mark to demonstrate the 
plausibility of any such gains.


Thank you for reading :-)



-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J4YNTDMOHXDR46JMRTX6BIZAWR4NL72Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Nick Coghlan]

On Thu., 12 Dec. 2019, 10:03 pm Mark Shannon,  wrote:

>
> Explicit limits are much easier to test. Does code outside the limit
> fail in the expected fashion and code just under the limit work correctly?
>
> What I want, is to allow more efficient use of resources without
> inconveniently low or unspecified limits. There will always be some
> limits on finite machines. If they aren't specified, they still exist,
> we just don't know what they are or how they will manifest themselves.
>

This is the main reason I'd like to see the PEP split into two distinct
aspects:

* articulating & testing specific limits (we already have some bigmem tests
that can only be run at full scale on server class machines with 20+ GiB of
RAM - don't forget that CPython runs on supercomputers, not just regular
machines).

* potentially lowering some specific limits in the CPython implementation

The first part is clearly valuable as a general capability, especially if
there's an API that lets different implementations report different limits.
For example, something like a "sys.implementation.limits" informational
API, modelled on the way that the top level "sys.implementation" data
structure already works.

Even when the limits are large, we have potential access to machines that
can accommodate them for testing purposes (e.g. I know at least one Linux
vendor that we're on good terms with that has servers with more than a
terabyte of RAM in their test fleet, and I presume other OS vendors and
cloud platform providers have similar capabilities).

By contrast, the second part really needs to be tackled on a case-by-case
basis, with a clear understanding of the kinds of failure modes we're
aiming to handle more gracefully, which kinds of performance improvements
we're aiming to enable, and which existing capabilities we'd be reducing.

Cheers,
Nick.


>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PTRLASLIT3ZPARSLDV4I5PWAGSVEYNBX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Mark Shannon]

On 11/12/2019 3:56 pm, Guido van Rossum wrote:
On Wed, Dec 11, 2019 at 2:14 AM Mark Shannon > wrote:



If the status quo were the result of considered decision, then it would
of course need considerable justification.
If, as is the case here, the status quo is a result of historical
accident and/or implementation details, then I think a weaker
justification is OK.


Whoa. The lack of limits in the status quo (no limits on various things 
except indirectly, through available memory) is most definitely the 
result of an intentional decision. "No arbitrary limits" was part of 
Python's initial design philosophy. We didn't always succeed (parse tree 
depth and call recursion depth come to mind) but that was definitely the 
philosophy. It was in contrast to other languages that did have 
arbitrary limits (e.g. Pascal's 255-char limit on strings, or C's 
machine-dependent integer size) and in several cases the implementation 
went through great lengths to avoid limits (e.g. we could have avoided a 
lot of dynamic memory (re)allocation if we'd limited line lengths or 
file sizes).


Sure, there are few *designed* in limits in Python at the moment. But in 
reality Python has lots of limits.


Because CPython is the reference implementation, Python is what CPython 
does; and CPython has lots of limits.


Classes.


One suggested way, by which a million classes might be created was using 
namedtuple:


>>> from collections import namedtuple
>>> l =  [ namedtuple(f"nt{i}", "a,b,c,d,e,f,g,h,i,j") for i in 
range(1000_000) ]


This takes 6Gb of resident memory on my machine (3.9alpha).
Most machines have roughly 4Gb per core (if you buy a bigger machine you 
get more CPUs and more RAM, roughly in that proportion).
So RAM effectively limits the number of classes to less than one million 
already (assuming you want to use your cores efficiently).
Imposing a limit on the number of classes and reducing memory footprint 
would allow more classes in practice, and would allow a lot more objects 
in more sensible programs.


Instructions per code-object


CPython will crash if this is in the 2**31 to 2**32 range as the 
compiler treats addresses as unsigned, but the interpreter treats them 
as signed.
Obviously this can be fixed, but it is an example of the sort of lurking 
bug that implicit limits can cause. These things are very expensive to 
test as you need a machine with hundreds of gigabytes of memory.




Explicit limits are much easier to test. Does code outside the limit 
fail in the expected fashion and code just under the limit work correctly?


What I want, is to allow more efficient use of resources without 
inconveniently low or unspecified limits. There will always be some 
limits on finite machines. If they aren't specified, they still exist, 
we just don't know what they are or how they will manifest themselves.


Cheers,
Mark.




You have an extreme need to justify why we should change now. "An 
infinite number of potential optimizations" does not cut it.

--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XVSDNGJ7YMUHG2XV5TGBY7WJ56KOEHDK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Richard Damon]

I've been restraining myself from commenting in this threads, as I don't
know enough details of the implementation of Python to make reasoned
arguments on how much such limits would help in the efficiency for
running a Python program.  I will way that in my many years of
programming experience I can't think of any great cases where a language
as part of the language definition limited to 'size' of a program to
good effect, and generally such limits relegate a language into being
seen as a 'toy language'. The biggest issue is that computers are
growing more powerful every day, and programs follow in getting bigger,
so any limit that we think of as more than sufficient soon becomes too
small (No one will need more than 640k of RAM).

I COULD easily see on the other hand, the language stating that some
implementations might impose certain limits like those being mentioned,
and maybe provide lower limits of those limits, that barring system
resource limits a program can expect to be able to use. If a program
needs higher limits, it can use a less limiting implementation. I could
see a fork of the current CPython implementation happening, where one
path keeps the current lack of limits, and another branch tries imposing
the limits to see how much benefit you really get. After seeing, one of
the forks might just die off as not being needed any more.

I would see those limits being added to the specific implementation that
provide identifiable advantage to making the system better. Being done,
at least initially, in a fork, there might not need to be a lot of
discussion on exactly what limits to try imposing, as 'Python' as a base
language wouldn't be changing, just some limits in a particular branch
implementation. it still might make a lot of sense for some discussion
to occur to find out if the limits being discussed are reasonable.

Perhaps the one big disadvantage of this idea is that the person wanting
to see what these limits might be able to do for the language would need
to be more involved in seeing it happen, as there won't be the easy
opportunity to propose a somewhat vague idea, get it approved, and have
someone else stuck with the job of getting it done.

-- 
Richard Damon
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QRP32EQPAQH53NR5NGHE55MCALEBJ5J6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Guido van Rossum]

On Wed, Dec 11, 2019 at 2:14 AM Mark Shannon  wrote:

>
> If the status quo were the result of considered decision, then it would
> of course need considerable justification.
> If, as is the case here, the status quo is a result of historical
> accident and/or implementation details, then I think a weaker
> justification is OK.
>

Whoa. The lack of limits in the status quo (no limits on various things
except indirectly, through available memory) is most definitely the result
of an intentional decision. "No arbitrary limits" was part of Python's
initial design philosophy. We didn't always succeed (parse tree depth and
call recursion depth come to mind) but that was definitely the philosophy.
It was in contrast to other languages that did have arbitrary limits (e.g.
Pascal's 255-char limit on strings, or C's machine-dependent integer size)
and in several cases the implementation went through great lengths to avoid
limits (e.g. we could have avoided a lot of dynamic memory (re)allocation
if we'd limited line lengths or file sizes).

You have an extreme need to justify why we should change now. "An infinite
number of potential optimizations" does not cut it.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EYV2CCV55UC7Z2EVG4DRTFUVH3WRHWNB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Mark Shannon]

On 10/12/2019 7:24 am, Kyle Stanley wrote:

Chris Angelico wrote:
 > We have people who believe that a bit
 > mask will slow things down, others who claim that improved cache
 > locality will speed things up, and Mark asks us to please justify our
 > objections with numbers. But surely it's up to Mark to show numbers
 > first?

+1. While it would be helpful for criticisms of the PEP to be more 
specific and provide more of a cost-benefit analysis, the burden of 
proof for demonstrating the benefits provided are ultimately up to the 
author(s) of the PEP. We require far more justification for making a 
change to impose the limits in the first place than we do for 
maintaining the status quo.


I agree that we require justification for changing the status quo.

If the status quo were the result of considered decision, then it would 
of course need considerable justification.
If, as is the case here, the status quo is a result of historical 
accident and/or implementation details, then I think a weaker 
justification is OK.




Personally, I don't think it would be reasonable to impose any of these 
limits without some form of concrete evidence that doing so will provide 
an improvement; in any combination of efficiency, performance, and/or 
security, as the PEP suggests would happen. But, I don't think exact 
numbers are needed. (I.E. adding these limits will improve performance 
across all Python programs by an average of 22.359%!).


On Mon, Dec 9, 2019 at 7:15 PM Chris Angelico > wrote:


On Tue, Dec 10, 2019 at 10:51 AM Steven D'Aprano
mailto:st...@pearwood.info>> wrote:
 >
 > On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote:
 > > > > On 09/12/2019 2:15 pm, Chris Angelico wrote:
 > > > You: "We should limit things. Stuff will be faster."
 > > > Others: "Really? Because bit masking is work. It'll be slower."
 >
 > I'm not an expert, but the impression I've got from various
discussions
 > on performance over the last few years is that the single biggest
 > bottleneck for CPU performance is memory locality. Cache misses
are so
 > expensive, and CPU instructions so fast, that memory locality is king
 > and the cost of bit masking is insignificant. In other words,
worrying
 > about the cost of bit masking in C code is so very 1990s.
 >
 > I could be wrong of course: I'm not an expert. And I don't think we
 > should take it for granted that this is the case, unless some
experts on
 > modern CPUs speak up and say that Mark is so obviously correct that a
 > demonstration is unnecessary.

And the speculation continues.

 > > > You: "Maybe we limit it somewhere else, whatever. It'll be
faster."
 >
 > That's a totally unfair and inaccurate representation of Mark's
 > position. The PEP doesn't say "let's put in arbitrary limits in
random
 > places for the lols", he proposed seven concrete limits and gave
reasons
 > for why he expects that they will improve memory efficiency, safety,
 > performance or all three.

*He expects that*. That's what I'm talking about. We have lots of
speculation and no evidence either way.

 > Having said that though, I think you are right that the PEP could do
 > with a bit more detail on the current status quo and existing limits,
 > and how the proposed changes will improve safety and memory use.

Exactly. Yes, I know that I massively oversimplified things in that
post. But you nonetheless acknowledge here that we are *still* quite
lacking in any actual evidence. We have people who believe that a bit
mask will slow things down, others who claim that improved cache
locality will speed things up, and Mark asks us to please justify our
objections with numbers. But surely it's up to Mark to show numbers
first?

ChrisA
___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/YN2BDJFPGHTEZJRDN7LXNDAGXWXB6XRA/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EDZOEKHLF52MG3JJ6HIVHLCGXQJRIM34/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Mark Shannon]

On 09/12/2019 10:29 pm, Gregory P. Smith wrote:
My overall problem with the PEP and a reason I'd reject it by default it 
that it is asking to pre-emptively impose limits on things, some of 
which we believe would cause problems to existing long running 
applications (limiting the total number of types for example), without 
having actually demonstrated practical benefits from such a change.  
Having an implementation that takes limits and runs with them to 
practical effect would provide motivation to consider adopting them.


Otherwise it doesn't feel like it solves a real problem, and could cause 
some existing users pain.  So what's our real motivation?


picking on some nits within the PEP 611 text:

 > Motivation: "poorly generated code could cause values to exceed 2^32"

Malicious or poorly generated code can always do anything so this isn't 
a strong motivation.  If you want a nice failure when we come near 
actual existing physical limits, that could be done today as a PR 
without a PEP.


If correctness is the motivation, we could be correct without changing 
the existing unchecked limits.


 > : "there are two ways to use a 32 bit refcount on a 64 bit machine. 
One is to limit each sub-interpreter to 32Gb of memory. The other is to 
use a saturating reference count, which would be a little bit slower, 
but allow unlimited memory allocation."


Please do not arbitrarily cap sub-interpreter memory usage to a small 
value.  32GiB is very small today.


I wasn't planning on doing so, merely pointing out that there is a very 
simple way to make a 32 bit reference count correct.
Limiting the number of objects or using a saturating count would be more 
sensible.




Also, at least one existing eternal refcount implementation I've had 
experience with demonstrated a notable hit to interpreter cpu 
performance as it required an additional test+branch within the 
extremely widely used Py_INCREF and Py_DECREF macro code.


Did they also reduce the size of object header, improving cache locality?
Do you have a reference?




-gps

On Mon, Dec 9, 2019 at 6:10 AM Mark Shannon > wrote:


Hi everyone,

Thanks again for all your comments on PEP 611.

I would like to ask a favour; please be more specific in your comments.

Ideally state which part of the PEP you are disagreeing with and why
you
disagree with the relevant part of the rationale/motivation.

Also, when asking for limits to be raised or removed entirely, could
you
state what you perceive to be the costs and benefits of larger limits.
What do you believe is an acceptable cost in memory or runtime for
larger limits?

For example, you might say that the limit of one million lines of code
per module is too small, and that it is worth a small, say 1%,
impact on
speed to allow a larger of limit of 100 million.

If you believe a limit would have no cost, then please give a
explanation of why that is so.

Merely saying that you would like a larger limit is pointless.
If there were no cost to arbitrarily large limits, then I wouldn't have
proposed the PEP in the first place.

Bear in mind that the costs of higher limits are paid by everyone, but
the benefits are gained by few.

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/ZXDAJKRVSF6UUE5UEPE5PMXYXOLJ5A4V/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3AUUCWOSLFXUR3DAU7LJ5RHODRIETVR7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Rhodri James]

On 10/12/2019 00:01, Chris Angelico wrote:

Exactly. Yes, I know that I massively oversimplified things in that
post. But you nonetheless acknowledge here that we are*still*  quite
lacking in any actual evidence. We have people who believe that a bit
mask will slow things down, others who claim that improved cache
locality will speed things up, and Mark asks us to please justify our
objections with numbers. But surely it's up to Mark to show numbers
first?


+1

--
Rhodri James *-* Kynesim Ltd
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Y2SKFQBYZRWWLRWD35M6NKSFQWDMECZP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Kyle Stanley]

Chris Angelico wrote:
> We have people who believe that a bit
> mask will slow things down, others who claim that improved cache
> locality will speed things up, and Mark asks us to please justify our
> objections with numbers. But surely it's up to Mark to show numbers
> first?

+1. While it would be helpful for criticisms of the PEP to be more specific
and provide more of a cost-benefit analysis, the burden of proof for
demonstrating the benefits provided are ultimately up to the author(s) of
the PEP. We require far more justification for making a change to impose
the limits in the first place than we do for maintaining the status quo.

Personally, I don't think it would be reasonable to impose any of these
limits without some form of concrete evidence that doing so will provide an
improvement; in any combination of efficiency, performance, and/or
security, as the PEP suggests would happen. But, I don't think exact
numbers are needed. (I.E. adding these limits will improve performance
across all Python programs by an average of 22.359%!).

On Mon, Dec 9, 2019 at 7:15 PM Chris Angelico  wrote:

> On Tue, Dec 10, 2019 at 10:51 AM Steven D'Aprano 
> wrote:
> >
> > On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote:
> > > > > On 09/12/2019 2:15 pm, Chris Angelico wrote:
> > > > You: "We should limit things. Stuff will be faster."
> > > > Others: "Really? Because bit masking is work. It'll be slower."
> >
> > I'm not an expert, but the impression I've got from various discussions
> > on performance over the last few years is that the single biggest
> > bottleneck for CPU performance is memory locality. Cache misses are so
> > expensive, and CPU instructions so fast, that memory locality is king
> > and the cost of bit masking is insignificant. In other words, worrying
> > about the cost of bit masking in C code is so very 1990s.
> >
> > I could be wrong of course: I'm not an expert. And I don't think we
> > should take it for granted that this is the case, unless some experts on
> > modern CPUs speak up and say that Mark is so obviously correct that a
> > demonstration is unnecessary.
>
> And the speculation continues.
>
> > > > You: "Maybe we limit it somewhere else, whatever. It'll be faster."
> >
> > That's a totally unfair and inaccurate representation of Mark's
> > position. The PEP doesn't say "let's put in arbitrary limits in random
> > places for the lols", he proposed seven concrete limits and gave reasons
> > for why he expects that they will improve memory efficiency, safety,
> > performance or all three.
>
> *He expects that*. That's what I'm talking about. We have lots of
> speculation and no evidence either way.
>
> > Having said that though, I think you are right that the PEP could do
> > with a bit more detail on the current status quo and existing limits,
> > and how the proposed changes will improve safety and memory use.
>
> Exactly. Yes, I know that I massively oversimplified things in that
> post. But you nonetheless acknowledge here that we are *still* quite
> lacking in any actual evidence. We have people who believe that a bit
> mask will slow things down, others who claim that improved cache
> locality will speed things up, and Mark asks us to please justify our
> objections with numbers. But surely it's up to Mark to show numbers
> first?
>
> ChrisA
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/YN2BDJFPGHTEZJRDN7LXNDAGXWXB6XRA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EDZOEKHLF52MG3JJ6HIVHLCGXQJRIM34/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Chris Angelico]

On Tue, Dec 10, 2019 at 10:51 AM Steven D'Aprano  wrote:
>
> On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote:
> > > > On 09/12/2019 2:15 pm, Chris Angelico wrote:
> > > You: "We should limit things. Stuff will be faster."
> > > Others: "Really? Because bit masking is work. It'll be slower."
>
> I'm not an expert, but the impression I've got from various discussions
> on performance over the last few years is that the single biggest
> bottleneck for CPU performance is memory locality. Cache misses are so
> expensive, and CPU instructions so fast, that memory locality is king
> and the cost of bit masking is insignificant. In other words, worrying
> about the cost of bit masking in C code is so very 1990s.
>
> I could be wrong of course: I'm not an expert. And I don't think we
> should take it for granted that this is the case, unless some experts on
> modern CPUs speak up and say that Mark is so obviously correct that a
> demonstration is unnecessary.

And the speculation continues.

> > > You: "Maybe we limit it somewhere else, whatever. It'll be faster."
>
> That's a totally unfair and inaccurate representation of Mark's
> position. The PEP doesn't say "let's put in arbitrary limits in random
> places for the lols", he proposed seven concrete limits and gave reasons
> for why he expects that they will improve memory efficiency, safety,
> performance or all three.

*He expects that*. That's what I'm talking about. We have lots of
speculation and no evidence either way.

> Having said that though, I think you are right that the PEP could do
> with a bit more detail on the current status quo and existing limits,
> and how the proposed changes will improve safety and memory use.

Exactly. Yes, I know that I massively oversimplified things in that
post. But you nonetheless acknowledge here that we are *still* quite
lacking in any actual evidence. We have people who believe that a bit
mask will slow things down, others who claim that improved cache
locality will speed things up, and Mark asks us to please justify our
objections with numbers. But surely it's up to Mark to show numbers
first?

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YN2BDJFPGHTEZJRDN7LXNDAGXWXB6XRA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Steven D'Aprano]

On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote:
> > > On 09/12/2019 2:15 pm, Chris Angelico wrote:
> > You: "We should limit things. Stuff will be faster."
> > Others: "Really? Because bit masking is work. It'll be slower."

I'm not an expert, but the impression I've got from various discussions 
on performance over the last few years is that the single biggest 
bottleneck for CPU performance is memory locality. Cache misses are so 
expensive, and CPU instructions so fast, that memory locality is king 
and the cost of bit masking is insignificant. In other words, worrying 
about the cost of bit masking in C code is so very 1990s.

I could be wrong of course: I'm not an expert. And I don't think we 
should take it for granted that this is the case, unless some experts on 
modern CPUs speak up and say that Mark is so obviously correct that a 
demonstration is unnecessary.

Failing that, I think it would be good for the PEP to give some evidence 
that more compact data structures will be faster despite the cost of bit 
masking: references to reliable sources (let's say, a similar discussion 
on the Linux kernel mailing list), or some sample C code that skeptics 
can run.


> > You: "Maybe we limit it somewhere else, whatever. It'll be faster."

That's a totally unfair and inaccurate representation of Mark's 
position. The PEP doesn't say "let's put in arbitrary limits in random 
places for the lols", he proposed seven concrete limits and gave reasons 
for why he expects that they will improve memory efficiency, safety, 
performance or all three.


> > Others: "Will it? How much faster?"
> > You: "It'll be faster."
> 
> Mark, possibly you want to re-frame the PEP to be more like "this is
> good for correctness and enabling robust reasoning about the
> interpreter, which has a variety of benefits (and possibly speed will
> be one of them eventually)"? My impression is that you see speedups as
> a secondary motivation, while other people are getting the impression
> that speedups are the entire motivation, so one way or the other the
> text is confusing people.

In fairness to Mark, the PEP is *already* framed in terms of safety and 
memory use. One has to read all the way to literally the very last 
sentence of the "Motivation" sentence before "speed" is mentioned:

"There is also the potential for a more efficient instruction format, 
speeding up interpreter dispatch."


To get to that point, you have to go past:

- at least five references to efficiency;
- at least four references to safety (vulnerability to overflow);
- a reference to memory locality being the bottleneck on modern CPUs.


In the rest of the document, Mark refers to speed three more times:

- "instructions can be represented in a compact 64 bit form allowing
  very fast passes over the instruction sequence."

- "Using super-instructions would make that the 32 bit format
  almost as compact as the 16 bit format, and significantly faster."

- and a reference to optionally switching to a saturating ref count 
  mechanism, which would allow the use of unlimited memory but would
  be "a little bit slower".

(The above counts are based on Mark's initial post to the list, archived 
here:

https://mail.python.org/archives/list/python-dev@python.org/thread/QM4QUJOBQORN5WJ2WZ4RVSHSQG52VKCQ/#QM4QUJOBQORN5WJ2WZ4RVSHSQG52VKCQ

and may not reflect the current state of the PEP.)

Having said that though, I think you are right that the PEP could do 
with a bit more detail on the current status quo and existing limits, 
and how the proposed changes will improve safety and memory use.



-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2YRB5T2WTTFMXKRK5SCGTDN7OG4VL7XW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Gregory P. Smith]

My overall problem with the PEP and a reason I'd reject it by default it
that it is asking to pre-emptively impose limits on things, some of which
we believe would cause problems to existing long running applications
(limiting the total number of types for example), without having actually
demonstrated practical benefits from such a change.  Having an
implementation that takes limits and runs with them to practical effect
would provide motivation to consider adopting them.

Otherwise it doesn't feel like it solves a real problem, and could cause
some existing users pain.  So what's our real motivation?

picking on some nits within the PEP 611 text:

> Motivation: "poorly generated code could cause values to exceed 2^32"

Malicious or poorly generated code can always do anything so this isn't a
strong motivation.  If you want a nice failure when we come near actual
existing physical limits, that could be done today as a PR without a PEP.

If correctness is the motivation, we could be correct without changing the
existing unchecked limits.

> : "there are two ways to use a 32 bit refcount on a 64 bit machine. One
is to limit each sub-interpreter to 32Gb of memory. The other is to use a
saturating reference count, which would be a little bit slower, but allow
unlimited memory allocation."

Please do not arbitrarily cap sub-interpreter memory usage to a small
value.  32GiB is very small today.

Also, at least one existing eternal refcount implementation I've had
experience with demonstrated a notable hit to interpreter cpu performance
as it required an additional test+branch within the extremely widely used
Py_INCREF and Py_DECREF macro code.

-gps

On Mon, Dec 9, 2019 at 6:10 AM Mark Shannon  wrote:

> Hi everyone,
>
> Thanks again for all your comments on PEP 611.
>
> I would like to ask a favour; please be more specific in your comments.
>
> Ideally state which part of the PEP you are disagreeing with and why you
> disagree with the relevant part of the rationale/motivation.
>
> Also, when asking for limits to be raised or removed entirely, could you
> state what you perceive to be the costs and benefits of larger limits.
> What do you believe is an acceptable cost in memory or runtime for
> larger limits?
>
> For example, you might say that the limit of one million lines of code
> per module is too small, and that it is worth a small, say 1%, impact on
> speed to allow a larger of limit of 100 million.
>
> If you believe a limit would have no cost, then please give a
> explanation of why that is so.
>
> Merely saying that you would like a larger limit is pointless.
> If there were no cost to arbitrarily large limits, then I wouldn't have
> proposed the PEP in the first place.
>
> Bear in mind that the costs of higher limits are paid by everyone, but
> the benefits are gained by few.
>
> Cheers,
> Mark.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ZXDAJKRVSF6UUE5UEPE5PMXYXOLJ5A4V/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EYXC7CDHVC4W2MHY5BTKAVH4IFSFV6LO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Nathaniel Smith]

> > On 09/12/2019 2:15 pm, Chris Angelico wrote:
> You: "We should limit things. Stuff will be faster."
> Others: "Really? Because bit masking is work. It'll be slower."
> You: "Maybe we limit it somewhere else, whatever. It'll be faster."
> Others: "Will it? How much faster?"
> You: "It'll be faster."

Mark, possibly you want to re-frame the PEP to be more like "this is
good for correctness and enabling robust reasoning about the
interpreter, which has a variety of benefits (and possibly speed will
be one of them eventually)"? My impression is that you see speedups as
a secondary motivation, while other people are getting the impression
that speedups are the entire motivation, so one way or the other the
text is confusing people.

In particular, right now the most detailed example is the compacted
object header bit, which makes it a magnet for critique. Also, I don't
understand how this idea would work at all :-). So I'd either remove
it or else make it more detailed, one or the other.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5NQWKURB45J5NIZWD5R7GDTEDAGY7U7S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Paddy McCarthy]

Mark wrote, in reply to me:

> On 09/12/2019 3:01 pm, Paddy McCarthy wrote:
> > "Bear in mind that the costs of higher limits are paid by everyone, but
> > the benefits are gained by few."
> >
> > Is there some evidence for the above statement? One of the issues with C
> > programming is the need for limits Its hard to choose the right limits
> > leading to so many failures. You propose limiting Python for performance
> > gains but loose in ease of use and security.
>
> How does this make anything less secure?
>
Many basic exploits have components taking advantage of limits known as
buffer overflows

Suppose we can improve the interpreter in such a way that it takes
> advantage of the one million bytecode per code object and speeds it up
> by 0.1% (I can do a *lot* better than that).
> What would the performance gain save globally? $100k, $1M p.a.?
> What does it cost? A few developer hours.


You are looking to unproven execution gains. Adding an arbitrary limit
might make it 0.1% faster to get the *wrong *result .
Optimisation isn't just speedThe less mind-clutter spent on such* micro
optimisations* might leave one able to think of a better algorithm
or a better data structure that could give thousands of % speedups.
You should only quote C or C++ compiler limitations as a warning of what *not
*to do. Python wins by being different; by being more
orthoganal; by having *less *arbitrary restrictions.

Case in point:
http://paddy3118.blogspot.com/2019/11/quasi-random-sobol-sequence.html
Itook decade old C++ library, converted it
to straight python, Python allowed me to see the opportunity for
improvement leading to a Python version that is half the speed of
the C++ code and generates points on demand. That's how Python wins big
time. Having to worry about stifling limits is the wrong
direction for the language to take.

On Mon, 9 Dec 2019 at 21:26, David Mertz  wrote:

> I think a much more sensible approach than mandating a limit because "who
> knows, it might speed something up" would be finding the speedup first.
>
> Probably that means one limit at a time too. E.g. maybe some patch imposes
> the 1 million LOC limit and demonstrates a repeatable benchmark improvement
> because of some coffee change that allows. That cold be interesting.
>
> Even them, I wouldn't want some arbitrary round number just for its own
> sake. For example, if taking 10 bits away from a word that holds a LOC
> index speeds something up, make the LOC limit 4,194,304 (2**22)... Maybe.
> If you only need 9 bits for that use, make the limit twice as much.
>
> On Mon, Dec 9, 2019, 12:53 PM Mark Shannon  wrote:
>
>>
>> On 09/12/2019 2:15 pm, Chris Angelico wrote:
>> > On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon  wrote:
>> >> Bear in mind that the costs of higher limits are paid by everyone, but
>> >> the benefits are gained by few.
>> >
>> > Can we get some stats on what the costs of higher limits (or having no
>> > limit at all) is? Or, putting it the other way: Since CPython
>> > currently does not enforce limits, what are the benefits of placing
>> > those limits? Merely having a limit doesn't in itself give a benefit
>> > (and is a (minor) cost); it would help the discussion if we knew
>> > exactly WHAT the costs of the higher limits were.
>>
>> Given there is an infinite number of potential optimizations that it
>> would enable, it is a bit hard to put a number on it :)
>>
>> It is also impossible to put precise numbers on the speedups of a
>> particular optimizations unless it is implemented. I suspect no one is
>> going to do that unless paid to do so, or are guaranteed that the work
>> won't be thrown away because the PEP is rejected.
>>
>> Cheers,
>> Mark.
>>
>> >
>> > ChrisA
>> > ___
>> > Python-Dev mailing list -- python-dev@python.org
>> > To unsubscribe send an email to python-dev-le...@python.org
>> > https://mail.python.org/mailman3/lists/python-dev.python.org/
>> > Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNGC5DZXOAR555ZUGNJGWSFV3QI/
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>> >
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/BWTOEHKYZ3NES4XPZA7QA57UHTVGRMQZ/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/MAOXN66JYZAN4L4ER6SH43TP3YMKHCVY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[David Mertz]

I think a much more sensible approach than mandating a limit because "who
knows, it might speed something up" would be finding the speedup first.

Probably that means one limit at a time too. E.g. maybe some patch imposes
the 1 million LOC limit and demonstrates a repeatable benchmark improvement
because of some coffee change that allows. That cold be interesting.

Even them, I wouldn't want some arbitrary round number just for its own
sake. For example, if taking 10 bits away from a word that holds a LOC
index speeds something up, make the LOC limit 4,194,304 (2**22)... Maybe.
If you only need 9 bits for that use, make the limit twice as much.

On Mon, Dec 9, 2019, 12:53 PM Mark Shannon  wrote:

>
> On 09/12/2019 2:15 pm, Chris Angelico wrote:
> > On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon  wrote:
> >> Bear in mind that the costs of higher limits are paid by everyone, but
> >> the benefits are gained by few.
> >
> > Can we get some stats on what the costs of higher limits (or having no
> > limit at all) is? Or, putting it the other way: Since CPython
> > currently does not enforce limits, what are the benefits of placing
> > those limits? Merely having a limit doesn't in itself give a benefit
> > (and is a (minor) cost); it would help the discussion if we knew
> > exactly WHAT the costs of the higher limits were.
>
> Given there is an infinite number of potential optimizations that it
> would enable, it is a bit hard to put a number on it :)
>
> It is also impossible to put precise numbers on the speedups of a
> particular optimizations unless it is implemented. I suspect no one is
> going to do that unless paid to do so, or are guaranteed that the work
> won't be thrown away because the PEP is rejected.
>
> Cheers,
> Mark.
>
> >
> > ChrisA
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNGC5DZXOAR555ZUGNJGWSFV3QI/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/BWTOEHKYZ3NES4XPZA7QA57UHTVGRMQZ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MAOXN66JYZAN4L4ER6SH43TP3YMKHCVY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Chris Angelico]

On Tue, Dec 10, 2019 at 4:56 AM Mark Shannon  wrote:
>
>
> On 09/12/2019 2:15 pm, Chris Angelico wrote:
> > On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon  wrote:
> >> Bear in mind that the costs of higher limits are paid by everyone, but
> >> the benefits are gained by few.
> >
> > Can we get some stats on what the costs of higher limits (or having no
> > limit at all) is? Or, putting it the other way: Since CPython
> > currently does not enforce limits, what are the benefits of placing
> > those limits? Merely having a limit doesn't in itself give a benefit
> > (and is a (minor) cost); it would help the discussion if we knew
> > exactly WHAT the costs of the higher limits were.
>
> Given there is an infinite number of potential optimizations that it
> would enable, it is a bit hard to put a number on it :)

Then pick one optimization and show us what it can do. Let's say you
pick "1,000,000 lines in a single source file", and show that this
limit (or rather, changing code to *assume* this limit) improves the
performance of, say, printing of tracebacks. A few benchmarks showing
how small files benefit from the banning of gigantic files would be
extremely helpful.

> It is also impossible to put precise numbers on the speedups of a
> particular optimizations unless it is implemented. I suspect no one is
> going to do that unless paid to do so, or are guaranteed that the work
> won't be thrown away because the PEP is rejected.

Fair, but even imprecise numbers would be extremely helpful. At the
moment, what we have is (paraphrased):

You: "We should limit things. Stuff will be faster."
Others: "Really? Because bit masking is work. It'll be slower."
You: "Maybe we limit it somewhere else, whatever. It'll be faster."
Others: "Will it? How much faster?"
You: "It'll be faster."

I'm not asking you to prove that CPython 3.9 could be exactly 7.183%
faster with these changes, just some indication that this will
actually improve things. You've asked us to say what we believe is "an
acceptable cost in memory or runtime", but at the moment, we're not
paying ANY cost or getting ANY benefit, so it's really hard to
randomly guess at things.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J6VCCAIMUWM3L2H63TECFVF5OOHUSOFA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Mark Shannon]

On 09/12/2019 2:15 pm, Chris Angelico wrote:

On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon  wrote:

Bear in mind that the costs of higher limits are paid by everyone, but
the benefits are gained by few.


Can we get some stats on what the costs of higher limits (or having no
limit at all) is? Or, putting it the other way: Since CPython
currently does not enforce limits, what are the benefits of placing
those limits? Merely having a limit doesn't in itself give a benefit
(and is a (minor) cost); it would help the discussion if we knew
exactly WHAT the costs of the higher limits were.


Given there is an infinite number of potential optimizations that it 
would enable, it is a bit hard to put a number on it :)


It is also impossible to put precise numbers on the speedups of a 
particular optimizations unless it is implemented. I suspect no one is 
going to do that unless paid to do so, or are guaranteed that the work 
won't be thrown away because the PEP is rejected.


Cheers,
Mark.



ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNGC5DZXOAR555ZUGNJGWSFV3QI/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BWTOEHKYZ3NES4XPZA7QA57UHTVGRMQZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Rhodri James]

On 09/12/2019 14:05, Mark Shannon wrote:

I would like to ask a favour; please be more specific in your comments.

Ideally state which part of the PEP you are disagreeing with and why you 
disagree with the relevant part of the rationale/motivation.


This is rather hard to do when I don't think you have made the case for 
limits yet.  Certainly not convincingly.  Without that argument, all I 
can really do is say that I think you're wrong.


--
Rhodri James *-* Kynesim Ltd
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XTN4GMCV4PIOEPCFFUPJDP25RYHA4SIX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Joao S. O. Bueno]

Also,
I've pointed at two of those limits that worry me some - namely
number of classes and number of co-routines, and had not
seen so far any feedback even whether they actually make sense.

A simple data analysis tasks that create a co-routine per row, and submit
those
for workers with more than 1 million rows is something that  just works
today, and is not even
a burden in a desktop machine would hit such limits.

Is there an easy way to locate the data-structures that would be changed on
cpython
that would need to be changed to limit classes and co-routines? I'd like to
have at
least a little more concrete  idea of what could possibly be optimized if
they were capped.


  js
 -><-

On Mon, 9 Dec 2019 at 11:22, Chris Angelico  wrote:

> On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon  wrote:
> > Bear in mind that the costs of higher limits are paid by everyone, but
> > the benefits are gained by few.
>
> Can we get some stats on what the costs of higher limits (or having no
> limit at all) is? Or, putting it the other way: Since CPython
> currently does not enforce limits, what are the benefits of placing
> those limits? Merely having a limit doesn't in itself give a benefit
> (and is a (minor) cost); it would help the discussion if we knew
> exactly WHAT the costs of the higher limits were.
>
> ChrisA
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNGC5DZXOAR555ZUGNJGWSFV3QI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2AVYQHPY3VSW6O7OFAMD2PKSXBS2YQH4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[David Mertz]

I have not seen any benchmarks supporting the claim that proposed limits
would ACTUALLY have any positive performance effect. While some of the
claims of performance gain are *plausible*, I also would not be very
surprised if some of them caused performance loss.

For example, requiring bit masking to get information from words feels like
it probably costs more than using extra words in the first place.

On Mon, Dec 9, 2019, 9:08 AM Mark Shannon  wrote:

> Hi everyone,
>
> Thanks again for all your comments on PEP 611.
>
> I would like to ask a favour; please be more specific in your comments.
>
> Ideally state which part of the PEP you are disagreeing with and why you
> disagree with the relevant part of the rationale/motivation.
>
> Also, when asking for limits to be raised or removed entirely, could you
> state what you perceive to be the costs and benefits of larger limits.
> What do you believe is an acceptable cost in memory or runtime for
> larger limits?
>
> For example, you might say that the limit of one million lines of code
> per module is too small, and that it is worth a small, say 1%, impact on
> speed to allow a larger of limit of 100 million.
>
> If you believe a limit would have no cost, then please give a
> explanation of why that is so.
>
> Merely saying that you would like a larger limit is pointless.
> If there were no cost to arbitrarily large limits, then I wouldn't have
> proposed the PEP in the first place.
>
> Bear in mind that the costs of higher limits are paid by everyone, but
> the benefits are gained by few.
>
> Cheers,
> Mark.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ZXDAJKRVSF6UUE5UEPE5PMXYXOLJ5A4V/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KARK535W7DRYJRXRXNSMTG6NV5XXX7QK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Chris Angelico]

On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon  wrote:
> Bear in mind that the costs of higher limits are paid by everyone, but
> the benefits are gained by few.

Can we get some stats on what the costs of higher limits (or having no
limit at all) is? Or, putting it the other way: Since CPython
currently does not enforce limits, what are the benefits of placing
those limits? Merely having a limit doesn't in itself give a benefit
(and is a (minor) cost); it would help the discussion if we knew
exactly WHAT the costs of the higher limits were.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNGC5DZXOAR555ZUGNJGWSFV3QI/
Code of Conduct: http://python.org/psf/codeofconduct/