Re: [Python-ideas] Objectively Quantifying Readability

[Dan Sommers]

On Tue, 01 May 2018 10:42:53 +1000, Steven D'Aprano wrote:

> - people are not good judges of readability;

WTF?  By definition, people are the *only* judge of readability.¹

I happen to be an excellent judge of whether a given block of code is
readable to me.

OTOH, if you mean is that I'm a bad judge of what makes code readable to
you, and that you're a bad judge of what makes code readable to me, then
I agree.  :-)

Dan

¹ Well, okay, compilers will tell you that your code is unreadable, but
they're known to be fairly pedantic.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Objectively Quantifying Readability

[Matt Arcidy]

On Mon, Apr 30, 2018 at 5:42 PM, Steven D'Aprano  wrote:
> On Mon, Apr 30, 2018 at 11:28:17AM -0700, Matt Arcidy wrote:
>
>> A study has been done regarding readability in code which may serve as
>> insight into this issue. Please see page 8, fig 9 for a nice chart of
>> the results, note the negative/positive coloring of the correlations,
>> grey/black respectively.
>
> Indeed. It seems that nearly nothing is positively correlated to
> increased readability, aside from comments, blank lines, and (very
> weakly) arithmetic operators. Everything else hurts readability.
>
> The conclusion here is that if you want readable source code, you should
> remove the source code. *wink*
>
>
>> https://web.eecs.umich.edu/~weimerw/p/weimer-tse2010-readability-preprint.pdf
>>
>> The criteria in the paper can be applied to assess an increase or
>> decrease in readability between current and proposed changes.  Perhaps
>> even an automated tool could be implemented based on agreed upon
>> criteria.
>
>
> That's a really nice study, and thank you for posting it. There are some
> interested observations here, e.g.:
>
> - line length is negatively correlated with readability;
>
>   (a point against those who insist that 79 character line
>   limits are irrelevant since we have wide screens now)
>
> - conventional measures of complexity do not correlate well
>   with readability;
>
> - length of identifiers was strongly negatively correlated
>   with readability: long, descriptive identifier names hurt
>   readability while short variable names appeared to make
>   no difference;
>
>   (going against the common wisdom that one character names
>   hurt readability -- maybe mathematicians got it right
>   after all?)
>
> - people are not good judges of readability;
>
> but I think the practical relevance here is very slim. Aside from
> questions about the validity of the study (it is only one study, can the
> results be replicated, do they generalise beyond the narrowly self-
> selected set of university students they tested?) I don't think that it
> gives us much guidance here. For example:

I don't propose to replicate correlations.  I don't see these
"standard" terminal conclusions as forgone when looking at the idea as
a whole, as opposed to the paper itself, which they may be.  The
authors crafted a method and used that method to do a study, I like
the method.  I think I can agree with your point about the study
without validating or invalidating the method.

>
> 1. The study is based on Java, not Python.

An objective measure can be created, based or not on the paper's
parameters, but it clearly would need to be adjusted to a specific
language, good point.

Here "objective" does not mean "with absolute correctness" but
"applied the same way such that a 5 is always a 5, and a 5 is always
greater than 4."  I think I unfortunately presented the paper as "The
Answer" in my initial email, but I didn't intend to say "each detail
must be implemented as is" but more like "this is a thing which can be
done."  Poor job on my part.

>
> 2. It looks at a set of pre-existing source code features.
>
> 3. It gives us little or no help in deciding whether new syntax will or
> won't affect readability: the problem of *extrapolation* remains.
>
> (If we know that, let's say, really_long_descriptive_identifier_names
> hurt readability, how does that help us judge whether adding a new kind
> of expression will hurt or help readability?)

A new feature can remove symbols or add them.  It can increase density
on a line, or remove it.  It can be a policy of variable naming, or it
can specifically note that variable naming has no bearing on a new
feature.  This is not limited in application.  It's just scoring.
When anyone complains about readability, break out the scoring
criteria and assess how good the _comparative_ readability claim is:
2 vs 10?  4 vs 5?  The arguments will no longer be singularly about
"readability," nor will the be about the question of single score for
a specific statement.  The comparative scores of applying the same
function over two inputs gives a relative difference.  This is what
measures do in the mathematical sense.

Maybe the "readability" debate then shifts to arguing criteria: "79?
Too long in your opinion!"  A measure will at least break
"readability" up and give some structure to that argument.  Right now
"readability" comes up and starts a semi-polite flame war.  Creating
_any_ criteria will help narrow the scope of the argument.

Even when someone writes perfectly logical statements about it, the
statements can always be dismantled because it's based in opinion.
By creating a measure, objectivity is forced.  While each criterion is
less or more subjective, the measure will be applied objectively to
each instance, the same way, to get a score.

>
> 4. The authors themselves warn that it is descriptive, not prescriptive,
> for example replacing long identifier names with randomly selected 

Re: [Python-ideas] A "local" pseudo-function

[Tim Peters]

[MRAB]
>> Any binding that's not specified as local is bound in the parent scope:

[Tim]
> Reverse-engineering the example following, is this a fair way of
> making that more precise?
>
> Given a binding-target name N in scope S, N is bound in scope T, where
> T is the closest-containing scope (which may be S itself) for which T
> is either
>
> 1. established by a "local:" block that declares name N
>
> or
>
> 2. not established by a "local: block

Here's an example where I don't know what the consequences of "the
rules" should be:

def f():
a = 10
local a:
def showa():
print("a is", a)
showa() # 10
a = 20
showa() # 20
a = 30
showa() # 10

The comments show what the output would be under the "nothing about
scope rules change" meaning.  They're all obvious (since there is is
no new scope then - it's all function-local).

But under the other meaning ...?

The twist here is that `def` is an executable statement in Python, and
is a "binding site" for the name of the function being defined.  So
despite that `showa` appears to be defined in a new nested lexical
scope, it's _actually_ bound as a function-local name.  That's bound
to be surprising to people from other languages:  "I defined it in a
nested lexical scope, but the name is still visible after that scope
ends?".

I don't know what the first `showa()` is intended to do.  Presumably
`a` is unbound at the start of the new nested scope?  So raises
NameError?  If so, comment that line out so we can make progress ;-)

It seems clear that the second `showa()` will display 20 under any reading.

But the third?  Now we're out of the `local a:` scope, but call a
function whose textual definition was inside that scope.  What does
`showa()` do now to find a's value?  f's local `a` had nothing to do
with the `a` in the nested scope, so presumably it shouldn't display
10 now.  What should it do?
Does the final state of the nested scope's locals need to preserved so
that showa() can display 30 instead?  Or ...?

Not necessarily complaining  - just fleshing out a bit my earlier
claim that a world of semantics need to be defined if anything akin to
a "real scope" is desired.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A "local" pseudo-function

[Tim Peters]

[MRAB ]
> ...
> The intention is that only the specified names are local.
>
> After all, what's the point of specifying names after the 'local' if _any_
> binding in the local scope was local?

Don't look at me ;-)  In the absence of use cases, I don't know which
problem(s) you're trying to solve.  All the use cases I've looked at
are adequately addressed by having some spelling of "local:" change
nothing at all about Python's current scope rules.  If you have uses
in mind that require more than just that, I'd need to see them.

>> ...
>> If you agree that makes the feature probably unusable, you don't get
>> off the hook by saying "no, unlike current Python scopes, binding
>> sites have nothing to do with what's local to a new lexical scope
>> introduced by 'local:'".  The same question raised in the example
>> above doesn't go away:  in which scope(s) are 'r1' and 'r2' to be
>> bound?

> Any binding that's not specified as local is bound in the parent scope:

Reverse-engineering the example following, is this a fair way of
making that more precise?

Given a binding-target name N in scope S, N is bound in scope T, where
T is the closest-containing scope (which may be S itself) for which T
is either

1. established by a "local:" block that declares name N

or

2. not established by a "local: block


> local b:
> local c:
> c = 0 # Bound in the "local c" scope.

By clause #1 above, "c" is declared in the starting "local:" scope.

> b = 0 # Bound in the "local b" scope.

By clause #1 above, after searching one scope up to find `b` declared
in a "local:" scope

> a = 0 # Bound in the main scope (function, global, whatever)

By clause #2 above, after searching two scopes up and not finding any
"local:" scope declaring name "a".  By your original "the parent
scope", I would have expected this be bound in the "local b:" scope
(which is the parent scope of the "local c:" scope).

So that's _a_ possible answer.  It's not like the scoping rules in any
other language I'm aware of, but then Python's current scoping rules
are unique too.

Are those useful rules?  Optimal?  The first thing that popped into
your head?  The third?  Again I'd need to see use cases to even begin
to guess.

I agree it's well defined, though, and so miles ahead of most ideas ;-)

...

>> Note:  most of this doesn't come up in most other languages because
>> they require explicitly declaring in which scope a name lives.
>> Python's "infer that in almost all cases instead from examining
>> binding sites" has consequences.

> Would/should it be possible to inject a name into a local scope? You can't
> inject into a function scope, and names in a function scope can be
> determined statically (they are allocated slots), so could the same kind of
> thing be done for names in a local scope?

Sorry, I'm unclear on what "inject a name into a local scope" means.
Do you mean at runtime?

In Python's very early days, all scope namespaces were implemented as
Python dicts, and you could mutate those at runtime any way you liked.
Name lookup first tried the "local" dict ("the" because local scopes
didn't nest), then the "global" dict, then the "builtin" dict.  Names
could be added or removed from any of those at will.

People had a lot of fun playing with that, but nobody seriously
complained as that extreme flexibility was incrementally traded away
for faster runtime.

So now take it as given that the full set of names in a local scope
must be determinable at compile-time (modulo whatever hacks may still
exist to keep old "from module import *" code working - if any still
do exist).  I don't believe CPython has grown any optimizations
preventing free runtime mutation of global (module-level) or builtin
namespace mappings, but I may be wrong about that.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A "local" pseudo-function

[Stephen J. Turnbull]

Tim Peters writes:

 > Meaning "wonderfully clear" to the compiler, not necessarily to you
 > ;-)

Is the compiler African or European (perhaps even Dutch)?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Objectively Quantifying Readability

[Chris Angelico]

On Tue, May 1, 2018 at 10:42 AM, Steven D'Aprano  wrote:
> The conclusion here is that if you want readable source code, you should
> remove the source code. *wink*

That's more true than your winky implies. Which is more readable: a
Python function, or the disassembly of its corresponding byte-code?
Which is more readable: a "for item in items:" loop, or one that
iterates up to the length of the list and subscripts it each time? The
less code it takes to express the same concept, the easier it is to
read - and to debug.

So yes, if you want readable source code, you should have less source code.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Objectively Quantifying Readability

[Steven D'Aprano]

On Mon, Apr 30, 2018 at 11:28:17AM -0700, Matt Arcidy wrote:

> A study has been done regarding readability in code which may serve as
> insight into this issue. Please see page 8, fig 9 for a nice chart of
> the results, note the negative/positive coloring of the correlations,
> grey/black respectively.

Indeed. It seems that nearly nothing is positively correlated to 
increased readability, aside from comments, blank lines, and (very 
weakly) arithmetic operators. Everything else hurts readability.

The conclusion here is that if you want readable source code, you should 
remove the source code. *wink*

 
> https://web.eecs.umich.edu/~weimerw/p/weimer-tse2010-readability-preprint.pdf
> 
> The criteria in the paper can be applied to assess an increase or
> decrease in readability between current and proposed changes.  Perhaps
> even an automated tool could be implemented based on agreed upon
> criteria.


That's a really nice study, and thank you for posting it. There are some 
interested observations here, e.g.:

- line length is negatively correlated with readability;

  (a point against those who insist that 79 character line 
  limits are irrelevant since we have wide screens now)

- conventional measures of complexity do not correlate well
  with readability;

- length of identifiers was strongly negatively correlated
  with readability: long, descriptive identifier names hurt
  readability while short variable names appeared to make
  no difference;

  (going against the common wisdom that one character names
  hurt readability -- maybe mathematicians got it right 
  after all?)

- people are not good judges of readability;

but I think the practical relevance here is very slim. Aside from 
questions about the validity of the study (it is only one study, can the 
results be replicated, do they generalise beyond the narrowly self- 
selected set of university students they tested?) I don't think that it 
gives us much guidance here. For example:

1. The study is based on Java, not Python.

2. It looks at a set of pre-existing source code features.

3. It gives us little or no help in deciding whether new syntax will or 
won't affect readability: the problem of *extrapolation* remains.

(If we know that, let's say, really_long_descriptive_identifier_names 
hurt readability, how does that help us judge whether adding a new kind 
of expression will hurt or help readability?)

4. The authors themselves warn that it is descriptive, not prescriptive, 
for example replacing long identifier names with randomly selected two 
character names is unlikely to be helpful.

5. The unfamiliarity affect: any unfamiliar syntax is going to be less 
readable than a corresponding familiar syntax.


It's a great start to the scientific study of readability, but I don't 
think it gives us any guidance with respect to adding new features.


> Opinions about readability can be shifted from:
>  - "Is it more or less readable?"
> to
>  - "This change exceeds a tolerance for levels of readability given
> the scope of the change."

One unreplicated(?) study for readability of Java snippets does not give 
us a metric for predicting the readability of new Python syntax. While 
it would certainly be useful to study the possibly impact of adding new 
features to a language, the authors themselves state that this study is 
just "a framework for conducting such experiments".

Despite the limitations of the study, it was an interesting read, thank 
you for posting it.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A "local" pseudo-function

[MRAB]

On 2018-04-30 21:41, Tim Peters wrote:

[MRAB ]
> I think it should be lexically scoped.

That's certainly arguable, but that's why I like real-code driven
design:  abstract arguments never end, and often yield a dubious
in-real-life outcome after one side is worn out and the other side
"wins" by attrition ;-)


> The purpose of 'local' would be to allow you to use a name that _might_ be
> used elsewhere.
>
> The problem with a dynamic scope is that you might call some global function
> from within the local scope, but find that it's "not working correctly"
> because you've inadvertently shadowed a name that the function refers to.

Already explained at excessive length that there's nothing akin to
"dynamic scopes" here, except that both happen to restore a previous
binding at times.  That's a shallow coincidence.  It's no more
"dynamic scope" than that

 savea = a
 try:
 a += 1
 f(a)
 finally:
 a = savea

is "dynamic scoping".  It's merely saving/restoring a binding across a
block of code.


> Imagine, in a local scope, that you call a global function that calls 'len',
> but you've shadowed 'len'...

I'm not clear on whether you picked the name of a builtin to make a
subtle point not spelled out, but I don't think it matters.
Regardless of whether `len` refers to a builtin or a module global
inside your global function now, the _current_

def f():
  len = 12
  global_function()

has no effect at all on the binding of `len` seen inside
`global_function`.  Because my understanding of "local:" changes
absolutely nothing about Python's current scope rules, it's
necessarily the case that the same would be true in:

def f():
 local len:
 len = 12
 call_something()

The only difference from current semantics is that if

 print(len)

were added after the `local:` block, UnboundLocalError would be raised
(restoring the state of the function-local-with-or-without-'local:'
`len` to what it was before the block).

To have "local:" mean "new nested lexical scope" instead requires
specifying a world of semantics that haven't even been mentioned yet.

In Python today, in the absence of `global` and `nonlocal`
declarations, the names local to a given lexical scope are determined
entirely by analyzing binding sites.  If you intend something other
than that, then it needs to be spelled out.  But if you intend to keep
"and names appearing in binding sites are also local to the new
lexical scope", I expect that's pretty much useless.   For example,

 def f():
 ...
 local x. y:
 x = a*b
 y = a/b
 r1, r2 = x+y, x-y

That is, the programmer surely doesn't _intend_ to throw away r1 and
r2 when the block ends.  If they have to add a

 nonlocal r1, r2

declaration at the top of the block, maybe it would work as intended.
But it still wouldn't work unless `r1` and `r2` _also_ appeared in
binding sites in an enclosing lexical scope.  If they don't, you'd get
a compile-time error like

The intention is that only the specified names are local.

After all, what's the point of specifying names after the 'local' if 
_any_ binding in the local scope was local?



SyntaxError: no binding for nonlocal 'r1' found

To be more accurate, the message should really say "sorry, but I have
no idea in which scope you _intend_ 'r1' to live, because the only way
I could know that is to find a binding site for 'r1', and I can't find
any except inside _this_ scope containing the 'nonlocal'".  But that's
kind of wordy ;-)

If you agree that makes the feature probably unusable, you don't get
off the hook by saying "no, unlike current Python scopes, binding
sites have nothing to do with what's local to a new lexical scope
introduced by 'local:'".  The same question raised in the example
above doesn't go away:  in which scope(s) are 'r1' and 'r2' to be
bound?

Any binding that's not specified as local is bound in the parent scope:

local b:
    local c:
    c = 0 # Bound in the "local c" scope.
        b = 0 # Bound in the "local b" scope.
    a = 0 # Bound in the main scope (function, global, whatever)

There's more than one plausible answer to that, but in the absence of
real use cases how can they be judged?

Under "'local:' changes nothing at all about Python's scopes", the
answer is obvious:  `r1` and `r2` are function locals (exactly the
same as if "local:" hadn't been used).  There's nothing new about
scope to learn, and the code works as intended on the first try ;-)
Of course "local:" would be a misleading name for the construct,
though.

Going back to your original example, where a global (not builtin)
"len" was intended:

 def f():
 global len  # LINE ADDED HERE
 local len:
 len = 12
 global_function()

yes, in _that_ case the global-or-builtin "len" seen inside
`global_function` would change under my "nothing about scoping
changes" 

Re: [Python-ideas] A "local" pseudo-function

[Tim Peters]

[MRAB ]
> I think it should be lexically scoped.

That's certainly arguable, but that's why I like real-code driven
design:  abstract arguments never end, and often yield a dubious
in-real-life outcome after one side is worn out and the other side
"wins" by attrition ;-)


> The purpose of 'local' would be to allow you to use a name that _might_ be
> used elsewhere.
>
> The problem with a dynamic scope is that you might call some global function
> from within the local scope, but find that it's "not working correctly"
> because you've inadvertently shadowed a name that the function refers to.

Already explained at excessive length that there's nothing akin to
"dynamic scopes" here, except that both happen to restore a previous
binding at times.  That's a shallow coincidence.  It's no more
"dynamic scope" than that

savea = a
try:
a += 1
f(a)
finally:
a = savea

is "dynamic scoping".  It's merely saving/restoring a binding across a
block of code.


> Imagine, in a local scope, that you call a global function that calls 'len',
> but you've shadowed 'len'...

I'm not clear on whether you picked the name of a builtin to make a
subtle point not spelled out, but I don't think it matters.
Regardless of whether `len` refers to a builtin or a module global
inside your global function now, the _current_

def f():
 len = 12
 global_function()

has no effect at all on the binding of `len` seen inside
`global_function`.  Because my understanding of "local:" changes
absolutely nothing about Python's current scope rules, it's
necessarily the case that the same would be true in:

def f():
local len:
len = 12
call_something()

The only difference from current semantics is that if

print(len)

were added after the `local:` block, UnboundLocalError would be raised
(restoring the state of the function-local-with-or-without-'local:'
`len` to what it was before the block).

To have "local:" mean "new nested lexical scope" instead requires
specifying a world of semantics that haven't even been mentioned yet.

In Python today, in the absence of `global` and `nonlocal`
declarations, the names local to a given lexical scope are determined
entirely by analyzing binding sites.  If you intend something other
than that, then it needs to be spelled out.  But if you intend to keep
"and names appearing in binding sites are also local to the new
lexical scope", I expect that's pretty much useless.   For example,

def f():
...
local x. y:
x = a*b
y = a/b
r1, r2 = x+y, x-y

That is, the programmer surely doesn't _intend_ to throw away r1 and
r2 when the block ends.  If they have to add a

nonlocal r1, r2

declaration at the top of the block, maybe it would work as intended.
But it still wouldn't work unless `r1` and `r2` _also_ appeared in
binding sites in an enclosing lexical scope.  If they don't, you'd get
a compile-time error like

SyntaxError: no binding for nonlocal 'r1' found

To be more accurate, the message should really say "sorry, but I have
no idea in which scope you _intend_ 'r1' to live, because the only way
I could know that is to find a binding site for 'r1', and I can't find
any except inside _this_ scope containing the 'nonlocal'".  But that's
kind of wordy ;-)

If you agree that makes the feature probably unusable, you don't get
off the hook by saying "no, unlike current Python scopes, binding
sites have nothing to do with what's local to a new lexical scope
introduced by 'local:'".  The same question raised in the example
above doesn't go away:  in which scope(s) are 'r1' and 'r2' to be
bound?

There's more than one plausible answer to that, but in the absence of
real use cases how can they be judged?

Under "'local:' changes nothing at all about Python's scopes", the
answer is obvious:  `r1` and `r2` are function locals (exactly the
same as if "local:" hadn't been used).  There's nothing new about
scope to learn, and the code works as intended on the first try ;-)
Of course "local:" would be a misleading name for the construct,
though.

Going back to your original example, where a global (not builtin)
"len" was intended:

def f():
global len  # LINE ADDED HERE
local len:
len = 12
global_function()

yes, in _that_ case the global-or-builtin "len" seen inside
`global_function` would change under my "nothing about scoping
changes" reading, but would not under your reading.

That's worth _something_ ;-)  But without fleshing out the rules for
all the other stuff (like which scope(s) own r1 and r2 in the example
above) I can't judge whether it's worth enough to care.  All the
plausibly realistic use cases I've considered don't _really_ want a
full-blown new scope (just robust save/restore for a handful of
explicitly given names), and the example just above is contrived in
comparison.  Nobody types "global len" unless they 

Re: [Python-ideas] A "local" pseudo-function

[Tim Peters]

[Tim, still trying to define `iseven` in one statement]

> even = (lambda n: n == 0 or odd(n-1))
> odd = (lambda n: False if n == 0 else even(n-1))
> iseven = lambda n: even(n)
...

> [and the last attempt failed because a LOAD_GLOBAL was generated
>instead of a more-general runtime lookup]

So if I want LOAD_FAST instead, that has to be forced, leading to a
one-statement definition that's wonderfully clear:

iseven = lambda n: (
lambda
n=n,
even = (lambda n, e, o: n == 0 or o(n-1, e, o)),
odd = (lambda n, e, o: False if n == 0 else e(n-1, e, o)):
   even(n, even, odd)
)()

Meaning "wonderfully clear" to the compiler, not necessarily to you ;-)

Amusingly enough, that's a lot like the tricks we sometimes did in
Python's early days, before nested scopes were added, to get recursive
- or mutually referring - non-global functions to work at all.  That
is, since their names weren't in any scope they could access, their
names had to be passed as arguments (or, sure, stuffed in globals -
but that would have been ugly ;-) ).
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Objectively Quantifying Readability

[Matt Arcidy]

The number and type of arguments about readability as a justification,
or an opinion, or an opinion about an opinion seems counter-productive
to reaching conclusions efficiently.  I think they are very important
either way, but the justifications used are not rich enough in
information to be very useful.

A study has been done regarding readability in code which may serve as
insight into this issue. Please see page 8, fig 9 for a nice chart of
the results, note the negative/positive coloring of the correlations,
grey/black respectively.

https://web.eecs.umich.edu/~weimerw/p/weimer-tse2010-readability-preprint.pdf

The criteria in the paper can be applied to assess an increase or
decrease in readability between current and proposed changes.  Perhaps
even an automated tool could be implemented based on agreed upon
criteria.

Opinions about readability can be shifted from:
 - "Is it more or less readable?"
to
 - "This change exceeds a tolerance for levels of readability given
the scope of the change."

Still need to argue "exceeds ...given" and "tolerance", but at least
the readability score exists, and perhaps over time there will be
consensus.

Note this is an attempt to impact rhetoric in PEP (or other)
discussions, not about  supporting a particular PEP.  Please consider
this food for thought to increase the efficacy and efficiency of PEP
discussions, not as commenting on any specific current discussion,
which of course is the motivating factor of sending this email today.

I think using python implicitly accepts readability being partially
measurable, even if the resolution of current measure is too low to
capture the changes currently being discussed.  Perhaps using this
criteria can increase that resolution.

Thank you,
- Matt
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A "local" pseudo-function

[MRAB]

On 2018-04-30 03:49, Tim Peters wrote:

[Soni L. ]

That ain't shadow. That is dynamic scoping.


I don't believe either term is technically accurate, but don't really care.



Shadowing is something different:

def f():
a = 42
def g():
print(a)
local a:
a = 43
g()
g()

should print "42" both times, *if it's lexically scoped*.


Why?  The `local` statement, despite its name, and casual talk about
it, isn't _intended_ to create a new scope in any technically accurate
sense.  I think what it is intended to do has been adequately
explained several times already.  The `a` in `a = 42` is intended to
be exactly the same as the `a` in `a = 43`, changing nothing at all
about Python's lexical scoping rules.  It is _the_ `a` local to `f`.
If lexical scoping hasn't changed one whit (and it hasn't), the code
_must_ print 43 first.  Same as if `local a:` were replaced by `if
True:`.  `local` has no effect on a's value until the new "scope"
_ends_, and never any effect at all on a's visibility (a's "scope").
"local" or not, after `a = 43` there is no scope anywhere, neither
lexical nor dynamic, in which `a` is still bound to 42.  _The_ value
of `a` is 43 then, `local` or not.


[snip]
I think it should be lexically scoped.

The purpose of 'local' would be to allow you to use a name that _might_ 
be used elsewhere.


The problem with a dynamic scope is that you might call some global 
function from within the local scope, but find that it's "not working 
correctly" because you've inadvertently shadowed a name that the 
function refers to.


Imagine, in a local scope, that you call a global function that calls 
'len', but you've shadowed 'len'...

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A "local" pseudo-function

[Steven D'Aprano]

On Sun, Apr 29, 2018 at 01:36:31PM +1000, Chris Angelico wrote:

[...]
> > While I started off with Python 1.5, I wasn't part of the discussions
> > about nested scopes. But I'm astonished that you say that nested scopes
> > were controversial. *Closures* I would completely believe, but mere
> > lexical scoping? Astonishing.
> 
> I'm not sure how you can distinguish them:

Easily. Pascal, for example, had lexical scoping back in the 1970s, but 
no closures. I expect Algol probably did also, even earlier. So they are 
certainly distinct concepts.

(And yes, Pascal functions were *not* first class values.)


> What you expect here is lexical scope, yes. But if you have lexical 
> scope with no closures, the inner function can ONLY be used while its 
> calling function is still running. What would happen if you returned 
> 'inner' uncalled, and then called the result? How would it resolve the 
> name 'x'?

Failing to resolve 'x' is an option. It would simply raise NameError, 
the same as any other name lookup that doesn't find the name.

Without closures, we could say that names are looked up in the following 
scopes:

# inner function, called from inside the creating function
(1) Local to inner.
(2) Local to outer (nonlocal).
(3) Global (module).
(4) Builtins.


If you returned the inner function and called it from the outside of the 
factory function which created it, we could use the exact same name 
resolution order, except that (2) the nonlocals would be either absent 
or empty.

Obviously that would limit the usefulness of factory functions, but 
since Python 1.5 didn't have closures anyway, that would have been no 
worse that what we had.

Whether you have a strict Local-Global-Builtins scoping, or lexical 
scoping without closures, the effect *outside* of the factory function 
is the same. But at least with the lexical scoping option, inner 
functions can call each other while still *inside* the factory.

(Another alternative would be dynamic scoping, where nonlocals becomes 
the environment of the caller.)


> I can't even begin to imagine what lexical scope would do in
> the absence of closures. At least, not with first-class functions.

What they would likely do is raise NameError, of course :-)

An inner function that didn't rely on its surrounding nonlocal scope 
wouldn't be affected. Or if you had globals that happened to match the 
names it was relying on, the function could still work. (Whether it 
would work as you expected is another question.)

I expect that given the lack of closures, the best approach is to simply 
make sure that any attempt to refer to a nonlocal from the surrounding 
function outside of that function would raise NameError.

All in all, closures are much better :-)



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A "local" pseudo-function

[Tim Peters]

[Tim, on differences among Scheme-ish `let`, `let*`, `letrec` binding]
> ...
>
> You can play, if you like, with trying to define the `iseven` lambda
> here in one line by nesting lambdas to define `even` and `odd` as
> default arguments:
>
> even = (lambda n: n == 0 or odd(n-1))
> odd = (lambda n: False if n == 0 else even(n-1))
> iseven = lambda n: even(n)
>
> Scheme supplies `letrec` for when "mutually recursive" bindings are
> needed.  In Python that distinction isn't nearly as evidently needed,
> because Python's idea of closures doesn't capture all the bindings
> currently in effect,.  For example, when `odd` above is defined,
> Python has no idea at all what the then-current binding for `even` is
> - it doesn't even look for "even" until the lambda is _executed_.

Just FYI, I still haven't managed to do it as 1-liner (well, one
statement).  I expected the following would work, but it doesn't :-)

iseven = lambda n: (
   lambda n=n, \
  even = (lambda n: n == 0 or odd(n-1)), \
  odd = (lambda n: False if n == 0 else even(n-1)):
   even(n))()

Ugly and obscure, but why not?  In the inner lambda, `n`, `even`, and
`odd` are all defined in its namespace, so why does it fail anyway?

>>> iseven(6)
Traceback (most recent call last):
...
iseven(6)
...
even(n))()
...
even = (lambda n: n == 0 or odd(n-1)), \
NameError: name 'odd' is not defined

Because while Python indeed doesn't capture the current binding for
`odd` when the `even` lambda is compiled, it _does_ recognize that the
name `odd` is not local to the lambda at compile-time, so generates a
LOAD_GLOBAL opcode to retrieve `odd`'s binding at runtime.  But there
is no global `odd` (well, unless there is - and then there's no
guessing what the code would do).

For `even` to know at compile-time that `odd` will show up later in
its enclosing lambda's arglist requires that Python do `letrec`-style
binding instead.  For a start ;-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A "local" pseudo-function

[Steven D'Aprano]

On Sat, Apr 28, 2018 at 11:20:52PM -0500, Tim Peters wrote:
> [Tim]
> >> Enormously harder to implement than binding expressions, and the
> >> latter (to my eyes) capture many high-value use cases "good enough".
> 
> [Steven D'Aprano ]
> > And yet you're suggesting an alternative which is harder and more
> > confusing.
> 
> I am?  I said at the start that it was a "brain dump".  It was meant
> to be a point of discussion for anyone interested.  I also said I was
> more interested in real use cases from real code than in debating, and
> I wasn't lying about that ;-)

Ah, my mistake... I thought you were advocating sublocal scopes as well 
as just brain dumping the idea.


[...]
> > Even when I started, as a novice programmer who wouldn't have recognised
> > the term "lexical scoping" if it fell on my head from a great height, I
> > thought it was strange that inner functions couldn't see their
> > surrounding function's variables. Nested scopes just seemed intuitively
> > obvious: if a function sees the variables in the module surrounding it,
> > then it should also see the variables in any function surrounding it.
> >
> > This behaviour in Python 1.5 made functions MUCH less useful:
[...]
> > I think it is fair to say that inner functions in Python 1.5 were
> > crippled to the point of uselessness.
> 
> I don't think that's fair to say.  A great many functions are in fact
> ... functions ;-)  That is, they compute a result from the arguments
> passed to them.

Sure, but not having access to the surrounding function scope means that 
inner functions couldn't call other inner functions. Given:

def outer():
def f(): ...
def g(): ...

f cannot call g, or vice versa.

I think it was a noble experiment in how minimal you could make scoping 
rules and still be usable, but I don't think that particular aspect was 
a success.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Sublocal scoping at its simplest

[Chris Angelico]

On Mon, Apr 30, 2018 at 6:20 PM, Matt Arcidy  wrote:
> Does this mean indentation is now a scope, or colons are a scope, or is that
> over simplifying?

No, no, and yes. This is JUST about the 'except' statement, which
currently has the weird effect of unbinding the name it just bound.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Sublocal scoping at its simplest

[Matt Arcidy]

On Sat, Apr 28, 2018, 20:16 Chris Angelico  wrote:

> There's been a lot of talk about sublocal scopes, within and without
> the context of PEP 572. I'd like to propose what I believe is the
> simplest form of sublocal scopes, and use it to simplify one specific
> special case in Python.
>
> There are no syntactic changes, and only a very slight semantic change.
>
> def f():
> e = 2.71828
> try:
> 1/0
> except Exception as e:
> print(e)
> print(e)
> f()
>
> The current behaviour of the 'except... as' statement is as follows:
>
> 1) Bind the caught exception to the name 'e', replacing 2.71828
> 2) Execute the suite (printing "Division by zero")
> 3) Set e to None
> 4) Unbind e
>
> Consequently, the final print call raises UnboundLocalError. I propose
> to change the semantics as follows:
>
> 1) Bind the caught exception to a sublocal 'e'
> 2) Execute the suite, with the reference to 'e' seeing the sublocal
> 3) Set the sublocal e to None
> 4) Unbind the sublocal e
>
> At the unindent, the sublocal name will vanish, and the original 'e'
> will reappear. Thus the final print will display 2.71828, just as it
> would if no exception had been raised.
>
>
Does this mean indentation is now a scope, or colons are a scope, or is
that over simplifying?

either seems to be more consistent with the patterns set by class and
function defs, barring keywords.

not sure if relevant but curious.

I think with sublocal scope, reuse of a name makes more sense.  Currently,
if using sensible, descriptive names, it really doesn't make sense to go
from food = apple to food = car as the value between scopes, but it
happens.  And if from fruit = apple to fruit = orange (eg appending a msg
to a base string) it _could_ be nice to restore to apple once finished.

Obviously that's simple enough to do now, I am only illustrating my point.
I know bad code can be written with anything, this is not my point.  It can
be seen as enforcing that, what every nonsense someone writes like
fruit=car, there is at least some continuity of information represented by
the name... till they do it again once out of the sublocal scope of course.

as for the value of this use case, I do not know.


> The above definitions would become language-level specifications. For
> CPython specifically, my proposed implementation would be for the name
> 'e' to be renamed inside the block, creating a separate slot with the
> same name.
>
> With no debates about whether "expr as name" or "name := expr" or
> "local(name=expr)" is better, hopefully we can figure out whether
> sublocal scopes are themselves a useful feature :)
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Change magic strings to enums

[Jacco van Dorp]

2018-04-26 15:26 GMT+02:00 Nick Coghlan :
> On 26 April 2018 at 19:37, Jacco van Dorp  wrote:
>> I'm kind of curious why everyone here seems to want to use IntFlags
>> and other mixins. The docs themselves say that their use should be
>> minimized, and tbh I agree with them. Backwards compatiblity can be
>> maintained by allowing the old value and internally converting it to
>> the enum. Combinability is inherent to enum.Flags. There'd be no real
>> reason to use mixins as far as I can see ?
>
> Serialisation formats are a good concrete example of how problems can
> arise by switching out concrete types on people:
>
 import enum, json
 a = "A"
 class Enum(enum.Enum):
> ... a = "A"
> ...
 class StrEnum(str, enum.Enum):
> ... a = "A"
> ...
 json.dumps(a)
> '"A"'
 json.dumps(StrEnum.a)
> '"A"'
 json.dumps(Enum.a)
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "/usr/lib64/python3.6/json/__init__.py", line 231, in dumps
> return _default_encoder.encode(obj)
>   File "/usr/lib64/python3.6/json/encoder.py", line 199, in encode
> chunks = self.iterencode(o, _one_shot=True)
>   File "/usr/lib64/python3.6/json/encoder.py", line 257, in iterencode
> return _iterencode(o, 0)
>   File "/usr/lib64/python3.6/json/encoder.py", line 180, in default
> o.__class__.__name__)
> TypeError: Object of type 'Enum' is not JSON serializable
>
> The mixin variants basically say "If you run into code that doesn't
> natively understand enums, act like an instance of this type".
>
> Since most of the standard library has been around for years, and
> sometimes even decades, we tend to face a *lot* of backwards
> compatibility requirements along those lines.
>
> Cheers,
> Nick.

However, as the docs not, they will be comparable with each other,
which should throw an error. Since this is only a problem for this
case when serializing (since the functions would allow the old str
arguments for probably ever), shouldn't this be something caught when
you upgrade the version you run your script under ?

It's also a rather simple fix: json.dumps(Enum.a.value) would work just fine.

Now im aware that most people don't have 100% test coverage and such.
I also rather lack the amount of experience you guys have.

Guess im just a bit behind on the practicality beats purity here :)

Jacco
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A "local" pseudo-function

[Robert Vanden Eynde]

I really liked the syntax that mimicked lambda even if I find it verbose :

a = local x=1, y=2: x + y + 3

Even if I still prefer the postfix syntax :

a = x + 3 where x = 2

About scheme "let" vs "let*", the paralel in Python is :

a, b, c = 5, a+1, 2 # let syntax
a = 5; b = a+1; c = 2 # let* syntax

Which makes be wonder, we could use the semicolon in the syntax ?

a = local x = 1; y = x+1: x + y + 3

Or with the postfix syntax :

a = x + y + 3 where x = 1; y = x+1

Chaining where would be is syntax error :

a = x + y + 3 where x = 1 where y = x+1

Parenthesis could be mandatory if one wants to use tuple assignment :

a = local (x, y) = 1, 2: x + y + 3

When I see that, I really want to call it "def"

a = def (x, y) = 1, 2: x + y + 3
a = def x = 1; y = 2: x + y + 3

Which is read define x = 1 then y = 2 in x + y + 3

Using def would be obvious this is not a function call.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/