Re: [Python-ideas] Inline assignments using "given" clauses

[Tim Peters]

[Tim]
>> It's been noted several times recently that the example PEP 572 gives
>> as _not_ working:
>>
>> total = 0
>> progressive_sums = [total := total + value for value in data]
>>
>> was the original use case that prompted work on the PEP.  You gotta
>> admit that's ironic ;-)

[Nick]
> After pondering this case further, I think it's also worth noting that that
> *particular* example could also be addressed by:
>
> 1. Allowing augmented assignment *expressions*
> 2. Changing the scoping rules for augmented assignment operations in general
> such that they *don't change the scope of the referenced name*
>
> Writing "i += n" without first declaring the scope of "i" with "i = 0",
> "nonlocal i" or "global i" is one of the most common sources of
> UnboundLocalError after all, so I'd be surprised to find anyone that
> considered the current augmented assignment scoping rules to be outside the
> realm of reconsideration.

Yes, it's worth considering.  In my experience, I don't believe I've
ever had an UnboundLocalError for which a correct fix was to add
`nonlocal`.  Usually the correct fix was to add `global`, but that's
mostly due to an old habit of using piles of globals to count trips
through various code paths, used by a watchdog thread that
periodically wakes up to display (the global) counts.


> The accumulation example would then be written:
>
> total = 0
> progressive_sums = [total += value for value in data]
> if progressive_sums:
> assert total == progressive_sums[-1]

And the divide-out-small-primes example could be

factor = -42
while any(n % (factor += p - factor) == 0 for p in small_primes):
n //= factor

Heh ;-)


> The question would then turn to "What if you just want to bind the target
> name, without considering the old value?". And then *that's* where "NAME : =
> EXPR" would come in: as an augmented assignment operator that used augmented
> assignment scoping semantics, rather than regular local name binding
> semantics.

Plain old ":=" would somehow be viewed as being an augmented
assignment operator too? ... OK, the meaning is that augmented
assignment _and_ "::=" would resolve the target's scope in the way the
containing block resolves it.


> That would mean *directly* overturning PEP 3099's rejection of the idea of
> using "NAME := EXPR" to imply "nonlocal NAME" at function scope, but that's
> effectively on the table for implicit functions anyway (and I'd prefer to
> have ":=" be consistent everywhere, rather than having to special case the
> implicit scopes).

Creating a local in a containing scope by magic is never done by
Python today.  Extending that beyond "need" seems potentially
perilous.  For example, it can already be tedious to figure out which
names _are_ local to a function by staring at the function's code, but
people quickly get better at that over time; change the rules so that
they _also_ have to stare at all immediately contained  functions too
to figure it out, and it may become significantly harder (OK, I didn't
declare `x`, and a contained function did `x := 3.14` but `x` isn't
declared there either - I guess it's my `x` now).  Then again, if
they're doing that much function nesting they deserve whatever they
get ;-)

Restrict it to that only synthetically generated functions can pull
off this trick by magic (where there are real use cases to motivate
it), and they still don't have to look outside the body of a
function's text to figure it out.  Visually, there's no distinction
between the code running in the function's scope and in scopes
synthesized to implement comprehensions appearing in the function's
text.  The comprehensions aren't even indented more.

So, offhand, I'm not sure that the right way to address something you
view as a wart is to vastly expand its reach to 12 operators that
impose it on everyone everywhere every time they're used ;-)

Seriously, I do suspect that in

def f(...):
... no instances of `s` ...
s += f"START {time.time():.2f}"

it's overwhelmingly more likely that they simply forgot to do

s = ""

earlier in `f` than they actually wanted to append to whatever `s`
means in f's parent block..  That's a radical change to what people
have come to expect `NAME +=` to do.

OTOH, I don't (yet?) see a way the change could break code that
currently works, so it remains worth thinking about.

BTW, would

def f():
x := 3.14
x = 3.14

be a compile-time error?  Everyone agreed the analogous case would be
in synthetic functions.  Fine by me!
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A partial (wacky?) alternative to assignment expressions

[Dan Sommers]

On Tue, 15 May 2018 10:35:58 +1000, Steven D'Aprano wrote:

> We would need to flag which expression can be cached because it is
> PURE, and tag how far the CACHE operates over:
> 
>  
> 
> func(arg)
>  
> + func(arg)*2 + func(arg)**2 
> 
> 
> This would tell the compiler to only evaluate the sub-expression 
> "func(arg)" once, cache the result, and re-use it each other time it 
> sees that same sub-expression within the surrounding expression.

Does anyone else remember an ancient Python recipe that looked something
like what follows?

SENTINEL = object()
class Magic:
def __call__(self, arg=SENTINEL):
if arg != SENTINEL:
self.value = arg
return self.value
 magic = Magic()

And then you'd use it something like this:

if magic(g(x)) != 4:
print("probably not 4: ", magic())
else:
print("should be 4: ", magic())

magic(f(x)) + magic()*2 + magic()**2
consume_the_value_again(magic())

It's not perfect, but it's not too much worse than some of the other
suggestions floating around.  Sort of.

Dan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Tim Peters]

Just noting some real code I typed today where `given` works great if
it allows unpacking syntax, and assignment expressions don't:

while True:
head, matched, s = s.partition(sep)
if not matched:
break

Using `given`:

 while matched given head, matched, s = s.partition(sep):

Typing "matched " twice still sucks, though;-)

It doesn't work as well with assignment expressions even if they were
(re)generalized to allow unpacking syntax.  In this specific case, I'd
keep the original loop-and-a-half rather than do:

while (t := s.partition(sep})[1]:
head, matched, s = t

With unpacking syntax restored, same answer:

while (head, matched, s := s.partition(sep})[1]:

or
while [(head, matched, s := s.partition(sep}), matched][-1]:

to combine the worst annoyances of everything and add another ;-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Crazy idea: allow keywords as names in certain positions

[Terry Reedy]

On 5/14/2018 6:58 PM, Greg Ewing wrote:

Terry Reedy wrote:
the first two 'and's in "a and b.and # and error" are tagged. To not 
tag the second would require full parsing,


That particular case could be handled by just not colouring
any word following a dot.


OK, more parsing, even if not complete.  Other cases require looking at 
several other things before or after, depending on the exact proposal.

Okay, more parsing, depending on the exact rules.  Some of the variations (


Some other situations might result in false positives, but
there are cases like that already. Many Python colourisers
colour any occurrence of a builtin function name, even if
it has been shadowed, and people seem to live with that.


For IDLE:
>>> def int(): return 0 # int colored as defined name

>>> int()  # int colored as builtin.
0

Perhaps I should add a check whether defined names are builtins, and if 
so, color them as builtins even after 'def' and 'class'.



It might even be useful, if it alerts people that a name
they're using is reserved in some contexts, and so might
be best avoided.
Since IDLE is especially aimed at beginners, I could think the same 
about shadowing keywords, and leave them always marked as such.



Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A partial (wacky?) alternative to assignment expressions

[Franklin? Lee]

On Mon, May 14, 2018 at 8:35 PM, Steven D'Aprano  wrote:
> I'm hoping that the arguments for assignment expressions will be over by
> Christmas *wink* so as a partial (and hopefully less controversial)
> alternative, what do people think of the idea of flagging certain
> expressions as "pure functions" so the compiler can automatically cache
> results from it?
>
> Let me explain: one of the use-cases for assignment expressions is to
> reduce repetition of code which may be expensive. A toy example:
>
> func(arg) + func(arg)*2 + func(arg)**2
>
> If func() is a pure function with no side-effects, that is three times
> as costly as it ought to be:
>
> (f := func(arg)) + f*2 + f**2
>
> Functional languages like Haskell can and do make this optimization all
> the time (or so I am lead to believe), because the compiler knows that
> func must be a pure, side-effect-free function. But the Python
> interpreter cannot do this optimization for us, because it has no way of
> knowing whether func() is a pure function.

While I support the idea of marking pure functions somehow, automating
this optimization is not a simple matter.

Haskell (or at least its standard compiler) usually doesn't do it for you:
"""
GHC doesn't actually perform CSE as often as you might expect. The
trouble is, performing CSE can affect the strictness/laziness of the
program. So GHC does do CSE, but only in specific circumstances ---
see the GHC manual. (Section??)

Long story short: "If you care about CSE, do it by hand."
"""
https://wiki.haskell.org/GHC_optimisations#Common_subexpression_elimination

Pure functions do, however, help with constant propagation during compilation.

A PyPy blog post goes into a bit of detail about how pure functions
can be helpful:
https://morepypy.blogspot.com/2011/03/controlling-tracing-of-interpreter-with_15.html
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A partial (wacky?) alternative to assignment expressions

[Tim Peters]

[Steven D'Aprano ]
> I'm hoping that the arguments for assignment expressions will be over by
> Christmas *wink* so as a partial (and hopefully less controversial)
> alternative, what do people think of the idea of flagging certain
> expressions as "pure functions" so the compiler can automatically cache
> results from it?
>
> Let me explain: one of the use-cases for assignment expressions is to
> reduce repetition of code which may be expensive. A toy example:
>
> func(arg) + func(arg)*2 + func(arg)**2
>
> If func() is a pure function with no side-effects, that is three times
> as costly as it ought to be:
>
> (f := func(arg)) + f*2 + f**2
>
> Functional languages like Haskell can and do make this optimization all
> the time (or so I am lead to believe), because the compiler knows that
> func must be a pure, side-effect-free function. But the Python
> interpreter cannot do this optimization for us, because it has no way of
> knowing whether func() is a pure function.
>
> Now for the wacky idea: suppose we could tell the interpreter to cache
> the result of some sub-expression, and re-use it within the current
> expression? That would satisfy one use-case for assignment operators,
> and perhaps weaken the need for := operator.
>
> Good idea? Dumb idea?

Despite that Haskell can do optimizations like this , its "let ... in
..." and "... where ..." constructs (which create names for
expressions, for use in another expression or code block) are widely
used anyway.  They don't care about the optimization (they already get
it), but about improving clarity.  In Haskell they'd spell it like,
e.g., (mixing Python with Haskell keywords in UPPERCASE)

:LET fa = func(arg) IN fa + fa*2 + fa**2

which the compiler may (but almost certainly won't) optimize further to

LET fa = func(arg) IN fa * (3 + fa)

if it knows that fa is of a type for which that makes sense.

In Python today, I expect most people would do it as

t = func(arg)
t + 2*t + t*t  # or t*(3+t)

because they also know that multiplying t by itself once is usually
faster than squaring ;-)  And they wouldn't _want_ all the redundant
typing in

 func(arg) + func(arg)*2 + func(arg)**2

anyway.

So I'm not saying "good" or "bad", but that it needs a more compelling use case.


> Good idea, but you want the assignment operator regardless?

I'd probably write the example the way "I expect most people would do
it" above even if we do get assignment expressions.


> I don't have a suggestion for syntax yet, so I'm going to make up syntax
> which is *clearly and obviously rubbish*, a mere placeholder, so don't
> bother telling me all the myriad ways it sucks. I know it sucks, that's
> deliberate. Please focus on the *concept*, not the syntax.
>
> We would need to flag which expression can be cached because it is PURE,
> and tag how far the CACHE operates over:
>
> 
> 
> func(arg)
> 
> + func(arg)*2 + func(arg)**2
> 

That syntax is clearly and obviously rubbish!  It sucks.  You're welcome ;-)


> This would tell the compiler to only evaluate the sub-expression
> "func(arg)" once, cache the result, and re-use it each other time it
> sees that same sub-expression within the surrounding expression.
>
> To be clear: it doesn't matter whether or not the sub-expression
> actually is pure. And it doesn't have to be a function call: it could be
> anything legal in an expression.
>
> If we had this, with appropriately awesome syntax, would that negate the
> usefulness of assignment expresions in your mind?

The use cases I envision for that have no intersection with use cases
I have for assignment expressions, so, no.

My first thought about where it might be handy probably has no
intersection with what you were thinking of either ;-)



 math.ceil
 math.floor

def int_away_from_zero(x):
if x >= 0:
return math.ceil(x)
else:
 return math.floor(x)


The body of `int_away_from_zero()` is the way I _want_ to write it.
But in heavily used functions it's expensive to look up "math", then
look up its "ceil" (or "floor") attribute, on every call.  So stuff
like this often abuses default arguments instead:

def int_away_from_zero(x, mc=math.ceil, mf=math.floor):
if x >= 0:
return mc(x)
else:
 return mf(x)

As the function grows over time, the default arg abuse grows, and the
body of the function gets more obscure as more-&-more "tiny names" are
introduced to save on repeated global and module attribute lookups.

Indeed, in many cases I'd like to wrap an entire module in   ,  with oodles of "module.attribute" thingies
in the  block.  _Most_ of my code gets no benefit from Python's
"extremely dynamic" treatment of module.attribute.  It would be great
if Python could do those lookups once at module import time, then

Re: [Python-ideas] A partial (wacky?) alternative to assignment expressions

[David Mertz]

The time machine is used again! We HAVE a spelling: @functions.lru_cache()

The problem is that there's still a time/space trade-off. If you might call
a (pure) function with millions of different values, you likely don't want
to cache them all. The current spelling makes this configurable, but
there's no universal right answer.

On Mon, May 14, 2018, 8:36 PM Steven D'Aprano  wrote:

> I'm hoping that the arguments for assignment expressions will be over by
> Christmas *wink* so as a partial (and hopefully less controversial)
> alternative, what do people think of the idea of flagging certain
> expressions as "pure functions" so the compiler can automatically cache
> results from it?
>
> Let me explain: one of the use-cases for assignment expressions is to
> reduce repetition of code which may be expensive. A toy example:
>
> func(arg) + func(arg)*2 + func(arg)**2
>
> If func() is a pure function with no side-effects, that is three times
> as costly as it ought to be:
>
> (f := func(arg)) + f*2 + f**2
>
> Functional languages like Haskell can and do make this optimization all
> the time (or so I am lead to believe), because the compiler knows that
> func must be a pure, side-effect-free function. But the Python
> interpreter cannot do this optimization for us, because it has no way of
> knowing whether func() is a pure function.
>
> Now for the wacky idea: suppose we could tell the interpreter to cache
> the result of some sub-expression, and re-use it within the current
> expression? That would satisfy one use-case for assignment operators,
> and perhaps weaken the need for := operator.
>
> Good idea? Dumb idea?
>
> Good idea, but you want the assignment operator regardless?
>
> I don't have a suggestion for syntax yet, so I'm going to make up syntax
> which is *clearly and obviously rubbish*, a mere placeholder, so don't
> bother telling me all the myriad ways it sucks. I know it sucks, that's
> deliberate. Please focus on the *concept*, not the syntax.
>
> We would need to flag which expression can be cached because it is PURE,
> and tag how far the CACHE operates over:
>
> 
> 
> func(arg)
> 
> + func(arg)*2 + func(arg)**2
> 
>
> This would tell the compiler to only evaluate the sub-expression
> "func(arg)" once, cache the result, and re-use it each other time it
> sees that same sub-expression within the surrounding expression.
>
> To be clear: it doesn't matter whether or not the sub-expression
> actually is pure. And it doesn't have to be a function call: it could be
> anything legal in an expression.
>
> If we had this, with appropriately awesome syntax, would that negate the
> usefulness of assignment expresions in your mind?
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A partial (wacky?) alternative to assignment expressions

[Chris Angelico]

On Tue, May 15, 2018 at 10:35 AM, Steven D'Aprano  wrote:
> We would need to flag which expression can be cached because it is PURE,
> and tag how far the CACHE operates over:
>
> 
> 
> func(arg)
> 
> + func(arg)*2 + func(arg)**2
> 
>
> This would tell the compiler to only evaluate the sub-expression
> "func(arg)" once, cache the result, and re-use it each other time it
> sees that same sub-expression within the surrounding expression.
>
> To be clear: it doesn't matter whether or not the sub-expression
> actually is pure. And it doesn't have to be a function call: it could be
> anything legal in an expression.
>
> If we had this, with appropriately awesome syntax, would that negate the
> usefulness of assignment expresions in your mind?
>

Not for me, no. It doesn't eliminate lengthy expressions, only costly
ones. It also doesn't deduplicate for convenience of editing. So
that's one goal achieved, two not achieved.

If the function is actually pure, all you need is lru_cache and you
get that result. If it isn't, you've hit one of the two hardest
problems in computing, good luck.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] A partial (wacky?) alternative to assignment expressions

[Steven D'Aprano]

I'm hoping that the arguments for assignment expressions will be over by 
Christmas *wink* so as a partial (and hopefully less controversial) 
alternative, what do people think of the idea of flagging certain 
expressions as "pure functions" so the compiler can automatically cache 
results from it?

Let me explain: one of the use-cases for assignment expressions is to 
reduce repetition of code which may be expensive. A toy example:

func(arg) + func(arg)*2 + func(arg)**2

If func() is a pure function with no side-effects, that is three times 
as costly as it ought to be:

(f := func(arg)) + f*2 + f**2

Functional languages like Haskell can and do make this optimization all 
the time (or so I am lead to believe), because the compiler knows that 
func must be a pure, side-effect-free function. But the Python 
interpreter cannot do this optimization for us, because it has no way of 
knowing whether func() is a pure function.

Now for the wacky idea: suppose we could tell the interpreter to cache 
the result of some sub-expression, and re-use it within the current 
expression? That would satisfy one use-case for assignment operators, 
and perhaps weaken the need for := operator.

Good idea? Dumb idea?

Good idea, but you want the assignment operator regardless?

I don't have a suggestion for syntax yet, so I'm going to make up syntax 
which is *clearly and obviously rubbish*, a mere placeholder, so don't 
bother telling me all the myriad ways it sucks. I know it sucks, that's 
deliberate. Please focus on the *concept*, not the syntax.

We would need to flag which expression can be cached because it is PURE, 
and tag how far the CACHE operates over:

 

func(arg)
 
+ func(arg)*2 + func(arg)**2 


This would tell the compiler to only evaluate the sub-expression 
"func(arg)" once, cache the result, and re-use it each other time it 
sees that same sub-expression within the surrounding expression.

To be clear: it doesn't matter whether or not the sub-expression 
actually is pure. And it doesn't have to be a function call: it could be 
anything legal in an expression.

If we had this, with appropriately awesome syntax, would that negate the 
usefulness of assignment expresions in your mind?



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Crazy idea: allow keywords as names in certain positions

[Greg Ewing]

Terry Reedy wrote:
the first two 'and's in "a and b.and # and error" are tagged. 
To not tag the second would require full parsing,


That particular case could be handled by just not colouring
any word following a dot.

Some other situations might result in false positives, but
there are cases like that already. Many Python colourisers
colour any occurrence of a builtin function name, even if
it has been shadowed, and people seem to live with that.

It might even be useful, if it alerts people that a name
they're using is reserved in some contexts, and so might
be best avoided.

--
Greg

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] High Precision datetime

[Wes Turner]

>From "[Python-Dev] PEP 564: Add new time functions with nanosecond
resolution" (2017-10-16 hh:mm ss[...] -Z)
https://groups.google.com/forum/m/#!topic/dev-python/lLJuW_asYa0 :

> Maybe that's why we haven't found any CTCs (closed timelike curves) yet.
>
> Aligning simulation data in context to other events may be enlightening:
is there a good library for handing high precision time units in Python
(and/or CFFI)?

There's not yet an ISO8601-like standard for this level of time/date
precision.

Correlating particle events between experiments does require date+time.

On Monday, May 14, 2018, David Mertz  wrote:

> Chris is certainly right. A program that deals with femtosecond intervals
> should almost surely start by defining a "start of experiment" epoch where
> microseconds are fine. Then within that epoch, events should be monotonic
> integers for when measured or calculated times are marked.
>
> I can easily see reasons why a specialized wrapped int for
> FemtosecondsFromStart could be useful. But that's still a specialized need
> for a third party library. One possible use of this class might be to
> interoperate with datetimes or timedeltas. Conceivably sick
> interoperability could be dealing with leap seconds when needed. But
> "experiment time" should be a simple monotonic and uniform counter.
>
> On Mon, May 14, 2018, 6:35 PM Chris Barker - NOAA Federal via Python-ideas
>  wrote:
>
>> >
>> > UTC and leap seconds aren't a problem.
>>
>> Of course they are a problem— why else would they not be implemented
>> in datetime?
>>
>> But my point if that given datetimestamp or calculation could be off
>> by a second or so depending on whether and how leap seconds are
>> implemented.
>>
>> It just doesn’t seem like a good idea to be handling months and
>> femptoseconds with the same “encoding”
>>
>> -CHB
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] High Precision datetime

[David Mertz]

Chris is certainly right. A program that deals with femtosecond intervals
should almost surely start by defining a "start of experiment" epoch where
microseconds are fine. Then within that epoch, events should be monotonic
integers for when measured or calculated times are marked.

I can easily see reasons why a specialized wrapped int for
FemtosecondsFromStart could be useful. But that's still a specialized need
for a third party library. One possible use of this class might be to
interoperate with datetimes or timedeltas. Conceivably sick
interoperability could be dealing with leap seconds when needed. But
"experiment time" should be a simple monotonic and uniform counter.

On Mon, May 14, 2018, 6:35 PM Chris Barker - NOAA Federal via Python-ideas <
python-ideas@python.org> wrote:

> >
> > UTC and leap seconds aren't a problem.
>
> Of course they are a problem— why else would they not be implemented
> in datetime?
>
> But my point if that given datetimestamp or calculation could be off
> by a second or so depending on whether and how leap seconds are
> implemented.
>
> It just doesn’t seem like a good idea to be handling months and
> femptoseconds with the same “encoding”
>
> -CHB
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] High Precision datetime

[Chris Barker - NOAA Federal via Python-ideas]

>
> UTC and leap seconds aren't a problem.

Of course they are a problem— why else would they not be implemented
in datetime?

But my point if that given datetimestamp or calculation could be off
by a second or so depending on whether and how leap seconds are
implemented.

It just doesn’t seem like a good idea to be handling months and
femptoseconds with the same “encoding”

-CHB
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[David Mertz]

So does NumPy and sckit-learn use the trailing underscore convention.
Albeit, sklearn uses it for (almost) all the model attributes, not just
those it thinks might clash.

On Mon, May 14, 2018, 2:12 PM Terry Reedy  wrote:

> On 5/14/2018 10:02 AM, Clint Hepner wrote:
> >
> >> On 2018 May 14 , at 6:47 a, Daniel Moisset 
> wrote:
> >>
> >> Following up some of the discussions about the problems of adding
> keywords and Guido's proposal of making tokenization context-dependent, I
> wanted to propose an alternate way to go around the problem.
> >
> > My main objection to what follows is that it doesn't seem to offer any
> benefit over the current practice of appending an underscore (_) to a
> keyword to make it a valid identifier.
>
> Tkinter uses this convention for a few option names that clash.
>
> --
> Terry Jan Reedy
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Carl Smith]

Sorry to think out loud, but if the lexer marked `foo` as a generic `Word`
token, that
could be a keyword or a name, then the parser could look at the value of
each `Word`
token, and if the context is `import foo`, `class foo...`, `def foo...` or
`foo = ...`,
then `foo` is a name there and thereafter (and `x foo y` is a SyntaxError)
else...
you get the idea.


-- Carl Smith
carl.in...@gmail.com

On 14 May 2018 at 22:06, Carl Smith  wrote:

> Just to be clear, if `foo` was introduced as a new infix operator,
> projects that used `foo`
> as a name would not be able to also use `foo` as an infix operator in the
> file that defines
> `foo` as a name, but could use the operator throughout the rest of their
> project.
>
> -- Carl Smith
> carl.in...@gmail.com
>
> On 14 May 2018 at 21:52, Carl Smith  wrote:
>
>> I can only think of three ways to reference a name defined in a different
>> file: In an import
>> statement, as properties of objects and as keyword arguments.
>>
>> Import statements are implicit assignments, so if Python allowed the
>> following grammar,
>> you could still import the odd thing that had a reserved name, without
>> bringing that name
>> into your local namespace.
>>
>> from  import  as 
>>
>> Property names always follow a dot, where only a name is valid, so Python
>> could allow
>> this too:
>>
>> .
>>
>> Keyword arguments are also generally unambiguous, as they have to appear
>> within the
>> parens of an invocation, before the equals sign:
>>
>> (=)
>>
>> If Python allowed those three examples (but still prevented users from
>> *defining* names
>> that are keywords) new keywords could be introduced without breaking old
>> code , and the
>> language would only require relatively minor tweaking.
>>
>> -- Carl Smith
>> carl.in...@gmail.com
>>
>> On 14 May 2018 at 19:11, Terry Reedy  wrote:
>>
>>> On 5/14/2018 10:02 AM, Clint Hepner wrote:
>>>

 On 2018 May 14 , at 6:47 a, Daniel Moisset 
> wrote:
>
> Following up some of the discussions about the problems of adding
> keywords and Guido's proposal of making tokenization context-dependent, I
> wanted to propose an alternate way to go around the problem.
>

 My main objection to what follows is that it doesn't seem to offer any
 benefit over the current practice of appending an underscore (_) to a
 keyword to make it a valid identifier.

>>>
>>> Tkinter uses this convention for a few option names that clash.
>>>
>>> --
>>> Terry Jan Reedy
>>>
>>>
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Carl Smith]

Just to be clear, if `foo` was introduced as a new infix operator, projects
that used `foo`
as a name would not be able to also use `foo` as an infix operator in the
file that defines
`foo` as a name, but could use the operator throughout the rest of their
project.

-- Carl Smith
carl.in...@gmail.com

On 14 May 2018 at 21:52, Carl Smith  wrote:

> I can only think of three ways to reference a name defined in a different
> file: In an import
> statement, as properties of objects and as keyword arguments.
>
> Import statements are implicit assignments, so if Python allowed the
> following grammar,
> you could still import the odd thing that had a reserved name, without
> bringing that name
> into your local namespace.
>
> from  import  as 
>
> Property names always follow a dot, where only a name is valid, so Python
> could allow
> this too:
>
> .
>
> Keyword arguments are also generally unambiguous, as they have to appear
> within the
> parens of an invocation, before the equals sign:
>
> (=)
>
> If Python allowed those three examples (but still prevented users from
> *defining* names
> that are keywords) new keywords could be introduced without breaking old
> code , and the
> language would only require relatively minor tweaking.
>
> -- Carl Smith
> carl.in...@gmail.com
>
> On 14 May 2018 at 19:11, Terry Reedy  wrote:
>
>> On 5/14/2018 10:02 AM, Clint Hepner wrote:
>>
>>>
>>> On 2018 May 14 , at 6:47 a, Daniel Moisset 
 wrote:

 Following up some of the discussions about the problems of adding
 keywords and Guido's proposal of making tokenization context-dependent, I
 wanted to propose an alternate way to go around the problem.

>>>
>>> My main objection to what follows is that it doesn't seem to offer any
>>> benefit over the current practice of appending an underscore (_) to a
>>> keyword to make it a valid identifier.
>>>
>>
>> Tkinter uses this convention for a few option names that clash.
>>
>> --
>> Terry Jan Reedy
>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Carl Smith]

I can only think of three ways to reference a name defined in a different
file: In an import
statement, as properties of objects and as keyword arguments.

Import statements are implicit assignments, so if Python allowed the
following grammar,
you could still import the odd thing that had a reserved name, without
bringing that name
into your local namespace.

from  import  as 

Property names always follow a dot, where only a name is valid, so Python
could allow
this too:

.

Keyword arguments are also generally unambiguous, as they have to appear
within the
parens of an invocation, before the equals sign:

(=)

If Python allowed those three examples (but still prevented users from
*defining* names
that are keywords) new keywords could be introduced without breaking old
code , and the
language would only require relatively minor tweaking.

-- Carl Smith
carl.in...@gmail.com

On 14 May 2018 at 19:11, Terry Reedy  wrote:

> On 5/14/2018 10:02 AM, Clint Hepner wrote:
>
>>
>> On 2018 May 14 , at 6:47 a, Daniel Moisset 
>>> wrote:
>>>
>>> Following up some of the discussions about the problems of adding
>>> keywords and Guido's proposal of making tokenization context-dependent, I
>>> wanted to propose an alternate way to go around the problem.
>>>
>>
>> My main objection to what follows is that it doesn't seem to offer any
>> benefit over the current practice of appending an underscore (_) to a
>> keyword to make it a valid identifier.
>>
>
> Tkinter uses this convention for a few option names that clash.
>
> --
> Terry Jan Reedy
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Crazy idea: allow keywords as names in certain positions

[Ethan Furman]

On 05/13/2018 10:57 PM, Greg Ewing wrote:

Rob Cliffe via Python-ideas wrote:



If you forbid redefining keywords, you remove the whole point of this proposal:


I mean the keywords that are in the language as of now.
There will never be a need to redefine those, since no
current code uses them as names.


Part of the point of the proposal is to be able to use existing keywords (at 
least, I thought it was).

--
~Ethan~

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Crazy idea: allow keywords as names in certain positions

[Tim Peters]

[Terry Reedy ]
> ...
> The impact on real-time syntax coloring should be considered.  An re-based
> colorizer, like IDLE's, tags every appearance of keywords outside of strings
> and comments, syntactically correct or not. For instance, the first two
> 'and's in "a and b.and # and error" are tagged. To not tag the second would
> require full parsing, presumably by compiling to AST, and more support code.
> I suspect this would would make coloring slower.  Whether too slow for
> existing machines, I don't know.

I can pretty much guarantee it would be slower - way slower.

But regexps could still do the bulk of it, provided the contexts
remain as simple-minded as the specific ones Guido suggested:

For example, we could allow keywords after 'def' and after a period,

So when the IDLE colorizer regexp finds a match with the "KEYWORD"
group name, don't color it at once.  Instead look back from the start
of the _possible_ keyword match, to see whether it's preceded by

 period maybe_whitespace
or
 "def"-starting-at-a-word-boundary  at_least_one_whitespace

and if so skip coloring.

There are many ways to code that, but - ya - they're annoying.  Alas,
the re module's negative lookbehind assertion isn't powerful enough
for it, else those checks could be added directly to the keyword part
of the colorizing regexp.  I expect (but don't know) that lookbehinds
in MRAB's "regex" module are strong enough for it.

Of course that's just IDLE.  It was remarkably easy to teach my
primary editor (Source Insight) all sorts of stuff about .py files,
but its "keywords" subsystem is restricted to specifying literal
strings to look for.  The kinds of tricks above probably require that
the tool have access to a real programming language (like IDLE and
Emacs enjoy).
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Sorry for yet another self discussion

[Terry Reedy]

On 5/10/2018 3:58 PM, stefano wrote:

I know that "self" parameter have been discussed a lot, but still I didn't
find this proposal. If it was instead take my sincere apologies and please
forget this mail.

The disturbing part of the "self parameter" is the asymmetry of the
definition and the call.


You are free to use the full signature of the method by calling the 
method on the class, where it is defined.  Self *is* a parameter of the 
function and therefore *is* part of its signature.


>>> l = []
>>> l.append(1)
>>> list.append(l, 2)
>>> l
[1, 2]

Note that you can make a function defined outside of any class a method 
of some class by making it an attribute of the class.


An intelligent editor like IDLE changes the call signature tip according 
to the call form.  "l.append(" brings up "(object, /)" while 
"list.append(" brings up "(self, object, /)", along with a note 
explaining that '/' marks the preceding arguments as positional-only.



So I was thinking: why not do define the methods
like: "def self.whatevermethod(par1, par2, etc)" instead of "def
whatevermethod(self, par1, par2, etc)"?


This or something like it has been proposed and rejected before.


This will allow the call and the definition to be written exactly in the
same way, still leaving the "clarity" of the additional input for the
function.


When you call 'self.method', you are not calling type(self).method.  You 
are instead calling the bound method 'self.method', which has the 
abbreviated signature.  This proposal would create an asymmetry between 
the function definition and the call of the function.


--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Terry Reedy]

On 5/14/2018 10:02 AM, Clint Hepner wrote:



On 2018 May 14 , at 6:47 a, Daniel Moisset  wrote:

Following up some of the discussions about the problems of adding keywords and 
Guido's proposal of making tokenization context-dependent, I wanted to propose 
an alternate way to go around the problem.


My main objection to what follows is that it doesn't seem to offer any benefit 
over the current practice of appending an underscore (_) to a keyword to make 
it a valid identifier.


Tkinter uses this convention for a few option names that clash.

--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Crazy idea: allow keywords as names in certain positions

[Terry Reedy]

On 5/13/2018 2:19 PM, Guido van Rossum wrote:

The idea I had (not for the first time :-) is that in many syntactic 
positions we could just treat keywords as names, and that would free up 
these keywords.


This trades the simplicity of 'using a keyword as an identifier always 
fails immediately' for the flexibility of 'using a keyword as an 
identifier may work, or may fail later'.  The failure would be 
predictable in the sense of being being deterministic for any particular 
code string but unpredictable in the sense of not knowing what code 
would be written.  It would also be unpredictable when people use a 
keyword as variable without knowing that they had done so.


The proposal reminds me of the current tkinter situation when using 
tcl/tk compiled without thread support: calling a tkinter function 
(method) from worker threads may work, at least initially, but may 
eventually fail.  The main problem here is that this is not documented. 
But I suspect that if and when it is 
(https://bugs.python.org/issue33479), most will 'not do that' and only a 
few intrepid souls will do the necessary experiments.


The impact on real-time syntax coloring should be considered.  An 
re-based colorizer, like IDLE's, tags every appearance of keywords 
outside of strings and comments, syntactically correct or not. For 
instance, the first two 'and's in "a and b.and # and error" are tagged. 
To not tag the second would require full parsing, presumably by 
compiling to AST, and more support code.  I suspect this would would 
make coloring slower.  Whether too slow for existing machines, I don't know.


--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Tim Peters]

>> total = 0
>> progressive_sums = [total := total + value for value in data]

[Greg Ewing ]
> I'm skeptical that it's a good idea to encourage this kind of thing
> in the first place.

Adding a feature is creating possibility for its use, not encouraging
its use.  I'd hate to, ,e.g., see people encouraging use of ternary-if
either,  _Sometimes_ ternary-if is exactly right, both shortening code
and increasing clarity.  But not often.  Ditto for metaclasses, the
ability to nest blocks 20 levels deep, on & on.  That they can be
abused, and sometimes are abused, is a very weak argument against
them.  On a scale of 1 to 100?  Maybe 23 ;-)

The specific use case that grabbed Guido's attention on this one
wasn't the above, but:

while any(n % p == 0 for p in small_primes):
// divide p out of n - but which p succeeded?

 _General_ features turn out to have multiple good uses, not all of
which are anticipated.  I think this is an example of that.  It's easy
to miss that the argument to `any()`  runs in an entirely different
scope, so that its local `p` vanishes when `any()` completes.

It's not proposed to change that either, because people overwhelmingly
seem to want `for` targets not to leak.  Fine by me.

Under the proposal, it's straightforward to use an assignment
expression to "export" p's last materialized value:

while any(n % (factor := p) == 0 for p in small_primes):
n //= factor

To my eyes, the most surprising thing about that is why it's necessary
to "export" p's winning value to begin with.  "Because for-targets
always vanish, so there's another form of explicit binding that
doesn't" is a reasonably satisfying explanation.

Nick would be happiest, and I'd be "happy enough", if all forms of
binding remained "local" and instead we fiddled the syntax to allow
explicitly declaring the desired scope.  Then, e.g.,

while any(
n % p == 0 for p in small_primes):
n //= p

wouldn't need an assignment expression at all.  But Guido doesn't like
that.  I think he wants to hide, so far as humanly possible, that
listcomps and genexps happen to implemented now by synthesizing
functions.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] High Precision datetime

[Chris Angelico]

On Tue, May 15, 2018 at 2:05 AM, Chris Barker via Python-ideas
 wrote:
> But my question is whether high precision timedeltas belongs with "calendar
> time" at all.
>
> What with UTC and leap seconds, and all that, it gets pretty ugly, when down
> to the second or sub-second, what a given datetime really means.

UTC and leap seconds aren't a problem. When there's a leap second, you
have 23:59:60 (or you repeat 23:59:59, if you can't handle second
#60). That's pretty straight-forward, perfectly well-defined.

No, the REAL problems come from relativity.

> If I were to work with high precision measurements, experiments, etc, I'd
> use a "nanoseconds since" representation, where the "epoch" would likely be
> the beginning of the experiment, of something relevant.

That's an unrelated form of time calculation. For that kind of thing,
you probably want to ignore calendars and use some form of monotonic
time; but also, if you want to go to (or below) nanosecond resolution,
you'll need your clock to actually be that accurate, which most likely
means you're not using a computer's clock. Femtosecond timestamping
would basically be just taking numbers given to you by an external
device and using them as sequence points - clocks and calendars become
irrelevant. The numbers might as well be frame numbers in a
super-high-speed filming of the event.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] High Precision datetime

[Chris Barker via Python-ideas]

On Thu, May 10, 2018 at 6:13 PM, Alexander Belopolsky <
alexander.belopol...@gmail.com> wrote:

> > Is there interest in a PEP for extending time, datetime / timedelta for
> arbitrary or extended precision fractional seconds?
>
> Having seen the utter disaster that similar ideas brought to numpy, I would
> say: no.
>

I'm not sure the "disaster" was due to this idea nor, frankly, is
datetime64 a disaster at all, though certainly far from perfect.

But my question is whether high precision timedeltas belongs with "calendar
time" at all.

What with UTC and leap seconds, and all that, it gets pretty ugly, when
down to the second or sub-second, what a given datetime really means.

If I were to work with high precision measurements, experiments, etc, I'd
use a "nanoseconds since" representation, where the "epoch" would likely be
the beginning of the experiment, of something relevant.

Note that this issued in netcdf CF formats, datetimes are expressed in
things like:

"hours since 1970-01-01:00:00"

granted, it's mostly so that the values can be stored as an array of a
simple scalars, but it does allow precision and an epoch that are suited to
the data at hand.

NOTE: One source of the "disaster" of numpy's datetime64 is you can set teh
precision, but NOT the epoch -- which is kind of problematic if you really
want femtosecond precision for something not in 1970 :-)

-CHB











> On the other hand, nanoseconds are slowly making their way to the stdlib
> and to add nanoseconds to datetime we only need a fully backward compatible
> implementation, not even a PEP.
>
> See .
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Sorry for yet another self discussion

[Chris Barker via Python-ideas]

> So I was thinking: why not do define the methods
> > like: "def self.whatevermethod(par1, par2, etc)" instead of "def
> > whatevermethod(self, par1, par2, etc)"?
>

because "self" in this case is a class instance, passed in at method call
time.

but "whatevermethod" is a class attribute.

note the distinction between the class object (shared by all instances) and
the instance object. I'm not saying the compiler couldn't make it work, but
I am saying that it would confuse the issue -- as it would not be defining
an attribute of self, even though it looks like it is.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Alexandre Brault]

On 2018-05-14 06:47 AM, Daniel Moisset wrote:
> Following up some of the discussions about the problems of adding
> keywords and Guido's proposal of making tokenization
> context-dependent, I wanted to propose an alternate way to go around
> the problem.
>
> My proposal essentially boils down to:
>
>  1. The character "$" can be used as a prefix of identifiers. formally, 
> *identifier * ::= ["$"] |xid_start|
> 
> 
>  |xid_continue|
> 
> *
>  2. The "$" character is not part of the name. So the program
> "foo=3;print($foo)" prints 3. So does the program "$foo=3;
> print(foo)". Both set an entry to globals["foo"] and keep
> globals["$foo"] unset. 
>  3. if "$" appears in a token, it's always an identifier. So "$with",
> "$if", "$return" are all identifiers.
>
> If you overcome the "yikes, that looks like awk/bash/perl/php, and I
> don't like those", and consider it as an escape for
> "unusual"/"deprecation" situations, I think it's not a bad chose, and
> allows to a simple solutions to many problems that have been in
> discussion recently and not so recently. [examples below]
>
> For me the benefits of this approach are:
>
>   * It's very simple to explain how to use and its semantics
>   * It (seems to me it) should be easy to explain to a python
> apprentice what a "$" means in code they read on a
> book/blogpost/manual
>   * It's very easy to implement, minimal changes in the tokenizer
>   * It's also easy to implement/integrate in other tools (editors with
> syntax highlighters, code formatters, etc)
>   * It is easy to see that it's 100% backwards compatible (I
> understand that "$" has never been used in python before)
>   * It is relatively unsurprising in the sense that other languages
> are already using $ to label names (there may be some point of
> confusion to people coming from javascript where "$" is a valid
> character in names and is not ignored). 
>   * It gives python devs and users a clear, easy and universal upgrade
> path when keywords are added (language designers: Add a __future__
> import to enable keyword in python N+1, add warnings to change kw
> --> $kw in python N+2, and then turn it on by default in python
> N+3... ; developers: add the import when they want to upgrade ,
> and fix their code with a search when adding the import or
> after getting a warning).
>   * It allows you to use new features even if some libraries were
> written for older python versions, depending the deprecation
> period (this could be improved with sth I'll write in another
> email, but that's the topic for another proposal)
>   * When clashes occur, which they always do, there's one obvious way
> to disambiguate (see today the "class_" argument for
> gettext.translation, the "lambd" argument for random.expovariate,
> the "class_" filter in libraries like pyquery for CSS class,
> functions like pyquery, sqlalchemy.sql.operators.as_ , etc. Not
> counting all the "cls" argument to every classmethod ever)
>   * If we're worried about over proliferation of "$" in code, I'm
> quite sure given past experience that just a notice in PEP 8 of
> "only with $ in names to prevent ambiguity" should be more than
> enough for the community
>
>
> What are the drawbacks that you find in this?
> Best,
>    Daniel
>
For the record, C# does something similar to help interface with other
CLR languages with different keywords: any token starting with @ is an
identifier even if the unprefixed token would be a reserved keyword.

More on that: https://ericlippert.com/2013/09/09/verbatim-identifiers/

Alex

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Daniel Moisset]

On 14 May 2018 at 15:02, Clint Hepner  wrote:

>
> > On 2018 May 14 , at 6:47 a, Daniel Moisset 
> wrote:
> >
> > Following up some of the discussions about the problems of adding
> keywords and Guido's proposal of making tokenization context-dependent, I
> wanted to propose an alternate way to go around the problem.
>
> My main objection to what follows is that it doesn't seem to offer any
> benefit over the current practice of appending an underscore (_) to a
> keyword to make it a valid identifier.
>
>
A secondary benefit I forgot to mention in my previous reply (and which is
not covered by adding an underscore suffix) is that the original name being
preserved it's better for interfacing with external systems that may use
those names:

For example, this code:
https://github.com/dsc/pyquery/blob/fd725469701a8f47840f142df5c9b6d4479ea58e/pyquery/pyquery.py#L584
(and some code below that) is essentially a workaround that is required
because they used a "_" suffix ibecause it's hard to pass "class" or "for"
as keys in **kwargs

-- 

Daniel Moisset
UK COUNTRY MANAGER

A:   1 Fore Street, EC2Y 9DT London 
P:   +44 7398 827139 <+44+7398+827139>
M:   dmois...@machinalis.com   |   S:   dmoisset

  

Machinalis Limited is a company registered in England and Wales. Registered
number: 10574987.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Daniel Moisset]

On 14 May 2018 at 15:02, Clint Hepner  wrote:

>
> > On 2018 May 14 , at 6:47 a, Daniel Moisset 
> wrote:
> >
> > Following up some of the discussions about the problems of adding
> keywords and Guido's proposal of making tokenization context-dependent, I
> wanted to propose an alternate way to go around the problem.
>
> My main objection to what follows is that it doesn't seem to offer any
> benefit over the current practice of appending an underscore (_) to a
> keyword to make it a valid identifier.
>

There is a key difference: if an optional keyword is added (through a "from
__future__ import some_keyword"), I still can use in a simple ways names
from modules that do not have the future import enabled.

Using just an underscore suffix or similar name *change* is fine when the
language is static. But when the language changes, and I can not modify all
the third party libraries, being able to refer to *the original name*
instead of a modified one is a significant need that can  not be covered by
"just add a _ suffix".

Best,
   D.

-- 

Daniel Moisset
UK COUNTRY MANAGER

A:   1 Fore Street, EC2Y 9DT London 
P:   +44 7398 827139 <+44+7398+827139>
M:   dmois...@machinalis.com   |   S:   dmoisset

  

Machinalis Limited is a company registered in England and Wales. Registered
number: 10574987.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Greg Ewing]

Steven D'Aprano wrote:
To reiterate what Tim already pointed out, that original usecase 
required a way to feed values *into* the comprehension.


https://mail.python.org/pipermail/python-ideas/2018-February/048971.html

There's no need for dedicated syntax for that if we can just set an 
variable and have it show up in the comprehension.


Yes, but doing it that way also allows things to come *out*
of the comprehension, which is something we previously thought
was such a bad idea that we went to considerable lengths to
stop it happening. Now we seem to be seriously considering
going to further lengths to allow it again for *some* things.

Seems to me we should have a pretty compelling reason for
doing that. I don't feel very compelled by what I've seen
so far.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Clint Hepner]

> On 2018 May 14 , at 6:47 a, Daniel Moisset  wrote:
> 
> Following up some of the discussions about the problems of adding keywords 
> and Guido's proposal of making tokenization context-dependent, I wanted to 
> propose an alternate way to go around the problem.

My main objection to what follows is that it doesn't seem to offer any benefit 
over the current practice of appending an underscore (_) to a keyword to make 
it a valid identifier.

> 
> My proposal essentially boils down to:
>   • The character "$" can be used as a prefix of identifiers. formally, 
> identifier   ::= ["$"] xid_start xid_continue*
>   • The "$" character is not part of the name. So the program 
> "foo=3;print($foo)" prints 3. So does the program "$foo=3; print(foo)". Both 
> set an entry to globals["foo"] and keep globals["$foo"] unset. 

What is the benefit here? ``globals`` returns a mapping that represents the 
global scope, but that doesn't mean the mapping *is* the global scope. Aside 
from dubious uses of globals()['class'] to sidestep parsing issues, '$class' 
would still be, for all practical purposes, the "real" name of the identifier.

>   • if "$" appears in a token, it's always an identifier. So "$with", 
> "$if", "$return" are all identifiers.
> If you overcome the "yikes, that looks like awk/bash/perl/php, and I don't 
> like those", and consider it as an escape for "unusual"/"deprecation" 
> situations, I think it's not a bad chose, and allows to a simple solutions to 
> many problems that have been in discussion recently and not so recently. 
> [examples below]

The same argument applies to the underscore (or could, since it would be 
trivial to promise that no future keyword will end with, or even contain, an 
underscore).

Defining new *keywords* to take the $-prefix could be done in a 
backwards-compatible way, although IMO the $ is too ugly to be a realistic 
choice. There might be a Unicode character that would make a good prefix, but 
given the reluctance to extend the core grammar beyond 7-bit ASCII, I've 
haven't spent any time looking for good candidates.

> For me the benefits of this approach are:
>   • It's very simple to explain how to use and its semantics

>   • It (seems to me it) should be easy to explain to a python apprentice 
> what a "$" means in code they read on a book/blogpost/manual

>   • It's very easy to implement, minimal changes in the tokenizer

>   • It's also easy to implement/integrate in other tools (editors with 
> syntax highlighters, code formatters, etc)
>   • It is easy to see that it's 100% backwards compatible (I understand 
> that "$" has never been used in python before)

The above 5 points all apply to appending an underscore to a keyword to create 
a valid identifier.

>   • It is relatively unsurprising in the sense that other languages are 
> already using $ to label names (there may be some point of confusion to 
> people coming from javascript where "$" is a valid character in names and is 
> not ignored).

Given the variation in how other languages use $, I'm not sure this is a point 
in favor. There are plenty of questions
on Stack Overflow about how and when to use $ in bash, and much of the 
confusion appears to stem from how $ is used in Perl.
And that ignores the cases where $ is either optional (arithmetic expressions) 
or *should not* (the read built-in, the -v conditional operator, etc) be used.

For that matter, why make $ special and restrict it to prefix position, instead 
of simply allowing $ as a valid identifier character and declaring that no 
keyword will ever use it?

>  
>   • It gives python devs and users a clear, easy and universal upgrade 
> path when keywords are added (language designers: Add a __future__ import to 
> enable keyword in python N+1, add warnings to change kw --> $kw in python 
> N+2, and then turn it on by default in python N+3... ; developers: add the 
> import when they want to upgrade , and fix their code with a search 
> when adding the import or after getting a warning).

Nothing about this process is specific to the $-prefix; it applies just as well 
to the practice of appending _ when it becomes
necessary.

>   • It allows you to use new features even if some libraries were written 
> for older python versions, depending the deprecation period (this could be 
> improved with sth I'll write in another email, but that's the topic for 
> another proposal)

I'll withhold judgement on this point, then, but it's not obvious how this 
allows an old library that uses a newly-minted keyword
as an identifier name to continue working.

>   • When clashes occur, which they always do, there's one obvious way to 
> disambiguate (see today the "class_" argument for gettext.translation, the 
> "lambd" argument for random.expovariate, the "class_" filter in libraries 
> like pyquery for CSS class, functions like pyquery, 
> sqlalchemy.sql.operators.as_ , etc. 

Re: [Python-ideas] Inline assignments using "given" clauses

[Greg Ewing]

Tim Peters wrote:


total = 0
progressive_sums = [total := total + value for value in data]


I'm skeptical that it's a good idea to encourage this
kind of thing in the first place.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "given" vs ":=" in list comprehensions

[Nick Coghlan]

On 14 May 2018 at 08:24, Ed Kellett  wrote:

> On 2018-05-14 05:02, Nick Coghlan wrote:
> > The same grammar adjustment that I believe will allow "given" to be used
> as
> > both a postfix keyword and as a regular name would also work for "where".
> > However, "where" still has the problem of semantically conflicting with
> > SQL's use of it to introduce a filter clause, whereas Hypothesis uses
> > "given" to bind names to values (just a little more indirectly than would
> > be the case for assignment expressions)
>
> I suspect that SQL is not high on the list of languages people might
> confuse with Python.


If we used "where" as a name binding keyword, ORM docs like
http://docs.sqlalchemy.org/en/latest/orm/query.html, and
https://docs.djangoproject.com/en/2.0/topics/db/queries/ would need to be
modified to explain that "SQL WHERE" and "Python where" do very different
things.

It's better to just avoid the potential for that problem entirely (either
by using a symbolic notation, or by using a different keyword)


> FWIW, as I'm sure will have been mentioned, Haskell
> uses "where", and people seem to manage fine with it.
>

Unfortunately, Haskell's adoption numbers and reputation as a difficult to
learn language don't back up that assumption (I doubt that outcome has
anything to do with their use of "where", it just means "Haskell uses it
that way" can't be credited as evidence that something won't cause
confusion)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Nick Coghlan]

On 14 May 2018 at 06:10, Tim Peters  wrote:

> [Greg Ewing ']
> > This whole discussion started because someone wanted a way
> > to bind a temporary result for use *within* a comprehension.
>
> It's been noted several times recently that the example PEP 572 gives
> as _not_ working:
>
> total = 0
> progressive_sums = [total := total + value for value in data]
>
> was the original use case that prompted work on the PEP.  You gotta
> admit that's ironic ;-)
>

After pondering this case further, I think it's also worth noting that that
*particular* example could also be addressed by:

1. Allowing augmented assignment *expressions*
2. Changing the scoping rules for augmented assignment operations in
general such that they *don't change the scope of the referenced name*

Writing "i += n" without first declaring the scope of "i" with "i = 0",
"nonlocal i" or "global i" is one of the most common sources of
UnboundLocalError after all, so I'd be surprised to find anyone that
considered the current augmented assignment scoping rules to be outside the
realm of reconsideration.

The accumulation example would then be written:

total = 0
progressive_sums = [total += value for value in data]
if progressive_sums:
assert total == progressive_sums[-1]

The question would then turn to "What if you just want to bind the target
name, without considering the old value?". And then *that's* where "NAME :
= EXPR" would come in: as an augmented assignment operator that used
augmented assignment scoping semantics, rather than regular local name
binding semantics.

That would mean *directly* overturning PEP 3099's rejection of the idea of
using "NAME := EXPR" to imply "nonlocal NAME" at function scope, but that's
effectively on the table for implicit functions anyway (and I'd prefer to
have ":=" be consistent everywhere, rather than having to special case the
implicit scopes).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline changes to context, or PEP572

[Steven D'Aprano]

On Sat, May 12, 2018 at 08:13:01PM -0400, Juancarlo Añez wrote:
> A new thread just to suggest taking the discussion about PEP572 well beyond
> python-ideas (PyConn is good for that).
> 
> The least anyone should want is a language change that immediately gets
> tagged on the networks as "don't use", or "use only for...", etc.

*Every* feature should be tagged as "use only for ...".

Every feature has uses that it works for, and uses that are 
inappropriate. With old features, we rarely think about the bad uses.
 
(When was the last time you needed to tell someone not to use a while 
loop when a for loop over an iterable was better? Apart from beginners 
reading "Learn Python The Hard Way", I've never seen anyone do that.)

With new features, some people do nothing but think about the bad uses.


> To be honest, I'll likely be on the "don't use :=, unless" band of pundits

As we all should.

Just as we didn't run out and change every if...else block to try to 
squeeze it into a ternary if operator, or turn every for loop into a 
comprehension, so we shouldn't use binding expressions unless they make 
the code better.


> (already a filibuster). ":=" is like going back to "reduce()", which is
> almost defunct thanks to.. us!

I don't think that it is nice of you to gloat about (almost) taking away 
a useful, much-beloved functional programming idiom. For some people, 
this comes across as "I want to make your code worse".

Fortunately, reduce is alive and well in the functools module.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "given" vs ":=" in list comprehensions

[Ed Kellett]

On 2018-05-14 05:02, Nick Coghlan wrote:
> The same grammar adjustment that I believe will allow "given" to be used as
> both a postfix keyword and as a regular name would also work for "where".
> However, "where" still has the problem of semantically conflicting with
> SQL's use of it to introduce a filter clause, whereas Hypothesis uses
> "given" to bind names to values (just a little more indirectly than would
> be the case for assignment expressions)

I suspect that SQL is not high on the list of languages people might
confuse with Python. FWIW, as I'm sure will have been mentioned, Haskell
uses "where", and people seem to manage fine with it.



signature.asc
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Crazy idea: allow keywords as names in certain positions

[Ed Kellett]

On 2018-05-14 12:35, Rhodri James wrote:
> On 13/05/18 19:19, Guido van Rossum wrote:
>> The idea I had (not for the first time:-)  is that in many syntactic
>> positions we could just treat keywords as names, and that would free up
>> these keywords.
> 
> I'm not familiar with the innards of the parser and it's wy too long
> since I sat through a parsing course, but can you turn that inside out?
> Are there times when the compiler knows it must be looking at a keyword,
> not a name?  I suspect not, given that arbitrary expressions can be
> statements, but someone else may have a more knowledgeable opinion.

Sure are. For example, just after `1 `, a name would be an error, while
a keyword would be fine. But, more to the point, there are times when
the parser knows (or could know) it can't be looking at a certain
keyword. Suppose you want to parse at the beginning of an expression
when you see "and". Currently, you're duty-bound to explode, because
"and" cannot begin an expression. You *could* equally well not explode
and, knowing that "and" cannot begin an expression, interpret it
completely unambiguously as a name.



signature.asc
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Crazy idea: allow keywords as names in certain positions

[Rhodri James]

On 13/05/18 19:19, Guido van Rossum wrote:

The idea I had (not for the first time:-)  is that in many syntactic
positions we could just treat keywords as names, and that would free up
these keywords.


I'm not familiar with the innards of the parser and it's wy too long 
since I sat through a parsing course, but can you turn that inside out? 
Are there times when the compiler knows it must be looking at a keyword, 
not a name?  I suspect not, given that arbitrary expressions can be 
statements, but someone else may have a more knowledgeable opinion.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Rhodri James]

On 12/05/18 01:41, Juancarlo Añez wrote:

   while (cmd := get_command()).token != CMD_QUIT:
 cmd.do_something()




while get_command() as cmd:
 if cmd.token == CMD_QUIT:
 break
 cmd.do_something()


This would be exactly one of the patterns Python currently forces on me 
that I don't like.  This "while expr as name" form is not a win in my book.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Rhodri James]

On 14/05/18 10:17, Greg Ewing wrote:

Tim Peters wrote:

Because you never _need_ to use an assignment expression to write a
listcomp/genexp.


This whole discussion started because someone wanted a way
to bind a temporary result for use *within* a comprehension.


I still don't find that argument compelling.  If you have a sufficiently 
complicated comprehension that you need a temporary result bound, IMHO 
you are well on the way to (if not far past) the point where converting 
it into a more traditional for-loop will be much clearer.  I'm not sure 
we should be encouraging people to write less clear code.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Steven D'Aprano]

On Mon, May 14, 2018 at 09:17:03PM +1200, Greg Ewing wrote:
> Tim Peters wrote:
> >Because you never _need_ to use an assignment expression to write a
> >listcomp/genexp.
> 
> This whole discussion started because someone wanted a way
> to bind a temporary result for use *within* a comprehension.
> Those use cases don't require leakage.

To reiterate what Tim already pointed out, that original usecase 
required a way to feed values *into* the comprehension.

https://mail.python.org/pipermail/python-ideas/2018-February/048971.html

There's no need for dedicated syntax for that if we can just set an 
variable and have it show up in the comprehension.


[...]
> It's no harder to explain that than it is to explain
> why
> 
>x = 42
>y = [x * x for x in range(5)]
>print(x)
> 
> prints 42 rather than whatever value was last bound to
> the x in the comprehension.

But it *is* harder to explain why comprehensions are their own scope in 
the first place. They don't look like they are in their own scope. They 
look like any other expression.

No other expression (apart from lambda) runs in its own scope. Every 
other scope relates to a clear lexical separation:

- modules
- classes
- functions

except for comprehensions, which just returns a plain old list.

(Generator expressions are a little fuzzier: they at least are 
equivalent to a lambda-with-yield, if such a thing existed.)

It is sometimes useful to be able to reach into a generator expression 
and manipulate a variable between calls. That would make it a coroutine 
(to use the pre-async language), and that's why yield is an expression 
that returns a value, not just a statement.

And it is sometimes useful to be able to see the value of comprehension 
variables after they have finished running.

We have given up all of that to allow ease of implementation to drive 
the semantics, and that's okay. Its a trade-off.

But we can decide the other way too, and choose more useful semantics 
for assignment expressions over an easier implementation.


> Seems to me it would be easier to explain that *all* names
> bound within a comprehension are local to the comprehension,
> than to have to say that some are and some aren't.

Of course it would be easier to explain. It wouldn't be as useful.

If we wanted "easy to explain", we'd be using BASIC circa 1974, we 
wouldn't have async, generators, comprehensions, exceptions, decorators, 
classes, metaclasses, descriptors, Unicode, or floating point numbers.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A cute Python implementation of itertools.tee

[Franklin? Lee]

The topic reminded me of Stephan Houben's streamtee.
https://github.com/stephanh42/streamtee/blob/master/streamtee.py

It was an attempt at a memory-efficient tee, but it turned out tee was
efficient enough. It uses a thunk-like method and recursion to remove
the need for an explicit linked list.

In the spirit of the topic, I tried to strip it down to the bare essentials:

def tee(iterable, n):
stream = _streamiter(iter(iterable))
return tuple(_wrapper(stream) for _ in range(n))

def _streamiter(itr):
result = (next(itr), _streamiter(itr))
del itr # Release unneeded reference.
while True:
yield result

def _wrapper(stream):
while True:
result, stream = next(stream)
yield result

Unfortunately, this code, too, breaks with the new StopIteration
propagation rules.

On Sun, Apr 15, 2018 at 1:05 AM, Tim Peters  wrote:
> Just for fun - no complaint, no suggestion, just sharing a bit of code
> that tickled me.
>
> The docs for `itertools.tee()` contain a Python work-alike, which is
> easy to follow.  It gives each derived generator its own deque, and
> when a new value is obtained from the original iterator it pushes that
> value onto each of those deques.
>
> Of course it's possible for them to share a single deque, but the code
> gets more complicated.  Is it possible to make it simpler instead?
>
> What it "really" needs is a shared singly-linked list of values,
> pointing from oldest value to newest.  Then each derived generator can
> just follow the links, and yield its next result in time independent
> of the number of derived generators.  But how do you know when a new
> value needs to be obtained from the original iterator, and how do you
> know when space for an older value can be recycled (because all of the
> derived generators have yielded it)?
>
> I ended up with almost a full page of code to do that, storing with
> each value and link a count of the number of derived generators that
> had yet to yield the value, effectively coding my own reference-count
> scheme by hand, along with "head" and "tail" pointers to the ends of
> the linked list that proved remarkably tricky to keep correct in all
> cases.
>
> Then I thought "this is stupid!  Python already does reference
> counting."  Voila!  Vast swaths of tedious code vanished, giving this
> remarkably simple implementation:
>
> def mytee(xs, n):
> last = [None, None]
>
> def gen(it, mylast):
> nonlocal last
> while True:
> mylast = mylast[1]
> if not mylast:
> mylast = last[1] = last = [next(it), None]
> yield mylast[0]
>
> it = iter(xs)
> return tuple(gen(it, last) for _ in range(n))
>
> There's no need to keep a pointer to the start of the shared list at
> all - we only need a pointer to the end of the list ("last"), and each
> derived generator only needs a pointer to its own current position in
> the list ("mylast").
>
> What I find kind of hilarious is that it's no help at all as a
> prototype for a C implementation:  Python recycles stale `[next(it),
> None]` pairs all by itself, when their internal refcounts fall to 0.
> That's the hardest part.
>
> BTW, I certainly don't suggest adding this to the itertools docs
> either.  While it's short and elegant, it's too subtle to grasp easily
> - if you think "it's obvious", you haven't yet thought hard enough
> about the problem ;-)
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Syntax idea: escaping names to avoid keyword ambiguity

[Daniel Moisset]

Following up some of the discussions about the problems of adding keywords
and Guido's proposal of making tokenization context-dependent, I wanted to
propose an alternate way to go around the problem.

My proposal essentially boils down to:

   1. The character "$" can be used as a prefix of identifiers. formally,

   *identifier  * ::= ["$"] xid_start

xid_continue 
*

   2. The "$" character is not part of the name. So the program
   "foo=3;print($foo)" prints 3. So does the program "$foo=3; print(foo)".
   Both set an entry to globals["foo"] and keep globals["$foo"] unset.
   3. if "$" appears in a token, it's always an identifier. So "$with",
   "$if", "$return" are all identifiers.

If you overcome the "yikes, that looks like awk/bash/perl/php, and I don't
like those", and consider it as an escape for "unusual"/"deprecation"
situations, I think it's not a bad chose, and allows to a simple solutions
to many problems that have been in discussion recently and not so recently.
[examples below]

For me the benefits of this approach are:

   - It's very simple to explain how to use and its semantics
   - It (seems to me it) should be easy to explain to a python apprentice
   what a "$" means in code they read on a book/blogpost/manual
   - It's very easy to implement, minimal changes in the tokenizer
   - It's also easy to implement/integrate in other tools (editors with
   syntax highlighters, code formatters, etc)
   - It is easy to see that it's 100% backwards compatible (I understand
   that "$" has never been used in python before)
   - It is relatively unsurprising in the sense that other languages are
   already using $ to label names (there may be some point of confusion to
   people coming from javascript where "$" is a valid character in names and
   is not ignored).
   - It gives python devs and users a clear, easy and universal upgrade
   path when keywords are added (language designers: Add a __future__ import
   to enable keyword in python N+1, add warnings to change kw --> $kw in
   python N+2, and then turn it on by default in python N+3... ; developers:
   add the import when they want to upgrade , and fix their code with a
   search when adding the import or after getting a warning).
   - It allows you to use new features even if some libraries were written
   for older python versions, depending the deprecation period (this could be
   improved with sth I'll write in another email, but that's the topic for
   another proposal)
   - When clashes occur, which they always do, there's one obvious way to
   disambiguate (see today the "class_" argument for gettext.translation, the
   "lambd" argument for random.expovariate, the "class_" filter in libraries
   like pyquery for CSS class, functions like
   pyquery, sqlalchemy.sql.operators.as_ , etc. Not counting all the "cls"
   argument to every classmethod ever)
   - If we're worried about over proliferation of "$" in code, I'm quite
   sure given past experience that just a notice in PEP 8 of "only with $ in
   names to prevent ambiguity" should be more than enough for the community


What are the drawbacks that you find in this?
Best,
   Daniel

[The rest of this post is just examples]

Example 1:

Python 3.92 has just added a future import that makes "given" a keyword.
Then you can do:

# This works because we have no future import
*from* hypothesis *import* given, strategies *as* st

@given(st.integers())
*def* foo(i):
x = f(i)**2 + f(i)**3



if you want to use the new feature (or upgraded to python 3.93 and started
receiving warnings) you can then change it to:


*from* __future__ *import* given_expression

*from* hypothesis *import* $given, strategies *as* st

@$given(st.integers()) *# If you forget the $ you get a SyntaxError*
*def* foo(i):
x = z**2 + z**3 *given* z = f(i)



And also you could do:

*from* __future__ *import* given_expression

*import* hypothesis

@hypothesis.$given(hypothesis.strategies.integers())
*def* foo(i):
x = z**2 + z**3 *given* z = f(i)



Or even, if you want to prevent the "$" all over your code:

*from __future__ import given_expressionfrom* hypothesis *import* $given
*as* hgiven, strategies *as* st


@hgiven(st.integers())
*def* foo(i):
x = z**2 + z**3 *given* z = f(i)





If you have some library which uses a new keyword as a method name (you
can't rename those with import ... as ...), it still works perfectly:

*from* mylib *import* SomeClass

instance = SomeClass()
instance.$given("foo")


This is also helpful as a universal way to solve name clashes between
python keywords and libraries that use some external concept that overlaps
with python

(from https://pythonhosted.org/pyquery/attributes.html ):

>>> *import* pyquery *as* pq
>>> p = pq(*''*)(*'p'*)
>>> p.attr(id=*'hello'*, 

Re: [Python-ideas] Inline assignments using "given" clauses

[Tim Peters]

[Greg Ewing ']
> This whole discussion started because someone wanted a way
> to bind a temporary result for use *within* a comprehension.

It's been noted several times recently that the example PEP 572 gives
as _not_ working:

total = 0
progressive_sums = [total := total + value for value in data]

was the original use case that prompted work on the PEP.  You gotta
admit that's ironic ;-)


> Those use cases don't require leakage.

I already said that - yes - some use cases don't want the leakage.
Others do.  And the vast majority of listcomps/genexps will likely
never use an assignment expression anyway.  If "no leak" wins, "want
leak" can't do what they want at all.  If "want leak" wins, the
subset-of-a-subset "no leak" cases are inconvenienced.

As I said, it's a tradeoff.  You're giving 0 weight to one of the sides.


>> Otherwise it's essentially impossible to explain why:
>>
>> total = 0
>> sums = [total := total + value for value in data]
>> assert sums[-1] == total
>>
>> "blows up", despite that its intent is obvious,unless you first
>> explain to a user how the listcomp is implemented via an invisible
>> synthetic function created by magic, inside of which `total`  has
>> nothing to do with the `total` they see on the first line.

> It's no harder to explain that than it is to explain
> why
>
>x = 42
>y = [x * x for x in range(5)]
>print(x)
>
> prints 42 rather than whatever value was last bound to
> the x in the comprehension.

You're overlooking the most relevant point:  The example blows with an
UnboundLocalError, which can't possibly be explained by looking at the
example as it stands, or by reference to shallow tricks.  You _need_
to drag in stuff about invisible (in the code) synthesized scopes in
which `total` is in fact an unbound local name.

You don't need that heavy machinery to explain why for-target names
don't leak; indeed, shallow implementation tricks were used to achieve
that well before synthetic functions _were_ used to implement
listcomps.


> Seems to me it would be easier to explain that *all* names
> bound within a comprehension are local to the comprehension,
> than to have to say that some are and some aren't.

for-targets are local, assignment expression targets aren't.  I agree
that's harder to explain, but on a scale of 1 to 10?  1.

It's a tradeoff.  And I cheerfully just gave 1 point to the side you
favor.  By my accounting, "leak" is still wining by about 136 to 17
;-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Jacco van Dorp]

I think we're approaching this from the wrong direction.

My point is, expression assignments dont have the complex case as
purpose - most coders wont try to maximize line information density.
If you're doing magic, you might as well spell it out over multiple
lines, because neither := nor given will be readable.

However, in the simple case, it does matter a lot - and there := beats
out given by a mile.

If you're breaking your lines to avoid line length violations, you
might as well put your assignments on a separate lines first. Im
inclined to argue that if assignment expressions of any form forces to
you make a multi-line statement, you're doing it wrong. (in the vast
majority of cases).

Consider it like the ternary operator - good to compact simple
constructs, but while you can use it for black magic, you probably
shouldn't.

How about we just explicitly advice to keep it simple, stupid, and
update PEP 8 to state that if assignment expressions take you to
multi-line, split of the assignment and use statements instead.

(truth be told, the general while (assignment expression isn't as
clear cut as if-uses, but I think the idea holds.))

Jacco
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A comprehension scope issue in PEP 572

[Tim Peters]

[Tim]
>> """
>> An assignment expression binds the target, except in a function F
>> synthesized to implement a list comprehension or generator expression
>> (see XXX).  In the latter case[1], the target is bound in the block
>> containing F, and errors may be detected:  If the target also appears
>> as an identifier target of a `for` loop header in F, a `SyntaxError`
>> exception is raised.  If the block containing F is a class block, a
>> `SyntaxError` exception is raised.
>>
>> Footnote:
>> [1] The intent is that runtime binding of the target occurs as if the
>> binding were performed in the block containing F.  Because that
>> necessarily makes the target not local in F, it's an error if the
>> target also appears in a `for` loop header, which is a local binding
>> for the same target.  If the containing block is a class block, F has
>> no access to that block's scope, so it doesn't make sense to consider
>> the containing block.  The target is bound in the containing block,
>> where it inherits that block's `global` or `nonlocal` declaration if
>> one exists, else establishes that the target is local to that block.
>> """

[Nick]
> This is getting pretty close to being precise enough to be at least
> potentially implementable (thanks!), but there are still two cases that
> would need to be covered:
>
> - what happens inside a lambda expression?

Section 4 of the Reference Manual doesn't contain the word "lambda",
because there's no need to.  "lambda" is just another way to create a
function, and the behavior of functions is already specified.

If you disagree, you mean something by "lambda expression" other than
what I take it to mean, and the best way to illustrate what you do
mean would be via giving a concrete example.  As far as I'm concerned,

   ... (lambda...:  expression) ...

is exactly the same as

def _hidden_unique_name(...):
return expression
... (_hidden_unique_name) 

Even at class scope ;-)

For example,

def f():
g = lambda n:  [(n := n+1) for i in range(1)]
return g(10)

is the same as:

def f():
def g(n):
return  [(n := n+1) for i in range(1)]
return g(10)

When the listcomp synthesizes a function, g's code block will
immediately contain it.  The text at the top says `n` is bound in the
containing block - which is g's.  `n` is already local to `g`, so that
part is just redundant in this case.  The synthetic function will take
10 (via its nonlocal cell), add 1, and again via the cell rebind g's
`n` to 11.  The synthetic function returns [11] and the rebound `n`
vanishes with its execution frame.

But that's all about how functions behave; "lambda" is just incidental.


> - what happens inside another comprehension or generator expression?

I don't know what you're asking about, so I'll just copy part of a
different reply:

"""
Where are the docs explaining how nested comprehensions work today?  I
haven't seen any.  If any such exist, I'd bet nothing about them needs
to be changed.  If none such exist, I don't see a need to write any
just for this PEP.  How do nested expressions of any kind work?  Same
thing.

The only thing the suggestion changes is the scope of assignment
expression targets in synthetic functions created to implement
comprehensions.  That has nothing at all to do with the possibility of
nesting, or with the structure of nesting.  Why do you think it does -
or might?
"""

Note that a day or two ago I posted a complete expansion of what

list(i + sum((i := i+1) for j in range(i)) + i
  for i in range(5))

would do.  There the inner genexp rebinds the outer genexp's local
for-target.  Atrocious.  Here's how you can do the same:

Replace `(i := i+1)` with `(LATER)`.

Generate the complete expansion for how the assignment-expression-free
derived statement is implemented, presumably following the detailed
docs that don't appear to exist ;-) for how that's done.

In the synthetic function created for the inner genexp, add

nonlocal i

at the top and replace

yield (LATER)

with

 yield (i := i+1)

Now I didn't replace anything with "LATER", because I never thought
adding a new binary operator had anything to do with this process to
begin with ;-)  All that's needed is to add cruft _after_ it's done to
establish the intended scopes for assignment statement targets.

If you ask how the inner genexp "knows" that it needs to access the
outer genexp's `i`, it doesn't directly.  It's simply that the
synthetic inner genexp function is nested inside the synthetic outer
genexp function, and the text at top says that the inner genexp binds
`i` in its containing block - which is the block for the synthetic
outer genexp.

If the functions synthesized for nested comprehensions don't _already_
nest in this way, then they'd already screw up merely accessing outer
comprehension names from within inner comprehensions.


Is there a reason to suspect that there's anything inherently unique
to 

Re: [Python-ideas] PEP 572: about the operator precedence of :=

[Brice Parent]

Le 10/05/2018 à 09:32, Terry Reedy a écrit :

On 5/9/2018 11:33 PM, Guido van Rossum wrote:

I now think that the best way out is to rule `:=` in the top level 
expression of an expression statement completely 


I would like to be able to interactively enter

>>> a: = f(2,4)

to have 'a' echoed as well as bound.


This behaviour is reachable by using:

print(a := f(2,4))

Otherwise, I can see coming a new wave of devs saying assignment with 
"=" is outdated, and we should use the new assignment operator ":=". 
Cleaning code by removing useless prints is something, having to go 
through the syntax to remove the ":" of some ":=", and having to test 
everything again to be sure we didn't break anything is another matter 
(long live unit tests! But still...).


Side note: I wouldn't recommand to use `print(a := f(2,4))` either, as 
it would be a print-with-a-side-effect. Which would then also break the 
code if removed the usual way by deleting the line (or commenting it if 
you like messy). Maybe, for another proposal of course, we could allow 
the shell to be more verbose, by printing any explicit assignment 
(whether we used "=" or ":=" wouldn't matter)?


-Brice
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Greg Ewing]

Can someone explain to me why it was considered a bad
thing that for-clauses leaked names in comprehensions,
but it will be a good thing for inline assignments
to leak names from them?

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/