Re: [Python-ideas] A "local" pseudo-function
[MRAB] >> There's another question that hasn't been asked yet: what should locals() >> and globals() return? [Tim, "globals()" is obvious, "locals()" can be surprising now] > ... And here recording the results of some code spelunking Dicts don't really have anything to do with how locals are implemented anymore; calling "locals()" now inside a function _constructs_ a dict on the fly out of lower-level implementation gimmicks, to be kinda-compatible with what locals() returned before nested scopes were introduced. The real distinctions of interest are recorded in the code object now, under these attributes, where I'm giving a fuller explanation than can be found in the docs, and where "referenced" doesn't distinguish between "binding merely retrieved" and "binding is changed": 1. co_varnames: tuple of names of locals not referenced in an enclosed local scope 2. co_cellvars: tuple of names of locals that are referenced in an enclosed local scope 3 .co_freevars: tuple of names local to an enclosing local scope and referenced in this enclosed code "locals()" now generally builds a dict out of all three of those name sources. #1 is the only one that made sense before nested scopes were introduced. The union of #1 and #2 is the set of names local to the code's scope; CPython implements their bindings in different ways, so needs to distinguish them. The names in #3 aren't local to the code block at all (they are local to some enclosing local scope), but for whatever reason are included in the code's "locals()" dict anyway. I would not have included them. For concreteness: def disp(func): print("for", func.__name__) code = func.__code__ for attr in "co_varnames", "co_cellvars", "co_freevars": print(" ", attr, getattr(code, attr)) def outer(): outer_only = 1 outer_used_inner = 2 outer_bound_by_inner = 3 def inner(): nonlocal outer_bound_by_inner inner_only = 4 outer_bound_by_inner = 5 inner_only = outer_used_inner inner() disp(outer) disp(inner) outer() And its output: for outer co_varnames ('outer_only', 'inner') co_cellvars ('outer_bound_by_inner', 'outer_used_inner') co_freevars () for inner co_varnames ('inner_only',) co_cellvars () co_freevars ('outer_bound_by_inner', 'outer_used_inner') ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
Steven D'Aprano wrote: y = 1 def func(): x = 2 return x+y Here, there's a local environment as well as an implicit global one. Surely we don't want to call this a closure? Python probably isn't the best choice of language for the point you're making, because even top-level functions in Python *do* carry a reference to their environment (the dict of the module they were defined in). C would be a better choice: int y = 1; int func(void) { int x = 2; return x + y; } int (*fp)() = func; Even here, I don't think there's anything wrong with calling fp a closure. It's just that it's a particularly simple form of closure -- since there's only one global environment, we don't need to bother explicitly carrying it around. However, that doesn't mean all functions are closures. An example of a non-closure might be a C++ member function pointer. It points to a piece of code, but you can't use it on its own, because it's missing a piece of environmental information (a value for "this") that you need to supply by hand when you call it. we have to consider what happens if we come across a Pascal implementation which *doesn't* use a (code_address, frame_pointer) pair. Is that no longer Pascal? In anything that implements Pascal semantics, a function parameter is going to have to be represented by *something* that specifies both code and environment. Code alone is not enough. > If we accept that "Pascal has closures", they're pretty crap closures, > since they don't let you do any of the interesting and useful things you > can do in languages with "real" closures like Scheme and Python. Not all of them, but it can do some of them. They're more useful than in Modula-2, for example, where nested functions can't even be passed as parameters. Anyway, that's why I said "a limited form of closure". You could read that two ways: (1) It's totally a closure, but there are limitations on what you can do with it (i.e. it's not first class). Or (2) it has some of the characteristics of a closure, but not all of them. In my mind I tend towards (1), but it doesn't really matter. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[MRAB] >>> Would/should it be possible to inject a name into a local scope? You can't >>> inject into a function scope, and names in a function scope can be >>> determined statically (they are allocated slots), so could the same kind of >>> thing be done for names in a local scope? ... [Steven D'Aprano] > I don't know what MRAB means by "inject", but I know what *I* mean, and > I have a real use-case for it. > > There is a long-running micro-optimization, often championed by Raymond, > for avoiding slow global lookups (and even slower builtin lookups, since > they require a global lookup to fail first) by turning them in local > lookups at function-definition time. Before nested scopes, it was also used to help nested functions call each other; e.g., def f(): def g(): ... # h needs to call g, but can't see f's locals def h(g=g): # but default arg expressions can see f's locals g() # works great :-) > E.g. some methods from the random.Random class: > > def randrange(self, start, stop=None, step=1, _int=int): > ... > def _randbelow(self, n, int=int, maxsize=1< Method=_MethodType, BuiltinMethod=_BuiltinMethodType): > ... > > (copied from 3.5, I don't know what the most recent version looks like) Doesn't matter; code like that has been in the std lib since the start, although Raymond is inordinately fond of it ;-) > That's a nice way to make the binding: > > _int = int > > occur once only, instead of putting it inside the function body which > then needs to be executed on ever call. Effectively it's a static > variable for the method, one which persists from one call to the next > without requiring re-initialisation. > > But it's ugly :-( Error-prone too, because absolutely nothing stops a user from calling these things with more positional arguments than are intended. I don't want to bother looking now, but there was _some_ "bug report" complaining that one of these "default arguments" went away somewhere, which a user was specifying intentionally to override some accidental implementation detail. However, while it's dead obviously "error prone" in theory, it's remarkable how few times I've heard of anyone getting burned by it in practice. > The function signature is full of *implementation details* instead of > the parameters needed for the method's interface. Look at _randbelow > which takes one actual parameter, n, plus FIVE fake parameters, int, > maxsize, type, Method and BuiltinMethod, none of which should ever be > passed arguments. > > So when I talk about injecting values into a function, that is the sort > of thing I'm referring to: at function definition time, push or inject a > reference to a known value (say, the builtin int) into something which > behaves as a static variable. > > It would be nice if we could do that without polluting the function > signature. I'm surprised that hasn't already been done. > I'll admit that the name "inject" came to me when I was > thinking of some hypothetical decorator: > > @inject(int=int, maxsize=1< def _randbelow(self, n): > ... > > that somehow pushed, or *injected*, those bindings into the function, > turning them into locals, but in an alternative universe where Guido > loved making new keywords, I'd use a static initialisation block and > stick it inside the def: > > def _randbelow(self, n): > static: > # this gets executed once only, at function definition time > int=int > maxsize=1< type=type > # body of _randbelow Blame computer scientists. I started my paying career working on Cray Research's Fortran compiler, a language in which NO words were reserved. The syntax was such that it was always possible to determine whether a language keyword or a user-supplied name was intended. That worked out great. But it seemed to require ad hoc parsing tricks. Computer scientists invented "general" parser generators, and then every new language started declaring that some words were reserved for the language's use. That was just to make things easier for parser-generator authors, not for users ;-) Maybe we could steal a trick from the way the async statements ("async def", "async for", "async with") were added without making "async" a reserved word? Even if user code already uses the name "static", just as in Fortran the only way to parse static: that makes sense is that it's introducing a block; USER_NAME: as a statement has no meaning. In any other context, the block-opening meaning of "static" makes no sense, so it must mean a user-defined name then. We could also add, e.g., MRAB local a, b, c: and Uncle Timmy not really local at all a, b, c: without introducing any actual ambiguity in code already using any of the leading words on those lines. It would be fun to
Re: [Python-ideas] A "local" pseudo-function
[MRAB] >>> Imagine renaming the specified names that are declared 'local' >>> throughout the nested portion: >>> >>> def f(): >>> a = 10 >>> local local_a: >>> def showa(): >>> print("a is", local_a) >>> showa() # Raises NameError in showa because nothing bound to >>> local_a yet. >>> local_a = 20 >>> showa() # 20 >>> local_a = 30 >>> showa() # Raises NameError in f because local_a not defined. [Tim] >> In a later message I showed executable-today Python code that I >> believe models your intent. That code does indeed display "30" for >> the last call, and while I myself don't see that it's _useful_ yet, >> the model is incomprehensible if it doesn't. >> >> It doesn't matter that local_a isn't defined at function scope, >> because showa() refers to the locals _in_ the "local a:" scope. The >> last value bound to local_a was 30, so that's what showa() needs to >> show. That the name `showa` is _bound_ in f's locals has nothing to >> do with whose locals showa() _uses_. Other current messages about >> "closures" go on more about that. [MRAB] > Ah, yes. I see what you mean. > > There's another question that hasn't been asked yet: what should locals() > and globals() return? Under my meaning, the same answer as always: exactly the same as if "local a:' were replaced by "if True:" ;-) Under yours, you're not going to like the answer. It seems obvious that since globals() refers to the module dict, nothing about that would change. You can add prints to my workalike-code-under-current-Python snippet to see what locals() _does_ return in various places. It's not what you'd expect (that locals() inside the `local a:` block is a dict with only key "local_a"), because of what I believe are undocumented implementation details. This is much easier to see in a bare-bones example: def f(): b = 12 def g(): nonlocal b b = 17 a = 42 print("inner", locals()) g() print("outer", locals()) f() The output: inner {'a': 42, 'b': 17} outer {'g': .g at 0x024E28C44AE8>, 'b': 17} That is, despite that `b` is explicitly declared as `nonlocal` in the inner function, and does in fact belong to the outer scope, it nevertheless shows up inside the inner function's locals(). The compiler isn't confused, but `locals()` existed long before nested scopes were added, and, e.g., no "nonlocals()` builtin was added later. PEP 227 (which introduced nested scopes) explicitly declined to add one, and said "it will not be possible to gain dictionary-style access to all visible scopes". I was quite surprised to see "b" as a key in the inner locals() above! But so long as it is, any new gimmick building on the current implementation would inherit that behavior. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-05-01 19:12, Tim Peters wrote: [MRAB] > Imagine renaming the specified names that are declared 'local' throughout > the nested portion: > > def f(): > a = 10 > local local_a: > def showa(): > print("a is", local_a) > showa() # Raises NameError in showa because nothing bound to local_a yet. > local_a = 20 > showa() # 20 > local_a = 30 > showa() # Raises NameError in f because local_a not defined. In a later message I showed executable-today Python code that I believe models your intent. That code does indeed display "30" for the last call, and while I myself don't see that it's _useful_ yet, the model is incomprehensible if it doesn't. It doesn't matter that local_a isn't defined at function scope, because showa() refers to the locals _in_ the "local a:" scope. The last value bound to local_a was 30, so that's what showa() needs to show. That the name `showa` is _bound_ in f's locals has nothing to do with whose locals showa() _uses_. Other current messages about "closures" go on more about that. Ah, yes. I see what you mean. There's another question that hasn't been asked yet: what should locals() and globals() return? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[MRAB] > By "inject" I mean putting a name into a namespace: > > import my_module > my_module.foo = 'FOO' > > You can't insert a name into a function's body to make a new local variable. So you do mean at runtime, I think. Then as before, you can do that with module and the builtin namespaces now, but all function locals need to be identifiable at compile time now. People often get confused about that when they read that "it's a binding" that makes a name local to a scope, but it's not "_doing_ a binding" that makes it local, merely that the name _syntactically_ appears as a target in a binding construct. That analysis is entirely done at compile time. In Python's very early days, all namespaces could be dynamically altered at any time in any way. As time went on, more & more restrictions were placed on runtime alteration of local scopes. I don't believe that, in current Python 3, dynamic alteration of local scopes is supported in any case anymore. Perhaps the last to go was runtime alteration of locals via `exec`. This still works in Python 2: >>> def f(): ... exec 'i = 1+2' ... print i >>> f() 3 >>> i # showing that it was function-local `i`, not global Traceback (most recent call last): File "", line 1, in NameError: name 'i' is not defined `exec` was a statement type in Python 2, so the compiler could recognize that and generate radically different name-lookup code in a scope containing an `exec` statement. But in Python 3, `exec` is just another builtin function, and the compiler knows nothing about what it does. Similar code run in Python 3 has no effect on f's locals. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-05-01 16:23, Steven D'Aprano wrote: On Mon, Apr 30, 2018 at 08:52:13PM -0500, Tim Peters wrote: > Would/should it be possible to inject a name into a local scope? You can't > inject into a function scope, and names in a function scope can be > determined statically (they are allocated slots), so could the same kind of > thing be done for names in a local scope? Sorry, I'm unclear on what "inject a name into a local scope" means. Do you mean at runtime? I don't know what MRAB means by "inject", but I know what *I* mean, and I have a real use-case for it. By "inject" I mean putting a name into a namespace: import my_module my_module.foo = 'FOO' You can't insert a name into a function's body to make a new local variable. [snip] ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Tue, May 01, 2018 at 09:26:09PM +1200, Greg Ewing wrote: > Steven D'Aprano wrote: > >So what was the closure? If the surrounding function was still running, > >there was no need to capture the running environment in a closure? > > You seem to be interpreting the word "closure" a bit > differently from most people. It doesn't imply anything > about whether a surrounding function is still running or > not. > > A closure is just a piece of code together with a runtime > environment. That's not *all* it is, because obviously *every* function has both a piece of code and a runtime environment. y = 1 def func(): x = 2 return x+y Here, there's a local environment as well as an implicit global one. Surely we don't want to call this a closure? If we do, then all functions are closures and the term is just a synonym for function. Wikipedia gives a definition: a closure (also lexical closure or function closure) is a technique for implementing lexically scoped name binding in a language with first-class functions. Operationally, a closure is a record storing a function together with an environment. https://en.wikipedia.org/wiki/Closure_%28computer_programming%29 but also warns As different languages do not always have a common definition of the lexical environment, their definitions of closure may vary also and explicitly says that Pascal (at least standard Pascal) didn't do closures: Traditional imperative languages such as Algol, C and Pascal either do not support nested functions (C) or do not support calling nested functions after the enclosing function has exited (GNU C, Pascal), thus avoiding the need to use closures. Neal Gafter gives what I believe is *the* standard definition of closures, at least in academic and functional programming circles, probably taken from Guy Steele and Scheme: A closure is a function that captures the bindings of free variables in its lexical context. http://gafter.blogspot.com.au/2007/01/definition-of-closures.html Since Steele basically wrote the book on closures, I think we ought to take his definition seriously :-) (That's also the definition I was thinking of.) By this definition, if there are no free variables, or no captured bindings, then its not a closure. func() above isn't a closure, since "x" isn't a free variable, and while "y" is, the value of it isn't captured. Unfortunately, the terminology has become a real mess, especially since the technique has moved into languages like Ruby and Javascript. Martin Fowler declares that closures, lambdas and blocks are the same thing: https://martinfowler.com/bliki/Lambda.html although he does admit that technically you can have lambdas that aren't closures. I think Fowler is abusing the terminology since all three of his terms are orthogonal: - closures can be created with def or lambda and are not necessarily anonymous; - lambda comes from the lambda calculus, where it refers to anonymous function expressions, not specifically whether they capture the value of free variables in their environment; - "block" typically refers to the structure of the source code; in Ruby it also refers to a kind of first-class anonymous callable object. Such blocks may, or may not, contain free variables. At least I don't use this definition of closure: "a closure is a callback that Just Works." https://reprog.wordpress.com/2010/02/27/closures-finally-explained/ *wink* > In typical Pascal implementations, a closure > is represented by a (code_address, frame_pointer) pair, > where the frame_pointer points to a chain of lexically > enclosing stack frames. The language rules make it possible > to manage the frames strictly stack-wise, which simplifies > the memory management, but that doesn't make the closure > any less of a closure. Well... it's a pretty feeble closure, since Pascal doesn't support functions as first-class values. More like second-class. The c2 wiki describes Pascal: However, when a function is passed to another function and later called, it will execute in the lexical context it was defined in, so it is, IN SOME SENSE [emphasis added], "closed over" that context. http://wiki.c2.com/?LexicalClosure but I think that sense is the same sense that func() above is "closed over" the free value y. The loosest possible sense. If we accept that "Pascal has closures", they're pretty crap closures, since they don't let you do any of the interesting and useful things you can do in languages with "real" closures like Scheme and Python. And because it is based on an implementation detail (as you say, it is only *typical*, not mandatory, for Pascal inner functions to be implemented this way), we have to consider what happens if we come across a Pascal implementation which *doesn't* use a (code_address, frame_pointer) pair. Is that no longer Pascal? This is the interesting, and
Re: [Python-ideas] A "local" pseudo-function
On 2018-05-01 04:40, Tim Peters wrote: [MRAB] >> Any binding that's not specified as local is bound in the parent scope: [Tim] > Reverse-engineering the example following, is this a fair way of > making that more precise? > > Given a binding-target name N in scope S, N is bound in scope T, where > T is the closest-containing scope (which may be S itself) for which T > is either > > 1. established by a "local:" block that declares name N > > or > > 2. not established by a "local: block Here's an example where I don't know what the consequences of "the rules" should be: def f(): a = 10 local a: def showa(): print("a is", a) showa() # 10 a = 20 showa() # 20 a = 30 showa() # 10 The body of "showa" is lexically nested in 'local', so its name "a" is the local scope's "a", not the function scope's "a". Imagine renaming the specified names that are declared 'local' throughout the nested portion: def f(): a = 10 local local_a: def showa(): print("a is", local_a) showa() # Raises NameError in showa because nothing bound to local_a yet. local_a = 20 showa() # 20 local_a = 30 showa() # Raises NameError in f because local_a not defined. The comments show what the output would be under the "nothing about scope rules change" meaning. They're all obvious (since there is is no new scope then - it's all function-local). But under the other meaning ...? The twist here is that `def` is an executable statement in Python, and is a "binding site" for the name of the function being defined. So despite that `showa` appears to be defined in a new nested lexical scope, it's _actually_ bound as a function-local name. That's bound to be surprising to people from other languages: "I defined it in a nested lexical scope, but the name is still visible after that scope ends?". I don't know what the first `showa()` is intended to do. Presumably `a` is unbound at the start of the new nested scope? So raises NameError? If so, comment that line out so we can make progress ;-) It seems clear that the second `showa()` will display 20 under any reading. But the third? Now we're out of the `local a:` scope, but call a function whose textual definition was inside that scope. What does `showa()` do now to find a's value? f's local `a` had nothing to do with the `a` in the nested scope, so presumably it shouldn't display 10 now. What should it do? Does the final state of the nested scope's locals need to preserved so that showa() can display 30 instead? Or ...? Not necessarily complaining - just fleshing out a bit my earlier claim that a world of semantics need to be defined if anything akin to a "real scope" is desired. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-05-01 02:52, Tim Peters wrote: [MRAB] > ... > The intention is that only the specified names are local. > > After all, what's the point of specifying names after the 'local' if _any_ > binding in the local scope was local? Don't look at me ;-) In the absence of use cases, I don't know which problem(s) you're trying to solve. All the use cases I've looked at are adequately addressed by having some spelling of "local:" change nothing at all about Python's current scope rules. If you have uses in mind that require more than just that, I'd need to see them. >> ... >> If you agree that makes the feature probably unusable, you don't get >> off the hook by saying "no, unlike current Python scopes, binding >> sites have nothing to do with what's local to a new lexical scope >> introduced by 'local:'". The same question raised in the example >> above doesn't go away: in which scope(s) are 'r1' and 'r2' to be >> bound? > Any binding that's not specified as local is bound in the parent scope: Reverse-engineering the example following, is this a fair way of making that more precise? Given a binding-target name N in scope S, N is bound in scope T, where T is the closest-containing scope (which may be S itself) for which T is either 1. established by a "local:" block that declares name N or 2. not established by a "local: block By "parent scope" I meant the function or global scope, a scope that's not a "local:" scope. I couldn't think of a good name for it. [snip] ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Tim] > ... > Here's an example where I don't know what the consequences of "the > rules" should be: > > def f(): > a = 10 > local a: > def showa(): > print("a is", a) > showa() # 10 > a = 20 > showa() # 20 > a = 30 > showa() # 10 > > The comments show what the output would be under the "nothing about > scope rules change" meaning. They're all obvious (since there is is > no new scope then - it's all function-local). > > But under the other meaning ...? > > The twist here is that `def` is an executable statement in Python, and > is a "binding site" for the name of the function being defined. So > despite that `showa` appears to be defined in a new nested lexical > scope, it's _actually_ bound as a function-local name. That's bound > to be surprising to people from other languages: "I defined it in a > nested lexical scope, but the name is still visible after that scope > ends?". > > I don't know what the first `showa()` is intended to do. Presumably > `a` is unbound at the start of the new nested scope? So raises > NameError? If so, comment that line out so we can make progress ;-) > > It seems clear that the second `showa()` will display 20 under any reading. > > But the third? Now we're out of the `local a:` scope, but call a > function whose textual definition was inside that scope. What does > `showa()` do now to find a's value? f's local `a` had nothing to do > with the `a` in the nested scope, so presumably it shouldn't display > 10 now. What should it do? > Does the final state of the nested scope's locals need to preserved so > that showa() can display 30 instead? Or ...? Just noting that a sane way to answer such questions is to model the intent with nested functions in current Python. It's annoying because you have to explicitly declare the _non_-local names instead, and also make dummy assignments to those names to tell Python in which scope they _should_ be bound. So, e.g., for the above you can write this: def f(): a = 10 # Have to give `showa` _some_ binding in its intended scope, # else the "nonlocal showa" below is a compile-time error showa = None def local_a(): nonlocal showa def showa(): print("a", a) #showa() # raises NameError a = 20 showa() # 20 a = 30 local_a() del local_a showa() # 30 print("but f's a is", a) # 10 The comments show what happens when you call `f()`, matching the guesses in the original message. One thing to note: if this does model the intent, it's essentially proof that it's implementable without intractable effort; e.g., the machinery needed to implement funky closures already exists. But another: since I've never seen code anything like this before, that casts doubt on whether the semantics are actually useful in real life. That's why I always ask for real use cases ;-) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Mon, Apr 30, 2018 at 08:52:13PM -0500, Tim Peters wrote: > > Would/should it be possible to inject a name into a local scope? You can't > > inject into a function scope, and names in a function scope can be > > determined statically (they are allocated slots), so could the same kind of > > thing be done for names in a local scope? > > Sorry, I'm unclear on what "inject a name into a local scope" means. > Do you mean at runtime? I don't know what MRAB means by "inject", but I know what *I* mean, and I have a real use-case for it. There is a long-running micro-optimization, often championed by Raymond, for avoiding slow global lookups (and even slower builtin lookups, since they require a global lookup to fail first) by turning them in local lookups at function-definition time. E.g. some methods from the random.Random class: def randrange(self, start, stop=None, step=1, _int=int): ... def _randbelow(self, n, int=int, maxsize=1<
Re: [Python-ideas] A "local" pseudo-function
Steven D'Aprano wrote: So what was the closure? If the surrounding function was still running, there was no need to capture the running environment in a closure? You seem to be interpreting the word "closure" a bit differently from most people. It doesn't imply anything about whether a surrounding function is still running or not. A closure is just a piece of code together with a runtime environment. In typical Pascal implementations, a closure is represented by a (code_address, frame_pointer) pair, where the frame_pointer points to a chain of lexically enclosing stack frames. The language rules make it possible to manage the frames strictly stack-wise, which simplifies the memory management, but that doesn't make the closure any less of a closure. Contrast this with Modula-2, where only top-level functions can be passed as parameters. When you pass a function in Modula-2, only the code address is passed, with no environment pointer. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Tue, May 01, 2018 at 06:06:06PM +1200, Greg Ewing wrote: > Steven D'Aprano wrote: > >Pascal, for example, had lexical scoping back in the 1970s, but > >no closures. > > Well, it kind of had a limited form of closure. You could pass > a procedure or function *in* to another procedure or function > as a parameter, but there was no way to return one or pass it > out in any way. This ensured that the passed-in procedure or > function couldn't outlive its lexical environment. So what was the closure? If the surrounding function was still running, there was no need to capture the running environment in a closure? Not a rhetorical question, I'm genuinely unsure. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
Steven D'Aprano wrote: Pascal, for example, had lexical scoping back in the 1970s, but no closures. Well, it kind of had a limited form of closure. You could pass a procedure or function *in* to another procedure or function as a parameter, but there was no way to return one or pass it out in any way. This ensured that the passed-in procedure or function couldn't outlive its lexical environment. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[MRAB] >> Any binding that's not specified as local is bound in the parent scope: [Tim] > Reverse-engineering the example following, is this a fair way of > making that more precise? > > Given a binding-target name N in scope S, N is bound in scope T, where > T is the closest-containing scope (which may be S itself) for which T > is either > > 1. established by a "local:" block that declares name N > > or > > 2. not established by a "local: block Here's an example where I don't know what the consequences of "the rules" should be: def f(): a = 10 local a: def showa(): print("a is", a) showa() # 10 a = 20 showa() # 20 a = 30 showa() # 10 The comments show what the output would be under the "nothing about scope rules change" meaning. They're all obvious (since there is is no new scope then - it's all function-local). But under the other meaning ...? The twist here is that `def` is an executable statement in Python, and is a "binding site" for the name of the function being defined. So despite that `showa` appears to be defined in a new nested lexical scope, it's _actually_ bound as a function-local name. That's bound to be surprising to people from other languages: "I defined it in a nested lexical scope, but the name is still visible after that scope ends?". I don't know what the first `showa()` is intended to do. Presumably `a` is unbound at the start of the new nested scope? So raises NameError? If so, comment that line out so we can make progress ;-) It seems clear that the second `showa()` will display 20 under any reading. But the third? Now we're out of the `local a:` scope, but call a function whose textual definition was inside that scope. What does `showa()` do now to find a's value? f's local `a` had nothing to do with the `a` in the nested scope, so presumably it shouldn't display 10 now. What should it do? Does the final state of the nested scope's locals need to preserved so that showa() can display 30 instead? Or ...? Not necessarily complaining - just fleshing out a bit my earlier claim that a world of semantics need to be defined if anything akin to a "real scope" is desired. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[MRAB] > ... > The intention is that only the specified names are local. > > After all, what's the point of specifying names after the 'local' if _any_ > binding in the local scope was local? Don't look at me ;-) In the absence of use cases, I don't know which problem(s) you're trying to solve. All the use cases I've looked at are adequately addressed by having some spelling of "local:" change nothing at all about Python's current scope rules. If you have uses in mind that require more than just that, I'd need to see them. >> ... >> If you agree that makes the feature probably unusable, you don't get >> off the hook by saying "no, unlike current Python scopes, binding >> sites have nothing to do with what's local to a new lexical scope >> introduced by 'local:'". The same question raised in the example >> above doesn't go away: in which scope(s) are 'r1' and 'r2' to be >> bound? > Any binding that's not specified as local is bound in the parent scope: Reverse-engineering the example following, is this a fair way of making that more precise? Given a binding-target name N in scope S, N is bound in scope T, where T is the closest-containing scope (which may be S itself) for which T is either 1. established by a "local:" block that declares name N or 2. not established by a "local: block > local b: > local c: > c = 0 # Bound in the "local c" scope. By clause #1 above, "c" is declared in the starting "local:" scope. > b = 0 # Bound in the "local b" scope. By clause #1 above, after searching one scope up to find `b` declared in a "local:" scope > a = 0 # Bound in the main scope (function, global, whatever) By clause #2 above, after searching two scopes up and not finding any "local:" scope declaring name "a". By your original "the parent scope", I would have expected this be bound in the "local b:" scope (which is the parent scope of the "local c:" scope). So that's _a_ possible answer. It's not like the scoping rules in any other language I'm aware of, but then Python's current scoping rules are unique too. Are those useful rules? Optimal? The first thing that popped into your head? The third? Again I'd need to see use cases to even begin to guess. I agree it's well defined, though, and so miles ahead of most ideas ;-) ... >> Note: most of this doesn't come up in most other languages because >> they require explicitly declaring in which scope a name lives. >> Python's "infer that in almost all cases instead from examining >> binding sites" has consequences. > Would/should it be possible to inject a name into a local scope? You can't > inject into a function scope, and names in a function scope can be > determined statically (they are allocated slots), so could the same kind of > thing be done for names in a local scope? Sorry, I'm unclear on what "inject a name into a local scope" means. Do you mean at runtime? In Python's very early days, all scope namespaces were implemented as Python dicts, and you could mutate those at runtime any way you liked. Name lookup first tried the "local" dict ("the" because local scopes didn't nest), then the "global" dict, then the "builtin" dict. Names could be added or removed from any of those at will. People had a lot of fun playing with that, but nobody seriously complained as that extreme flexibility was incrementally traded away for faster runtime. So now take it as given that the full set of names in a local scope must be determinable at compile-time (modulo whatever hacks may still exist to keep old "from module import *" code working - if any still do exist). I don't believe CPython has grown any optimizations preventing free runtime mutation of global (module-level) or builtin namespace mappings, but I may be wrong about that. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
Tim Peters writes: > Meaning "wonderfully clear" to the compiler, not necessarily to you > ;-) Is the compiler African or European (perhaps even Dutch)? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-30 21:41, Tim Peters wrote: [MRAB] > I think it should be lexically scoped. That's certainly arguable, but that's why I like real-code driven design: abstract arguments never end, and often yield a dubious in-real-life outcome after one side is worn out and the other side "wins" by attrition ;-) > The purpose of 'local' would be to allow you to use a name that _might_ be > used elsewhere. > > The problem with a dynamic scope is that you might call some global function > from within the local scope, but find that it's "not working correctly" > because you've inadvertently shadowed a name that the function refers to. Already explained at excessive length that there's nothing akin to "dynamic scopes" here, except that both happen to restore a previous binding at times. That's a shallow coincidence. It's no more "dynamic scope" than that savea = a try: a += 1 f(a) finally: a = savea is "dynamic scoping". It's merely saving/restoring a binding across a block of code. > Imagine, in a local scope, that you call a global function that calls 'len', > but you've shadowed 'len'... I'm not clear on whether you picked the name of a builtin to make a subtle point not spelled out, but I don't think it matters. Regardless of whether `len` refers to a builtin or a module global inside your global function now, the _current_ def f(): len = 12 global_function() has no effect at all on the binding of `len` seen inside `global_function`. Because my understanding of "local:" changes absolutely nothing about Python's current scope rules, it's necessarily the case that the same would be true in: def f(): local len: len = 12 call_something() The only difference from current semantics is that if print(len) were added after the `local:` block, UnboundLocalError would be raised (restoring the state of the function-local-with-or-without-'local:' `len` to what it was before the block). To have "local:" mean "new nested lexical scope" instead requires specifying a world of semantics that haven't even been mentioned yet. In Python today, in the absence of `global` and `nonlocal` declarations, the names local to a given lexical scope are determined entirely by analyzing binding sites. If you intend something other than that, then it needs to be spelled out. But if you intend to keep "and names appearing in binding sites are also local to the new lexical scope", I expect that's pretty much useless. For example, def f(): ... local x. y: x = a*b y = a/b r1, r2 = x+y, x-y That is, the programmer surely doesn't _intend_ to throw away r1 and r2 when the block ends. If they have to add a nonlocal r1, r2 declaration at the top of the block, maybe it would work as intended. But it still wouldn't work unless `r1` and `r2` _also_ appeared in binding sites in an enclosing lexical scope. If they don't, you'd get a compile-time error like The intention is that only the specified names are local. After all, what's the point of specifying names after the 'local' if _any_ binding in the local scope was local? SyntaxError: no binding for nonlocal 'r1' found To be more accurate, the message should really say "sorry, but I have no idea in which scope you _intend_ 'r1' to live, because the only way I could know that is to find a binding site for 'r1', and I can't find any except inside _this_ scope containing the 'nonlocal'". But that's kind of wordy ;-) If you agree that makes the feature probably unusable, you don't get off the hook by saying "no, unlike current Python scopes, binding sites have nothing to do with what's local to a new lexical scope introduced by 'local:'". The same question raised in the example above doesn't go away: in which scope(s) are 'r1' and 'r2' to be bound? Any binding that's not specified as local is bound in the parent scope: local b: local c: c = 0 # Bound in the "local c" scope. b = 0 # Bound in the "local b" scope. a = 0 # Bound in the main scope (function, global, whatever) There's more than one plausible answer to that, but in the absence of real use cases how can they be judged? Under "'local:' changes nothing at all about Python's scopes", the answer is obvious: `r1` and `r2` are function locals (exactly the same as if "local:" hadn't been used). There's nothing new about scope to learn, and the code works as intended on the first try ;-) Of course "local:" would be a misleading name for the construct, though. Going back to your original example, where a global (not builtin) "len" was intended: def f(): global len # LINE ADDED HERE local len: len = 12 global_function() yes, in _that_ case the global-or-builtin "len" seen inside `global_function` would change under my "nothing about scoping changes"
Re: [Python-ideas] A "local" pseudo-function
[MRAB] > I think it should be lexically scoped. That's certainly arguable, but that's why I like real-code driven design: abstract arguments never end, and often yield a dubious in-real-life outcome after one side is worn out and the other side "wins" by attrition ;-) > The purpose of 'local' would be to allow you to use a name that _might_ be > used elsewhere. > > The problem with a dynamic scope is that you might call some global function > from within the local scope, but find that it's "not working correctly" > because you've inadvertently shadowed a name that the function refers to. Already explained at excessive length that there's nothing akin to "dynamic scopes" here, except that both happen to restore a previous binding at times. That's a shallow coincidence. It's no more "dynamic scope" than that savea = a try: a += 1 f(a) finally: a = savea is "dynamic scoping". It's merely saving/restoring a binding across a block of code. > Imagine, in a local scope, that you call a global function that calls 'len', > but you've shadowed 'len'... I'm not clear on whether you picked the name of a builtin to make a subtle point not spelled out, but I don't think it matters. Regardless of whether `len` refers to a builtin or a module global inside your global function now, the _current_ def f(): len = 12 global_function() has no effect at all on the binding of `len` seen inside `global_function`. Because my understanding of "local:" changes absolutely nothing about Python's current scope rules, it's necessarily the case that the same would be true in: def f(): local len: len = 12 call_something() The only difference from current semantics is that if print(len) were added after the `local:` block, UnboundLocalError would be raised (restoring the state of the function-local-with-or-without-'local:' `len` to what it was before the block). To have "local:" mean "new nested lexical scope" instead requires specifying a world of semantics that haven't even been mentioned yet. In Python today, in the absence of `global` and `nonlocal` declarations, the names local to a given lexical scope are determined entirely by analyzing binding sites. If you intend something other than that, then it needs to be spelled out. But if you intend to keep "and names appearing in binding sites are also local to the new lexical scope", I expect that's pretty much useless. For example, def f(): ... local x. y: x = a*b y = a/b r1, r2 = x+y, x-y That is, the programmer surely doesn't _intend_ to throw away r1 and r2 when the block ends. If they have to add a nonlocal r1, r2 declaration at the top of the block, maybe it would work as intended. But it still wouldn't work unless `r1` and `r2` _also_ appeared in binding sites in an enclosing lexical scope. If they don't, you'd get a compile-time error like SyntaxError: no binding for nonlocal 'r1' found To be more accurate, the message should really say "sorry, but I have no idea in which scope you _intend_ 'r1' to live, because the only way I could know that is to find a binding site for 'r1', and I can't find any except inside _this_ scope containing the 'nonlocal'". But that's kind of wordy ;-) If you agree that makes the feature probably unusable, you don't get off the hook by saying "no, unlike current Python scopes, binding sites have nothing to do with what's local to a new lexical scope introduced by 'local:'". The same question raised in the example above doesn't go away: in which scope(s) are 'r1' and 'r2' to be bound? There's more than one plausible answer to that, but in the absence of real use cases how can they be judged? Under "'local:' changes nothing at all about Python's scopes", the answer is obvious: `r1` and `r2` are function locals (exactly the same as if "local:" hadn't been used). There's nothing new about scope to learn, and the code works as intended on the first try ;-) Of course "local:" would be a misleading name for the construct, though. Going back to your original example, where a global (not builtin) "len" was intended: def f(): global len # LINE ADDED HERE local len: len = 12 global_function() yes, in _that_ case the global-or-builtin "len" seen inside `global_function` would change under my "nothing about scoping changes" reading, but would not under your reading. That's worth _something_ ;-) But without fleshing out the rules for all the other stuff (like which scope(s) own r1 and r2 in the example above) I can't judge whether it's worth enough to care. All the plausibly realistic use cases I've considered don't _really_ want a full-blown new scope (just robust save/restore for a handful of explicitly given names), and the example just above is contrived in comparison. Nobody types "global len" unless they
Re: [Python-ideas] A "local" pseudo-function
[Tim, still trying to define `iseven` in one statement] > even = (lambda n: n == 0 or odd(n-1)) > odd = (lambda n: False if n == 0 else even(n-1)) > iseven = lambda n: even(n) ... > [and the last attempt failed because a LOAD_GLOBAL was generated >instead of a more-general runtime lookup] So if I want LOAD_FAST instead, that has to be forced, leading to a one-statement definition that's wonderfully clear: iseven = lambda n: ( lambda n=n, even = (lambda n, e, o: n == 0 or o(n-1, e, o)), odd = (lambda n, e, o: False if n == 0 else e(n-1, e, o)): even(n, even, odd) )() Meaning "wonderfully clear" to the compiler, not necessarily to you ;-) Amusingly enough, that's a lot like the tricks we sometimes did in Python's early days, before nested scopes were added, to get recursive - or mutually referring - non-global functions to work at all. That is, since their names weren't in any scope they could access, their names had to be passed as arguments (or, sure, stuffed in globals - but that would have been ugly ;-) ). ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-30 03:49, Tim Peters wrote: [Soni L.] That ain't shadow. That is dynamic scoping. I don't believe either term is technically accurate, but don't really care. Shadowing is something different: def f(): a = 42 def g(): print(a) local a: a = 43 g() g() should print "42" both times, *if it's lexically scoped*. Why? The `local` statement, despite its name, and casual talk about it, isn't _intended_ to create a new scope in any technically accurate sense. I think what it is intended to do has been adequately explained several times already. The `a` in `a = 42` is intended to be exactly the same as the `a` in `a = 43`, changing nothing at all about Python's lexical scoping rules. It is _the_ `a` local to `f`. If lexical scoping hasn't changed one whit (and it hasn't), the code _must_ print 43 first. Same as if `local a:` were replaced by `if True:`. `local` has no effect on a's value until the new "scope" _ends_, and never any effect at all on a's visibility (a's "scope"). "local" or not, after `a = 43` there is no scope anywhere, neither lexical nor dynamic, in which `a` is still bound to 42. _The_ value of `a` is 43 then, `local` or not. [snip] I think it should be lexically scoped. The purpose of 'local' would be to allow you to use a name that _might_ be used elsewhere. The problem with a dynamic scope is that you might call some global function from within the local scope, but find that it's "not working correctly" because you've inadvertently shadowed a name that the function refers to. Imagine, in a local scope, that you call a global function that calls 'len', but you've shadowed 'len'... ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sun, Apr 29, 2018 at 01:36:31PM +1000, Chris Angelico wrote: [...] > > While I started off with Python 1.5, I wasn't part of the discussions > > about nested scopes. But I'm astonished that you say that nested scopes > > were controversial. *Closures* I would completely believe, but mere > > lexical scoping? Astonishing. > > I'm not sure how you can distinguish them: Easily. Pascal, for example, had lexical scoping back in the 1970s, but no closures. I expect Algol probably did also, even earlier. So they are certainly distinct concepts. (And yes, Pascal functions were *not* first class values.) > What you expect here is lexical scope, yes. But if you have lexical > scope with no closures, the inner function can ONLY be used while its > calling function is still running. What would happen if you returned > 'inner' uncalled, and then called the result? How would it resolve the > name 'x'? Failing to resolve 'x' is an option. It would simply raise NameError, the same as any other name lookup that doesn't find the name. Without closures, we could say that names are looked up in the following scopes: # inner function, called from inside the creating function (1) Local to inner. (2) Local to outer (nonlocal). (3) Global (module). (4) Builtins. If you returned the inner function and called it from the outside of the factory function which created it, we could use the exact same name resolution order, except that (2) the nonlocals would be either absent or empty. Obviously that would limit the usefulness of factory functions, but since Python 1.5 didn't have closures anyway, that would have been no worse that what we had. Whether you have a strict Local-Global-Builtins scoping, or lexical scoping without closures, the effect *outside* of the factory function is the same. But at least with the lexical scoping option, inner functions can call each other while still *inside* the factory. (Another alternative would be dynamic scoping, where nonlocals becomes the environment of the caller.) > I can't even begin to imagine what lexical scope would do in > the absence of closures. At least, not with first-class functions. What they would likely do is raise NameError, of course :-) An inner function that didn't rely on its surrounding nonlocal scope wouldn't be affected. Or if you had globals that happened to match the names it was relying on, the function could still work. (Whether it would work as you expected is another question.) I expect that given the lack of closures, the best approach is to simply make sure that any attempt to refer to a nonlocal from the surrounding function outside of that function would raise NameError. All in all, closures are much better :-) -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Tim, on differences among Scheme-ish `let`, `let*`, `letrec` binding] > ... > > You can play, if you like, with trying to define the `iseven` lambda > here in one line by nesting lambdas to define `even` and `odd` as > default arguments: > > even = (lambda n: n == 0 or odd(n-1)) > odd = (lambda n: False if n == 0 else even(n-1)) > iseven = lambda n: even(n) > > Scheme supplies `letrec` for when "mutually recursive" bindings are > needed. In Python that distinction isn't nearly as evidently needed, > because Python's idea of closures doesn't capture all the bindings > currently in effect,. For example, when `odd` above is defined, > Python has no idea at all what the then-current binding for `even` is > - it doesn't even look for "even" until the lambda is _executed_. Just FYI, I still haven't managed to do it as 1-liner (well, one statement). I expected the following would work, but it doesn't :-) iseven = lambda n: ( lambda n=n, \ even = (lambda n: n == 0 or odd(n-1)), \ odd = (lambda n: False if n == 0 else even(n-1)): even(n))() Ugly and obscure, but why not? In the inner lambda, `n`, `even`, and `odd` are all defined in its namespace, so why does it fail anyway? >>> iseven(6) Traceback (most recent call last): ... iseven(6) ... even(n))() ... even = (lambda n: n == 0 or odd(n-1)), \ NameError: name 'odd' is not defined Because while Python indeed doesn't capture the current binding for `odd` when the `even` lambda is compiled, it _does_ recognize that the name `odd` is not local to the lambda at compile-time, so generates a LOAD_GLOBAL opcode to retrieve `odd`'s binding at runtime. But there is no global `odd` (well, unless there is - and then there's no guessing what the code would do). For `even` to know at compile-time that `odd` will show up later in its enclosing lambda's arglist requires that Python do `letrec`-style binding instead. For a start ;-) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sat, Apr 28, 2018 at 11:20:52PM -0500, Tim Peters wrote: > [Tim] > >> Enormously harder to implement than binding expressions, and the > >> latter (to my eyes) capture many high-value use cases "good enough". > > [Steven D'Aprano] > > And yet you're suggesting an alternative which is harder and more > > confusing. > > I am? I said at the start that it was a "brain dump". It was meant > to be a point of discussion for anyone interested. I also said I was > more interested in real use cases from real code than in debating, and > I wasn't lying about that ;-) Ah, my mistake... I thought you were advocating sublocal scopes as well as just brain dumping the idea. [...] > > Even when I started, as a novice programmer who wouldn't have recognised > > the term "lexical scoping" if it fell on my head from a great height, I > > thought it was strange that inner functions couldn't see their > > surrounding function's variables. Nested scopes just seemed intuitively > > obvious: if a function sees the variables in the module surrounding it, > > then it should also see the variables in any function surrounding it. > > > > This behaviour in Python 1.5 made functions MUCH less useful: [...] > > I think it is fair to say that inner functions in Python 1.5 were > > crippled to the point of uselessness. > > I don't think that's fair to say. A great many functions are in fact > ... functions ;-) That is, they compute a result from the arguments > passed to them. Sure, but not having access to the surrounding function scope means that inner functions couldn't call other inner functions. Given: def outer(): def f(): ... def g(): ... f cannot call g, or vice versa. I think it was a noble experiment in how minimal you could make scoping rules and still be usable, but I don't think that particular aspect was a success. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
I really liked the syntax that mimicked lambda even if I find it verbose : a = local x=1, y=2: x + y + 3 Even if I still prefer the postfix syntax : a = x + 3 where x = 2 About scheme "let" vs "let*", the paralel in Python is : a, b, c = 5, a+1, 2 # let syntax a = 5; b = a+1; c = 2 # let* syntax Which makes be wonder, we could use the semicolon in the syntax ? a = local x = 1; y = x+1: x + y + 3 Or with the postfix syntax : a = x + y + 3 where x = 1; y = x+1 Chaining where would be is syntax error : a = x + y + 3 where x = 1 where y = x+1 Parenthesis could be mandatory if one wants to use tuple assignment : a = local (x, y) = 1, 2: x + y + 3 When I see that, I really want to call it "def" a = def (x, y) = 1, 2: x + y + 3 a = def x = 1; y = 2: x + y + 3 Which is read define x = 1 then y = 2 in x + y + 3 Using def would be obvious this is not a function call. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Soni L.] > That ain't shadow. That is dynamic scoping. I don't believe either term is technically accurate, but don't really care. > Shadowing is something different: > > def f(): > a = 42 > def g(): > print(a) > local a: > a = 43 > g() > g() > > should print "42" both times, *if it's lexically scoped*. Why? The `local` statement, despite its name, and casual talk about it, isn't _intended_ to create a new scope in any technically accurate sense. I think what it is intended to do has been adequately explained several times already. The `a` in `a = 42` is intended to be exactly the same as the `a` in `a = 43`, changing nothing at all about Python's lexical scoping rules. It is _the_ `a` local to `f`. If lexical scoping hasn't changed one whit (and it hasn't), the code _must_ print 43 first. Same as if `local a:` were replaced by `if True:`. `local` has no effect on a's value until the new "scope" _ends_, and never any effect at all on a's visibility (a's "scope"). "local" or not, after `a = 43` there is no scope anywhere, neither lexical nor dynamic, in which `a` is still bound to 42. _The_ value of `a` is 43 then, `local` or not. The _only_ twist is that `local` wants to save/restore `a`'s binding before/after the suite of code it controls. That's not really about "scope" at all with its technical meaning. I don't much care if people casually _think_ about it as being "about.scope", though. Yes, the effect is similar to what you might see in a language with dynamic scoping _if_ it pushed a _copy_ of a's current binding on an evaluation stack at the start, and popped that (possibly mutated) copy later, but actual dynamic binding doesn't push copies, and Python isn't using any sort of dynamic evaluation stack regardless. That both restore previous bindings at times is a shallow coincidence. Neither is it really shadowing, which is lexical hiding of names. Neither is an accurate model for what it actually does, but, yes, it bears more _resemblance_ to dynamic scoping if you ignore that it's not dynamic scoping ;-) > If it's lexically scoped, this is just adding another scope: blocks. > (instead of the smallest possible scope being function scope) I expect that thinking about "scope" at all just confuses people here, unless they don't think too much about it ;-) Nothing about Python's scoping rules changes one whit. Exactly the same in the above could be achieved by replacing the `local a:` construct above by, e.g,, __save_a_with_a_unique_name = a a = 43 g() a = __save_a_with_a_unique_name Indeed, that's one way the compiler could _implement_ it. Nothing about a's scope is altered; it's merely a block-structured save-value/restore-value gimmick. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sun, Apr 29, 2018 at 9:28 PM, Tim Peterswrote: > [David Mertz ] > > Ooops. My proof [of] anti-concept has a flaw. It only "shadows" names > that > > already exist. Presumably that's the wrong idea, but it's easy enough to > > change if desired. > > with sublocal(a=6): > g("first in block") > a = 5 > g("set a to 5") > b = 19 > g("set b to 19") > g("after") > > Worm around that too, then going back to the example at the top, if > the manager's > > locals().update(_locals) > > had the intended effect, it would end up restoring `b` to 2 too, yes? > The only names that "should be" restored are the names in the `kws` > dict. > Actually, that wasn't my intention. As I imagined the semantics, I wanted a context manager that restored the "outside" context for anything defined "inside" the context. Allowing keyword arguments was just an extra "convenience" that was meant to be equivalent to defining/overwriting variables inside the body. So these would be equivalent: ## 1 with sublocal(): a = 1 b = 2 x = a + b # a, b now have their old values again ## 2 with sublocal(a=1, b=2): x = a + b # a, b now have their old values again ## 3 with sublocal(a=1): b = 2 x = a + b # a, b now have their old values again I knew I was ignoring nonlocals and nested function scopes. But just trying something really simple that looks like the un-proposal in some cases. Maybe there's no way in pure-Python to deal with the edge cases though. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[David Mertz] > Ooops. My proof on anti-concept has a flaw. It only "shadows" names that > already exist. Presumably that's the wrong idea, but it's easy enough to > change if desired. Even in the very early days when Python's runtime was more relentlessly simple-minded than it is now, these kinds of things were always hard to get right in all cases. But it's heartening to see _someone_ still has courage enough to leap into the void ;-) I see that the docs for `locals()` now say: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter. I thought that explained something I was seeing, but the problem turned out to be more obvious: the "locals()" inside your "sublocal()" context manager refer to _sublocal_'s own locals, not to the locals of the code invoking `sublocal()`.. So, e.g., run this: def run(): a = 1 b = 2 def g(tag): print(f"{tag}: a {a} b {b}") with sublocal(a=6): g("first in block") a = 5 g("set a to 5") b = 19 g("set b to 19") g("after") Here's output: first in block: a 1 b 2 # the `a=6` had no visible effect set a to 5: a 5 b 2 # golden set b to 19: a 5 b 19 # also golden after: a 5 b 19 # `a` wasn't restored to 1 To be very clear, the output is the same as if the `with` statement were replaced with `if True:`. But even if we crawled up the call stack to get the right locals() dict, looks like `exec` is no longer enough (in Python 3) to badger the runtime into making it work anyway: https://bugs.python.org/issue4831 """ > Specifically, what is the approved way to have exec() modify the local > environment of a function? There is none. To modify the locals of a function on the fly is not possible without several consequences: normally, function locals are not stored in a dictionary, but an array, whose indices are determined at compile time from the known locales. This collides at least with new locals added by exec. The old exec statement circumvented this, because the compiler knew that if an exec without globals/locals args occurred in a function, that namespace would be "unoptimized", i.e. not using the locals array. Since exec() is now a normal function, the compiler does not know what "exec" may be bound to, and therefore can not treat is specially. """ Worm around that (offhand I don't know how), and there are nonlocal names too. I don't know whether Python's current introspection features are enough to even find out which nonlocals have been declared, let alone to _which_ scope each nonlocal belongs. Worm around that too, then going back to the example at the top, if the manager's locals().update(_locals) had the intended effect, it would end up restoring `b` to 2 too, yes? The only names that "should be" restored are the names in the `kws` dict. So, in all, this may be a case where it's easier to implement in the compiler than to get working at runtime via ever-more-tortured Python code. And when that's all fixed, "a" can appear in both locals() and globals() (not to mention also in enclosing scopes), in which case what to do is unclear regardless how this is implemented. Which "a"(s) did the user _intend_ to shadow? The fun never ends ;-) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 04/29/2018 01:20 PM, Tim Peters wrote: So, e.g., """ a = 42 def showa(): print(a) def run(): global a local a: # assuming this existed a = 43 showa() showa() """ would print 43 and then 42. Which makes "local a:" sound senseless on the face of it ;-) "shadow" would be a more descriptive name for what it actually does. Yeah, "shadow" would be a better name than "local", considering that it effectively temporarily changes what other functions see as global. Talk about a debugging nightmare! ;) -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
Ooops. My proof on anti-concept has a flaw. It only "shadows" names that already exist. Presumably that's the wrong idea, but it's easy enough to change if desired. On Sun, Apr 29, 2018 at 5:24 PM, Paul Moorewrote: > On 29 April 2018 at 21:20, Tim Peters wrote: > > As covered most recently in an exchange with Tim Delaney, best I can > > tell absolutely nobody has wanted that. By "sublocal scope" they > > don't mean a full-fledged new scope at all, but a kind of limited > > "shadowing" of a handful of specific, explicitly given names. It acts > > like a context manager, if there were a way to clearly spell > > > > save the current state of these specific identifiers at the start (& > I > > couldn't care less whether they're local, nonlocal, or global - I > > don't know & don't care) > > > > then execute the code exactly as if this gimmick had never been used > > > > then, at the end, restore the specific identifier states we saved > > at the start > > So maybe adding such a primitive (maybe something live states = > sys.get_variable_state('a', 'b', 'c') and > sys.set_variable_state(states)) would be useful? Of course, we've > moved away from real use cases and back to theoretical arguments now, > so it's entirely possible that doing so would only solve problems that > no-one actually has... David Mertz' sublocal context manager would be > a good prototype of such a thing - at least good enough to demonstrate > that it's of no benefit in practice > > Paul > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 29 April 2018 at 21:20, Tim Peterswrote: > As covered most recently in an exchange with Tim Delaney, best I can > tell absolutely nobody has wanted that. By "sublocal scope" they > don't mean a full-fledged new scope at all, but a kind of limited > "shadowing" of a handful of specific, explicitly given names. It acts > like a context manager, if there were a way to clearly spell > > save the current state of these specific identifiers at the start (& I > couldn't care less whether they're local, nonlocal, or global - I > don't know & don't care) > > then execute the code exactly as if this gimmick had never been used > > then, at the end, restore the specific identifier states we saved > at the start So maybe adding such a primitive (maybe something live states = sys.get_variable_state('a', 'b', 'c') and sys.set_variable_state(states)) would be useful? Of course, we've moved away from real use cases and back to theoretical arguments now, so it's entirely possible that doing so would only solve problems that no-one actually has... David Mertz' sublocal context manager would be a good prototype of such a thing - at least good enough to demonstrate that it's of no benefit in practice Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Mon, Apr 30, 2018 at 6:45 AM, Tim Peterswrote: > [Chris Angelico ] >> So maybe the effective semantics should be: >> >> >>> (lambda a=3: (lambda b=a+1: (a, b))())() >> (3, 4) > > Almost, but by that point the idea that this is already "easily > spelled" via lambdas has become ludicrously difficult to argue with a > straight face ;-) Oh, I dunno, I've seen people argue that PHP is the right choice of language :-) ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Tim] Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the end: c = local(a=3, b=4, a*b) [Nikolaus Rath] >>> I think this can be done already with slighly different syntax: >>> >>> c = (lambda a=3, b=4: a*b)() >>> >>> The trailing () is a little ugly, but the semantics are much more >>> obvious. [Tim] >> But also broken, in a way that can't be sanely fixed. Covered before >> in other messages. Short course: >> >> >>> a = 10 >> >>> b = 20 >> >>> (lambda a=3, b=a+1: (a, b))() >> (3, 11) >> >> This context really demands (3, 4) instead. In Scheme terms, Python's >> lambda default arguments do "let" binding ("all at once"), but "let*" >> binding is what's needed ("one at a time, left to right, with bindings >> already done visible to later bindings"). [Chris Angelico ] > So maybe the effective semantics should be: > > >>> (lambda a=3: (lambda b=a+1: (a, b))())() > (3, 4) Almost, but by that point the idea that this is already "easily spelled" via lambdas has become ludicrously difficult to argue with a straight face ;-) By "almost", I mean there are other cases where even nesting Python lambdas doesn't capture the intent. In these cases, not only does the expression defining b refer to a, but _also_ the expression defining a refers to b. You can play, if you like, with trying to define the `iseven` lambda here in one line by nesting lambdas to define `even` and `odd` as default arguments: even = (lambda n: n == 0 or odd(n-1)) odd = (lambda n: False if n == 0 else even(n-1)) iseven = lambda n: even(n) Scheme supplies `letrec` for when "mutually recursive" bindings are needed. In Python that distinction isn't nearly as evidently needed, because Python's idea of closures doesn't capture all the bindings currently in effect,. For example, when `odd` above is defined, Python has no idea at all what the then-current binding for `even` is - it doesn't even look for "even" until the lambda is _executed_. But, to be fair, I'm not sure: iseven =- local( even = (lambda n: n == 0 or odd(n-1)), odd = (lambda n: False if n == 0 else even(n-1)). lambda n: even(n)) would have worked either. At the moment I'm certain it wouldn't. Last night I was pretty sure it would ;-) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
This doesn't address the fact no one actually needs it. But if we WANTED a sublocal() context manager, we could spell it something like this: In [42]: @contextmanager ...: def sublocal(**kws): ...: _locals = locals().copy() ...: _globals = globals().copy() ...: for k, v in kws.items(): ...: if k in locals(): ...: exec(f"locals()['{k}'] = {v}") ...: elif k in globals(): ...: exec(f"globals()['{k}'] = {v}") ...: yield ...: locals().update(_locals) ...: globals().update(_globals) ...: In [43]: a = 42 In [44]: with sublocal(a=43): ...: showa() ...: 43 In [45]: showa() 42 In [46]: with sublocal(): ...: a = 41 ...: showa() ...: 41 In [47]: showa() 42 On Sun, Apr 29, 2018 at 4:20 PM, Tim Peterswrote: > [Ethan Furman ] > > If we need a sublocal scope, I think the most Pythonic* route to have it > > would be: > > > > with sublocal(): > > blah blah > > > > As covered most recently in an exchange with Tim Delaney, best I can > tell absolutely nobody has wanted that. By "sublocal scope" they > don't mean a full-fledged new scope at all, but a kind of limited > "shadowing" of a handful of specific, explicitly given names. It acts > like a context manager, if there were a way to clearly spell > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sun, Apr 29, 2018 at 7:22 PM, Mikhail Vwrote: > On Sun, Apr 29, 2018 at 3:30 AM, Tim Peters wrote: > >> Time to note another subtlety: people don't _really_ want "a new >> scope" in Python. If they did, then _every_ name appearing in a > If there is demand for this, how about just introducing a > derived syntax for the "auto-called" def block, say, just "def" without a > name: > > def : > global a > x = 1; y = 2 > a = x + y > print (a) > Or even better, it would be better to avoid overloading "global" or "return", and use dedicated prefix for variables that are pushed to outer scope. I think it would look way better for cases with multiple variables: def func(): state = 0 def: localstate1 = state + 1 localstate2 = state + 2 & localstate1 & localstate2 print (localstate1) print (localstate2) Prefix in assignment would allow more expressive dispatching: def func(): state = 0 def: localstate1 = state + 1 localstate2 = state + 2 & M = state + 3 & L1, & L2 = localstate1, localstate2 print (L1, L2, M) Imo such syntax is closest do "def" block and should be so, because functions have very strong association with new scope definition, and same rules should work here. Not only it is more readable than "with local()", but also makes it easier to edit, comment/uncomment lines. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Ethan Furman] > If we need a sublocal scope, I think the most Pythonic* route to have it > would be: > > with sublocal(): > blah blah > > which would act just like local/global does now: > > - any assignment creates a new variable > - unless that variable has been declared global/nonlocal > - plain reads (no assignment ever happens) refer to > nonlocal/global/built-in names > ... As covered most recently in an exchange with Tim Delaney, best I can tell absolutely nobody has wanted that. By "sublocal scope" they don't mean a full-fledged new scope at all, but a kind of limited "shadowing" of a handful of specific, explicitly given names. It acts like a context manager, if there were a way to clearly spell save the current state of these specific identifiers at the start (& I couldn't care less whether they're local, nonlocal, or global - I don't know & don't care) then execute the code exactly as if this gimmick had never been used then, at the end, restore the specific identifier states we saved at the start It's the same kind of shadowing Python already does by magic for, e.g., `i`, in [i for i in range(3)] So, e.g., """ a = 42 def showa(): print(a) def run(): global a local a: # assuming this existed a = 43 showa() showa() """ would print 43 and then 42. Which makes "local a:" sound senseless on the face of it ;-) "shadow" would be a more descriptive name for what it actually does. > ... ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Mon, Apr 30, 2018 at 5:55 AM, Tim Peterswrote: > [Tim] >>> Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the >>> end: >>> >>> c = local(a=3, b=4, a*b) > > [Nikolaus Rath ] >> I think this can be done already with slighly different syntax: >> >> c = (lambda a=3, b=4: a*b)() >> >> The trailing () is a little ugly, but the semantics are much more >> obvious. > > But also broken, in a way that can't be sanely fixed. Covered before > in other messages. Short course: > a = 10 b = 20 (lambda a=3, b=a+1: (a, b))() > (3, 11) > > This context really demands (3, 4) instead. In Scheme terms, Python's > lambda default arguments do "let" binding ("all at once"), but "let*" > binding is what's needed ("one at a time, left to right, with bindings > already done visible to later bindings"). So maybe the effective semantics should be: >>> (lambda a=3: (lambda b=a+1: (a, b))())() (3, 4) ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Tim] >> Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the >> end: >> >> c = local(a=3, b=4, a*b) [Nikolaus Rath] > I think this can be done already with slighly different syntax: > > c = (lambda a=3, b=4: a*b)() > > The trailing () is a little ugly, but the semantics are much more > obvious. But also broken, in a way that can't be sanely fixed. Covered before in other messages. Short course: >>> a = 10 >>> b = 20 >>> (lambda a=3, b=a+1: (a, b))() (3, 11) This context really demands (3, 4) instead. In Scheme terms, Python's lambda default arguments do "let" binding ("all at once"), but "let*" binding is what's needed ("one at a time, left to right, with bindings already done visible to later bindings"). Of course in Scheme you explicitly type either "let" or "let*" (or "letrec", or ...) depending on what you want at the time, but "let*" is overwhelmingly what's wanted when it makes a difference (in the example at the top, it makes no difference at all). Otherwise you can't build up a complex result from little named pieces that may depend on pieces already defined. See,.e.g, the quadratic equation example in the original post, where this was implicit. > ... ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 04/28/2018 10:16 AM, Tim Peters wrote: ... but do realize that since PEP 572 dropped any notion of sublocal scopes, that recurring issue remains wholly unaddressed regardless. If we need a sublocal scope, I think the most Pythonic* route to have it would be: with sublocal(): blah blah which would act just like local/global does now: - any assignment creates a new variable - unless that variable has been declared global/nonlocal - plain reads (no assignment ever happens) refer to nonlocal/global/built-in names This has the advantages of: - no confusion about which variables are sublocal (acts like a new function scope) - no extra parens, assignments, expressions on "with sublocal():" line Possible enhancements: - give sublocal block a name "with sublocal() as blahblah:" and then reuse that block again later ("with blahblah:") or maybe pass it to other functions... Of course, as previously stated, this is orthogonal to PEP 572. -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-29 04:17 PM, Nikolaus Rath wrote: On Apr 27 2018, Tim Peterswrote: Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the end: c = local(a=3, b=4, a*b) I think this can be done already with slighly different syntax: c = (lambda a=3, b=4: a*b)() The trailing () is a little ugly, but the semantics are much more obvious. So maybe go with a variation that makes function evaluation implicit? c = lambda! a=3, b=4: a*b (reads terrible, but maybe someone has a better idea). if local(m = re.match(regexp, line)): print(m.group(0)) Of course, that wouldn't (and shouldn't) work anymore. But that's a good thing, IMO :-). Best, -Nikolaus Has anyone heard of Lua? Lexical scoping? Block scope? (Python doesn't have blocks and that sucks?) Etc? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Apr 27 2018, Tim Peterswrote: > Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the end: > > c = local(a=3, b=4, a*b) I think this can be done already with slighly different syntax: c = (lambda a=3, b=4: a*b)() The trailing () is a little ugly, but the semantics are much more obvious. So maybe go with a variation that makes function evaluation implicit? c = lambda! a=3, b=4: a*b (reads terrible, but maybe someone has a better idea). > if local(m = re.match(regexp, line)): > print(m.group(0)) Of course, that wouldn't (and shouldn't) work anymore. But that's a good thing, IMO :-). Best, -Nikolaus -- GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-29 18:01, Tim Peters wrote: [Tim] >> ... >> This is the kind of code about which there have been background >> complaints "forever": >> >> m1 = regexp1.match(line) >> m2 = regexp2.match(iine) >> if m1 and m2: >> do all sorts of stuff with m1 and/or m2, >> including perhaps modifying local variables >> and/or global variables >> and/or nonlocal variables >> >> The complaints are of two distinct kinds: >> >> 1. "I want to compute m1 and m2 _in_ the `if` test". >> >> 2. "I don't want these temp names (m1 and m2) accidentally >> conflicting with local names already in scope - if these names >> already exist, I want the temp names to shadow their >> current bindings until the `if` structure is done". >> >> So, >> >> if local(m1=regexp1.match(line), >>m2 = regexp2.match(iine), >>m1 and m2): >> >> intends to address both complaints via means embarrassingly obvious to >> the most casual observer ;-) [MRAB] > How about these: > > local m1, m2: > m1 = regexp1.match(line) > m2 = regexp2.match(line): > if m1 and m2: > ... > > > local m1, m2: > > if (m1 := regexp1.match(line)) and (m2 := regexp2.match(line)): > > ... > > local m1=regexp1.match(line), m2=regexp2.match(line): > if m1 and m2: They address complaint #2 in what seems to me a thoroughly Pythonic (direct, transparent, no more magical than necessary, easy to read) way. They don't address complaint #1 at all, but as you've shown (in the 2nd spelling) that isn't _inherently_ tied to complaint #2 (complaint #1 is what PEP 572 addresses). So _if_ PEP 572 is accepted, adding this form of a compound `local` statement too would address both of the listed complaints, at the "cost" of a bit more typing and adding a level of indentation. Neither of which bother me ;-) `local()` itself was also intended to address the even-more-in-the-background recurring desires for an expression (as opposed to statement) oriented way to use throwaway bindings; e.g., instead of temp = x + y - z + 1 r = temp**2 - 1/temp this instead: r = local(t=x + y - z + 1, t**2 - 1/t) It's not an accident that the shorter `t` is used in the latter than the former's `temp`: when people are wary of clobbering names by accident, they tend to use longer names that say "I'm just a temp - please don't _expect_ my binding to persist beyond the immediate uses on the next few lines":. Anyway, that kind of thing is common n functional languages, where "let" pile-of-bindings "in" expression kinds of constructs are widely used _as_ (sub)expressions themselves. local t = x + y - z + 1: r = t**2 - 1/t would be the same semantically, but they'd still complain about the "extra" typing and the visual "heaviness" of introducing a block for what they _think_ of as being "just another kind of expression". The `local()` I brought up was, I think, far too biased _toward_ that use. It didn't "play nice" with block-oriented uses short of excruciatingly deep magic. Your `local` statement is biased in the other direction, but that's a Good Thing :-) As well as: local t = x + y - z + 1: r = t**2 - 1/t I wonder if it could be rewritten as: r = local t = x + y - z + 1: t**2 - 1/t Would parentheses be needed? r = (local t = x + y - z + 1: t**2 - 1/t) It kind of resembles the use of default parameters with lambda! The names would be local to the suite if used as a statement or the following expression if used in an expression, either way, the bit after the colon. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
2018-04-29 17:52 GMT+03:00 MRAB: > > >> How about these: > > local m1, m2: > m1 = regexp1.match(line) > m2 = regexp2.match(line): > if m1 and m2: > ... > Is it possible to do the same thing, but with the help of `with` statement: with local('m1', 'm2'): m1 = regex1.match(line) m2 = regex2.match(line) if m1 and m2: ... With kind regards, -gdg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 04/27/2018 07:37 PM, Tim Peters wrote: Idea: introduce a "local" pseudo-function to capture the idea of initialized names with limited scope. Note: the thing I'm most interested in isn't debates, but in whether this would be of real use in real code. I keep going back and forth on the ":=" syntax as on the one hand I find the functionality very useful but on the other hand it's ugly and doesn't really read well. However, I can say I am solidly -1 on local, let, etc.: - local() vs locals(): very similar words with disparate meanings - more parens -- ugh - sublocal: one more scope for extra complexity -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Tim] >> ... >> This is the kind of code about which there have been background >> complaints "forever": >> >> m1 = regexp1.match(line) >> m2 = regexp2.match(iine) >> if m1 and m2: >> do all sorts of stuff with m1 and/or m2, >> including perhaps modifying local variables >> and/or global variables >> and/or nonlocal variables >> >> The complaints are of two distinct kinds: >> >> 1. "I want to compute m1 and m2 _in_ the `if` test". >> >> 2. "I don't want these temp names (m1 and m2) accidentally >> conflicting with local names already in scope - if these names >> already exist, I want the temp names to shadow their >> current bindings until the `if` structure is done". >> >> So, >> >> if local(m1=regexp1.match(line), >>m2 = regexp2.match(iine), >>m1 and m2): >> >> intends to address both complaints via means embarrassingly obvious to >> the most casual observer ;-) [MRAB] > How about these: > > local m1, m2: > m1 = regexp1.match(line) > m2 = regexp2.match(line): > if m1 and m2: > ... > > > local m1, m2: > > if (m1 := regexp1.match(line)) and (m2 := regexp2.match(line)): > > ... > > local m1=regexp1.match(line), m2=regexp2.match(line): > if m1 and m2: They address complaint #2 in what seems to me a thoroughly Pythonic (direct, transparent, no more magical than necessary, easy to read) way. They don't address complaint #1 at all, but as you've shown (in the 2nd spelling) that isn't _inherently_ tied to complaint #2 (complaint #1 is what PEP 572 addresses). So _if_ PEP 572 is accepted, adding this form of a compound `local` statement too would address both of the listed complaints, at the "cost" of a bit more typing and adding a level of indentation. Neither of which bother me ;-) `local()` itself was also intended to address the even-more-in-the-background recurring desires for an expression (as opposed to statement) oriented way to use throwaway bindings; e.g., instead of temp = x + y - z + 1 r = temp**2 - 1/temp this instead: r = local(t=x + y - z + 1, t**2 - 1/t) It's not an accident that the shorter `t` is used in the latter than the former's `temp`: when people are wary of clobbering names by accident, they tend to use longer names that say "I'm just a temp - please don't _expect_ my binding to persist beyond the immediate uses on the next few lines":. Anyway, that kind of thing is common n functional languages, where "let" pile-of-bindings "in" expression kinds of constructs are widely used _as_ (sub)expressions themselves. local t = x + y - z + 1: r = t**2 - 1/t would be the same semantically, but they'd still complain about the "extra" typing and the visual "heaviness" of introducing a block for what they _think_ of as being "just another kind of expression". The `local()` I brought up was, I think, far too biased _toward_ that use. It didn't "play nice" with block-oriented uses short of excruciatingly deep magic. Your `local` statement is biased in the other direction, but that's a Good Thing :-) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sun, Apr 29, 2018 at 3:30 AM, Tim Peterswrote: > > """ > Time to note another subtlety: people don't _really_ want "a new > scope" in Python. If they did, then _every_ name appearing in a > binding context (assignment statement target, `for` target, ...) for > the duration would vanish when the new scope ended. What they really > want is a new scope with an implied "nonlocal" declaration for every > name appearing in a binding context _except_ for the specific names > they're effectively trying to declare as being "sublocal" instead. > """ > > If by "new name" you mean that `x` didn't appear in any earlier line, > then Python's current analysis would classify `x` as local to the > current function (or as global if this is module-level code ...). > That wouldn't change. I have hard time understanding what is the demand here actually. (it's been too many posts and ideas to absorb) >From your description of "what people need" - how is this different from current "def"? Now I can use nested "def"s and call it right away: def d(): global a x = 1; y = 2 a = x + y d() print (a) And this will do the thing: here if the "a" variable is "new" then it will be initialized and pushed to the outer scope, right? If there is demand for this, how about just introducing a derived syntax for the "auto-called" def block, say, just "def" without a name: def : global a x = 1; y = 2 a = x + y print (a) Which would act just like a scope block without any new rules introduced. And for inline usage directly inside single-line expression - I don't think it is plausible to come up with very nice syntax anyway, and I bet at best you'll end up with something looking C-ish, e.g.: if (def {x=1; y=2; = x + y } ) : ... As a short-cut for the above multi-line scope block. Mikhail ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-29 07:57, Tim Peters wrote: [Tim Delaney] My big concern here involves the: if local(m = re.match(regexp, line)): print(m.group(0)) example. The entire block needs to be implicitly local for that to work - what happens if I assign a new name in that block? [Tim Peters] I really don't know what you're asking there. Can you make it concrete? If, e.g., you're asking what happens if this appeared after the `print`: x = 3.14 then the answer is "the same as what would happen if `local` had not been used". We can't know what that is without context, though. Maybe x is global. Maybe x was declared nonlocal earlier. Maybe it's function-local. ... [Tim D] That's exactly what I was asking, and as I understand what you're saying, we would have a local name m available in the indented block which went away when the block ended, but any names modified in the block are not local to the block. That seems likely to be a source of errors. If you what you _want_ is a genuinely new scope, yes. But no actual use cases so far wanted that at all. This is the kind of code about which there have been background complaints "forever": m1 = regexp1.match(line) m2 = regexp2.match(iine) if m1 and m2: do all sorts of stuff with m1 and/or m2, including perhaps modifying local variables and/or global variables and/or nonlocal variables The complaints are of two distinct kinds: 1. "I want to compute m1 and m2 _in_ the `if` test". 2. "I don't want these temp names (m1 and m2) accidentally conflicting with local names already in scope - if these names already exist, I want the temp names to shadow their current bindings until the `if` structure is done". So, if local(m1=regexp1.match(line), m2 = regexp2.match(iine), m1 and m2): intends to address both complaints via means embarrassingly obvious to the most casual observer ;-) How about these: local m1, m2: m1 = regexp1.match(line) m2 = regexp2.match(line): if m1 and m2: ... local m1, m2: if (m1 := regexp1.match(line)) and (m2 := regexp2.match(line)): ... local m1=regexp1.match(line), m2=regexp2.match(line): if m1 and m2: ... ? [snip] ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Tim Delaney] >>> My big concern here involves the: >>> >>> if local(m = re.match(regexp, line)): >>> print(m.group(0)) >>> >>> example. The entire block needs to be implicitly local for that to work >>> - >>> what happens if I assign a new name in that block? [Tim Peters] >> I really don't know what you're asking there. Can you make it >> concrete? If, e.g., you're asking what happens if this appeared after >> the `print`: >> >> x = 3.14 >> >> then the answer is "the same as what would happen if `local` had not >> been used". We can't know what that is without context, though. >> Maybe x is global. Maybe x was declared nonlocal earlier. Maybe it's >> function-local. ... [Tim D] > That's exactly what I was asking, and as I understand what you're saying, we > would have a local name m available in the indented block which went away > when the block ended, but any names modified in the block are not local to > the block. That seems likely to be a source of errors. If you what you _want_ is a genuinely new scope, yes. But no actual use cases so far wanted that at all. This is the kind of code about which there have been background complaints "forever": m1 = regexp1.match(line) m2 = regexp2.match(iine) if m1 and m2: do all sorts of stuff with m1 and/or m2, including perhaps modifying local variables and/or global variables and/or nonlocal variables The complaints are of two distinct kinds: 1. "I want to compute m1 and m2 _in_ the `if` test". 2. "I don't want these temp names (m1 and m2) accidentally conflicting with local names already in scope - if these names already exist, I want the temp names to shadow their current bindings until the `if` structure is done". So, if local(m1=regexp1.match(line), m2 = regexp2.match(iine), m1 and m2): intends to address both complaints via means embarrassingly obvious to the most casual observer ;-) This is, e.g., the same kind of name-specific "shadowing" magically done by list and dict comprehensions now, and by generator expressions. For example, [i**2 for i in range(10)] has no effect on whatever `i` meant before the listcomp was executed. > To clarify my understanding, if the names 'x' and 'm' did not exist prior to > the following code, what would x and m refer to after the block completed? > > if local(m = re.match(regexp, line)): > x = 1 > m = 2 I hope the explanation above made that clear. What's wanted is exactly what the current m = re.match(regexp, line): if m: x =1 m = 2 _would_ do if only there were a sane way to spell "save m's current status before that all started and restore it after that all ends". So they want `x == 1` after it's over, and `m` to raise NameError. >>> if local { m = re.match(regexp, line) }: >>> print(m.group(0)) >> OK, this is the only case in which you used it in an `if` or `while` >> expression. All the questions you asked of me at the start can be >> asked of this spelling too. >> You seemed to imply at the start that the >> right curly brace would always mark the end of the new scope. But if >> that's so, the `m` in `m.group()` has nothing to do with the `m` >> assigned to in the `local` block - _that_ scope ended before `print` >> was reached. > Yes - I think this is exactly the same issue as with your proposed syntax. Wholly agreed :-) >> So if you're not just trying to increase the level of complexity of >> what can appear in a local block, a fundamental problem still needs >> solving ;-) I suppose you could solve it like so: >> >> local { m = re.match(regexp, line) >>if m: >>print(m.group(0)) >> } >> >> but, besides losing the "shortcut", it would also mean something >> radically different if >> >>x = 3.14 >> >> appeared after the "print". Right? If a "local block" is taken >> seriously, then _all_ names bound inside it vanish when the block >> ends. > Indeed, and I don't have a proposal - just concerns it wold be very > difficult to explain and understand exactly what would happen in the case of > something like: > > if local(m = re.match(regexp, line)): > x = 1 > m = 2 Only names appearing as targets _in_ the `local(...)` are affected in any way. The states of those names are captured, then those names are bound to the values of the associated expressions in the `local(...)`, and when the scope of the `local` construct ends (which _is_ hard to explain!) those names' original states are restored. So the effects on names are actually pretty easy to explain: all and only the names appearing inside the `local(...)` are affected. > Regarding the syntax, I didn't want to really change your proposal, but just > thought the functionality was different enough from the function call it > appears to be that it probably merits different syntax. Probably so!
Re: [Python-ideas] A "local" pseudo-function
Tim] Peters wrote: >> The points to using function-call-like syntax were already covered >> ("nothing syntactically new to learn there", [Greg Ewing] [> The trouble is that one usually expects "nothing syntactically > new" to imply "nothing semantically new" as well, which is very > far from the case here. So I think that not using *any* new > syntax would actually be hurting users rather than helping them. I expect anyone who has programmed for over a year has proved beyond doubt that they can run a marathon even if forced to wear a backpack containing a ton of lead, with barbed wire wrapped around their feet ;-) They're indestructible. If they came from Perl, they'd even have a hearty laugh when they learned the syntax had utterly fooled them :-) > If you really want to leverage existing knowledge, I'd suggest > something based on lambda: > > let a = 3, b = 4: a + b > > This can be easily explained as a shorthand for > > (lambda a = 3, b = 4: a + b)() In that case, yes, but not in all. There's more than one kind of magic here, even outside of block constructs. See, e.g., the quadratic equation example in the original post local(D = b**2 - 4*a*c, sqrtD = math.sqrt(D), ... Try that with a lambda, and you get a NameError when computing sqrt(D) - or, worse, pick up an irrelevant value of D left over from earlier code. In Scheme terminology, what Python does with default args (keyword args too) is "let" (as if you evaluate all the expressions before doing any of the bindings), but what's wanted is "let*" (bindings are established left-to-right, one at a time, and each binding already established is visible in the expression part of each later binding). But even with `let*`, I'm not sure whether this would (or should, or shouldn't) work: local(even = (lambda n: n == 0 or odd(n-1)), odd = (lambda n: False if n == 0 else even(n-1)), odd(7)) Well, OK, I'm pretty sure it would work. But by design or by accident? ;-) > except, of course, for the magic needed to make it DWIM in an if > or while statement. I'm still pretty uncomfortable about that. That's because it's horrid. Honest, I'm not even convinced it's _worth_ "solving" , even if it didn't seem to require deep magic. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
Tim Peters wrote: (ABC had no lexical scoping either - nor, if I recall correctly, even textual nesting of its flavor of functions). If Python hadn't allowed textual nesting either, folks might have been content to leave it that way. But having textual nesting without lexical scoping was just weird and confusing! > A great many functions are in fact ... functions ;-) That is, they compute a result from the arguments passed to them. They don't need more than that, Yes, but they often make use of other functions to do that, and not being able to call other local functions in the same scope seemed like a perverse restriction. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sun, 29 Apr 2018 at 10:30, Tim Peterswrote: > [Tim Delaney ] > > My big concern here involves the: > > > > > if local(m = re.match(regexp, line)): > > print(m.group(0)) > > > > example. The entire block needs to be implicitly local for that to work - > > what happens if I assign a new name in that block? > > I really don't know what you're asking there. Can you make it > concrete? If, e.g., you're asking what happens if this appeared after > the `print`: > > x = 3.14 > > then the answer is "the same as what would happen if `local` had not > been used". We can't know what that is without context, though. > Maybe x is global. Maybe x was declared nonlocal earlier. Maybe it's > function-local. While it may be irrelevant to what you're asking, I > noted just before: > That's exactly what I was asking, and as I understand what you're saying, we would have a local name m available in the indented block which went away when the block ended, but any names modified in the block are not local to the block. That seems likely to be a source of errors. To clarify my understanding, if the names 'x' and 'm' did not exist prior to the following code, what would x and m refer to after the block completed? if local(m = re.match(regexp, line)): x = 1 m = 2 > >> if local(m = re.match(regexp, line)): > >> print(m.group(0)) > > > if local { m = re.match(regexp, line) }: > > print(m.group(0)) > > OK, this is the only case in which you used it in an `if` or `while` > expression. All the questions you asked of me at the start can be > asked of this spelling too. > > You seemed to imply at the start that the > right curly brace would always mark the end of the new scope. But if > that's so, the `m` in `m.group()` has nothing to do with the `m` > assigned to in the `local` block - _that_ scope ended before `print` > was reached. > Yes - I think this is exactly the same issue as with your proposed syntax. > So if you're not just trying to increase the level of complexity of > what can appear in a local block, a fundamental problem still needs > solving ;-) I suppose you could solve it like so: > > local { m = re.match(regexp, line) >if m: >print(m.group(0)) > } > > but, besides losing the "shortcut", it would also mean something > radically different if > >x = 3.14 > > appeared after the "print". Right? If a "local block" is taken > seriously, then _all_ names bound inside it vanish when the block > ends. Indeed, and I don't have a proposal - just concerns it wold be very difficult to explain and understand exactly what would happen in the case of something like: if local(m = re.match(regexp, line)): x = 1 m = 2 Regarding the syntax, I didn't want to really change your proposal, but just thought the functionality was different enough from the function call it appears to be that it probably merits different syntax. Tim Delaney ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
Tim Peters wrote: The points to using function-call-like syntax were already covered ("nothing syntactically new to learn there", The trouble is that one usually expects "nothing syntactically new" to imply "nothing semantically new" as well, which is very far from the case here. So I think that not using *any* new syntax would actually be hurting users rather than helping them. If you really want to leverage existing knowledge, I'd suggest something based on lambda: let a = 3, b = 4: a + b This can be easily explained as a shorthand for (lambda a = 3, b = 4: a + b)() except, of course, for the magic needed to make it DWIM in an if or while statement. I'm still pretty uncomfortable about that. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 28/04/2018 04:34, Yury Selivanov wrote: > Hi Tim, > > This is interesting. Even "as is" I prefer this to PEP 572. Below are some > comments and a slightly different idea inspired by yours (sorry!) > > On Fri, Apr 27, 2018 at 10:41 PM Tim Peterswrote: > [..] >> As an expression, it's > >> "local" "(" arguments ")" > >> - Because it "looks like" a function call, nobody will expect the targets >> of named arguments to be fancier than plain names. > [..] >> Everyone's favorite: > >> if local(m = re.match(regexp, line)): >> print(m.group(0)) > >> Here's where it's truly essential that the compiler know everything >> about "local", because in _that_ context it's required that the new >> scope extend through the end of the entire block construct (exactly > > It does look like a function call, although it has a slightly different > syntax. In regular calls we don't allow positional arguments to go after > keyword arguments. Hence the compiler/parser will have to know what > 'local(..)' is *regardless* of where it appears. > > If you don't want to make 'local' a new keyword, we would need to make the > compiler/parser to trace the "local()" name to check if it was imported or > is otherwise "local". This would add some extra complexity to already > complex code. Another problematic case is when one has a big file and > someone adds their own "def local()" function to it at some point, which > would break things. > > Therefore, "local" should probably be a keyword. Perhaps added to Python > with a corresponding "from __future__" import. > > The other way would be to depart from the function call syntax by dropping > the parens. (And maybe rename "local" to "let" ;)) In this case, the > syntax will become less like a function call but still distinct enough. We > will be able to unambiguously parse & compile it. The cherry on top is > that we can make it work even without a "__future__" import! > > When we implemented PEP 492 in Python 3.5 we did a little trick in > tokenizer to treat "async def" in a special way. Tokenizer would switch to > an "async" mode and yield ASYNC and AWAIT tokens instead of NAME tokens. > This resulted in async/await syntax available without a __future__ import, > while having full backwards compatibility. > > We can do a similar trick for "local" / "let" syntax, allowing the > following: > > "let" NAME "=" expr ("," NAME = expr)* ["," expr] > > * "if local(m = re.match(...), m):" becomes > "if let m = re.match(...), m:" > > * "c = local(a=3) * local(b=4)" becomes > "c = let a=3, b=4, a*b" or "c = (let a=3, b=4, a*b)" > > * for i in iterable: > if let i2=i*i, i2 % 18 == 0: >append i2 to the output list > > etc. > > Note that I don't propose this new "let" or "local" to return their last > assignment. That should be done explicitly (as in your "local(..)" idea): >`let a = 'spam', a`. Potentially we could reuse our function return > annotation syntax, changing the last example to `let a = "spam" -> a` but I > think it makes the whole thing to look unnecessarily complex. > > One obvious downside is that "=" would have a different precedence compared > to a regular assignment statement. But it already has a different precedent > in function calls, so maybe this isn't a big deal, considered that we'll > have a keyword before it. > > I think that "let" was discussed a couple of times recently, but it's > really hard to find a definitive reason of why it was rejected (or was it?) > in the ocean of emails about assignment expressions. > > Yury > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > If things were to go the keyword direction I personally think that a great deal of clarity could be achieved by borrowing somewhat from Pascal with either use as [, as ...]: # Local names only exist in this scope, # existing names are in scope unless overridden # new names introduced here go out to nesting scope OR (for those who prefer name first, a function like look, and slightly less typing): using(=[ ,=...]): # Local names only exist in this scope, # existing names are in scope unless overridden # new names introduced here go out to nesting scope Lastly how about: = using(=[ ,=...]) # in this case where might be better than using Presumably, in the latter cases, the using function would return a sub_local dictionary that would only exist to the end of the scope be that the current line or indented block which should minimise the required magic. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com
Re: [Python-ideas] A "local" pseudo-function
[Tim] >> Enormously harder to implement than binding expressions, and the >> latter (to my eyes) capture many high-value use cases "good enough". [Steven D'Aprano] > And yet you're suggesting an alternative which is harder and more > confusing. I am? I said at the start that it was a "brain dump". It was meant to be a point of discussion for anyone interested. I also said I was more interested in real use cases from real code than in debating, and I wasn't lying about that ;-) Since no real new use cases (let alone compelling ones) have turned up yet, I'm ready to drop it for now. > What's the motivation here for re-introducing sublocal > scopes, if they're hard to do and locals are "good enough"? That's why I wanted to see if there were significant unaddressed use cases. That _my_ particular itches would be scratched "good enough" if the PEP is accepted doesn't imply everyone's will be. And my particular itches will continue to annoy if the PEP is rejected. > That's not a rhetorical question: why have you suggested this sublocal > scoping idea? Putting an idea out for discussion isn't suggesting it be adopted. The list is named "python-ideas", not "python-advocacy-death-match" ;-) > PEP 572 stopped talking about sublocals back in revision 2 > or so, and as far as I can see, *not a single objection* since has > been that the variables weren't sublocal. Meh. Chris didn't seem all that thrilled about dropping them, and I saw a number of messages more-than-less supporting the idea _before_ they were dropped. When it became clear that the PEP didn't stand a chance _unless_ they were dropped, nobody was willing to die for it, because they weren't the PEP's _primary_ point. > For what it is worth, if we ever did introduce a sublocal scope, I > don't hate Nick's "given" block statement: > > https://www.python.org/dev/peps/pep-3150/ And Chris just tried introducing it again. That's in reference to the last sentence of your reply: I don't think that sublocal scopes is a recurring issue How many times does it have to come up before "recurring" applies? ;-) I've seen it come up many times over ... well, literally decades by now. > ... > While I started off with Python 1.5, I wasn't part of the discussions > about nested scopes. But I'm astonished that you say that nested scopes > were controversial. *Closures* I would completely believe, but mere > lexical scoping? Astonishing. But true. Guido agonized over it for a long time. Limiting to 3 scopes was a wholly deliberate design decision at the start, not just, .e.g, due to lack of time to implement lexical scoping at the start. And that shouldn't be surprising given Python's start as "somewhere between a scripting language and C", and the many influences carried over from Guido's time working on ABC's implementation team (ABC had no lexical scoping either - nor, if I recall correctly, even textual nesting of its flavor of functions). I'm glad he tried it! Everyone learned something from it. > Even when I started, as a novice programmer who wouldn't have recognised > the term "lexical scoping" if it fell on my head from a great height, I > thought it was strange that inner functions couldn't see their > surrounding function's variables. Nested scopes just seemed intuitively > obvious: if a function sees the variables in the module surrounding it, > then it should also see the variables in any function surrounding it. > > This behaviour in Python 1.5 made functions MUCH less useful: > > > >>> def outer(): > ... x = 1 > ... def inner(): > ... return x > ... return inner() > ... > >>> outer() > Traceback (innermost last): > File "", line 1, in ? > File "", line 5, in outer > File "", line 4, in inner > NameError: x > > > I think it is fair to say that inner functions in Python 1.5 were > crippled to the point of uselessness. I don't think that's fair to say. A great many functions are in fact ... functions ;-) That is, they compute a result from the arguments passed to them. They don't need more than that, although being able to access globals and builtins and import whatever they want from the standard library made them perfectly capable of doing a whole lot more than just staring at their arguments. To this day, _most_ of the nested functions I write would have worked fine under the original scoping rules, because that's all they need. Many of the rest are recursive, but would also work fine if I passed their names into them and rewrote the bits of code to do recursive calls via the passed-in name. But, yes, I am relieved I don't need to do the latter anymore ;-) ... [snip similar things about closures] ... >> Since then, Python has gone down a pretty bizarre path, inventing >> sublocal scopes on an ad hoc basis by "pure magic" when their absence >> in some specific context seemed just too unbearable to live with >> (e.g., in comprehensions). So we already have
Re: [Python-ideas] A "local" pseudo-function
On Sun, Apr 29, 2018 at 12:50 PM, Steven D'Apranowrote: > On Sat, Apr 28, 2018 at 12:16:16PM -0500, Tim Peters wrote: >> [Steven D'Aprano ] >> > Chris' PEP 572 started off with the concept that binding expressions >> > would create a "sub-local" scope, below function locals. After some >> > debate on Python-Ideas, Chris, Nick and Guido took the discussion off >> > list and decided to drop the sub-local scope idea as confusing and hard >> > to implement. >> >> Enormously harder to implement than binding expressions, and the >> latter (to my eyes) capture many high-value use cases "good enough". > > And yet you're suggesting an alternative which is harder and more > confusing. What's the motivation here for re-introducing sublocal > scopes, if they're hard to do and locals are "good enough"? > > That's not a rhetorical question: why have you suggested this sublocal > scoping idea? PEP 572 stopped talking about sublocals back in revision 2 > or so, and as far as I can see, *not a single objection* since has > been that the variables weren't sublocal. No objections, per se, but I do know a number of people were saddened at their loss. And it's not like eliminating sublocals solved problems without creating more; it's just a different set of trade-offs. Sublocals have their value. >> It was also the case that nesting scopes _at all_ was very >> controversial in Python's earliest years, and Guido resisted it >> mightily (with my full support). The only scopes at first were >> function-local, module-global, and builtin, and while functions could >> _textually_ nest, they had no access to enclosing local scopes. > > While I started off with Python 1.5, I wasn't part of the discussions > about nested scopes. But I'm astonished that you say that nested scopes > were controversial. *Closures* I would completely believe, but mere > lexical scoping? Astonishing. I'm not sure how you can distinguish them: > This behaviour in Python 1.5 made functions MUCH less useful: > > def outer(): > ... x = 1 > ... def inner(): > ... return x > ... return inner() > ... outer() > Traceback (innermost last): > File "", line 1, in ? > File "", line 5, in outer > File "", line 4, in inner > NameError: x > > > I think it is fair to say that inner functions in Python 1.5 were > crippled to the point of uselessness. What you expect here is lexical scope, yes. But if you have lexical scope with no closures, the inner function can ONLY be used while its calling function is still running. What would happen if you returned 'inner' uncalled, and then called the result? How would it resolve the name 'x'? I can't even begin to imagine what lexical scope would do in the absence of closures. At least, not with first-class functions. If functions aren't first-class objects, it's much easier, and a nested function serves as a refactored block of code. But then you can't even pass a nested function to a higher order function. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
I'm pretty sure the debate about braces defining scope in Python has long-since ended... -- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/ On April 28, 2018 9:37:57 PM Ken Hiltonwrote: > local { m = re.match(regexp, line) if m: print(m.group(0)) } Or how about making "local" a pseudo-statement of sorts? local (m=re.match(exp, string)) { if m: print(m.group(0)) } The grammar would be as follows: local_stmt = "local" "(" local_assignments [ "," local_assignments ... ] ")" "{" BLOCK "}" local_assignments = NAME "=" EXPR There would be no question about the scope of things in BLOCK - the variables would disappear after the closing "}". I say "pseudo"-statement because I'm wondering if something like this would be legal: things = list(map(lambda m: local (gp1=m.group(1)) { result = gp1 + ''.join(reversed(gp1)) result += gp1.replace('some', 'thing') return result }, re.finditer(exp, string))) I'm thinking specifically about the "lambda m: local (...) {...}". If that was made legal, it would finally allow for full-fledged anonymous functions. Indeed, the "local" (statement?) itself is actually almost equivalent to defining an anonymous function and executing it immediately, i.e. this: (lambda x=5: x*x)() would be equivalent to this: local (x=5) { return x * x } both evaluating to 25. Just some random thoughts! Sincerely, Ken Hilton ; -- ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sat, Apr 28, 2018 at 12:16:16PM -0500, Tim Peters wrote: > [Steven D'Aprano] > > Chris' PEP 572 started off with the concept that binding expressions > > would create a "sub-local" scope, below function locals. After some > > debate on Python-Ideas, Chris, Nick and Guido took the discussion off > > list and decided to drop the sub-local scope idea as confusing and hard > > to implement. > > Enormously harder to implement than binding expressions, and the > latter (to my eyes) capture many high-value use cases "good enough". And yet you're suggesting an alternative which is harder and more confusing. What's the motivation here for re-introducing sublocal scopes, if they're hard to do and locals are "good enough"? That's not a rhetorical question: why have you suggested this sublocal scoping idea? PEP 572 stopped talking about sublocals back in revision 2 or so, and as far as I can see, *not a single objection* since has been that the variables weren't sublocal. For what it is worth, if we ever did introduce a sublocal scope, I don't hate Nick's "given" block statement: https://www.python.org/dev/peps/pep-3150/ [...] > It was also the case that nesting scopes _at all_ was very > controversial in Python's earliest years, and Guido resisted it > mightily (with my full support). The only scopes at first were > function-local, module-global, and builtin, and while functions could > _textually_ nest, they had no access to enclosing local scopes. While I started off with Python 1.5, I wasn't part of the discussions about nested scopes. But I'm astonished that you say that nested scopes were controversial. *Closures* I would completely believe, but mere lexical scoping? Astonishing. Even when I started, as a novice programmer who wouldn't have recognised the term "lexical scoping" if it fell on my head from a great height, I thought it was strange that inner functions couldn't see their surrounding function's variables. Nested scopes just seemed intuitively obvious: if a function sees the variables in the module surrounding it, then it should also see the variables in any function surrounding it. This behaviour in Python 1.5 made functions MUCH less useful: >>> def outer(): ... x = 1 ... def inner(): ... return x ... return inner() ... >>> outer() Traceback (innermost last): File "", line 1, in ? File "", line 5, in outer File "", line 4, in inner NameError: x I think it is fair to say that inner functions in Python 1.5 were crippled to the point of uselessness. > Adding nested local scopes was also "confusing" at the time, and > indeed made the scoping rules far harder to explain to newbies, and > complicated the implementation. Then again, experienced programmers > overwhelmingly (unanimously?) welcomed the change after it was done. I agree with the above regarding closures, which are harder to explain, welcomed by experienced programmers, and often a source of confusion for newbies and experts alike: https://stackoverflow.com/questions/7546285/creating-lambda-inside-a-loop http://math.andrej.com/2009/04/09/pythons-lambda-is-broken/comment-page-1/ but I disagree that lexical scoping alone is or ever was confusing. Neither did Niklaus Wirth, who included it in Pascal, a language intended to be friendly for beginners *wink* > Since then, Python has gone down a pretty bizarre path, inventing > sublocal scopes on an ad hoc basis by "pure magic" when their absence > in some specific context seemed just too unbearable to live with > (e.g., in comprehensions). So we already have sublocal scopes, but in > no explicit form that can be either exploited or explained. I'm not entirely sure that comprehensions (including generator expressions) alone counts as "a path" :-) but I agree with this. I'm not a fan of comprehensions being their own scope. As far as I am concerned, leakage of comprehension variables was never a problem that needed to be solved, and was (very occasionally) a useful feature. Mostly for introspection and debugging. But the decision was made for generator comprehensions to be in their own scope, and from there I guess it was inevitable that list comprehensions would have to match. > > But the biggest problem is that this re-introduces exactly the same > > awful C mistake that := was chosen to avoid. Which of the following two > > contains the typo? > > > > local(spam=expression, eggs=expression, cheese = spam+eggs) > > > > local(spam=expression, eggs=expression, cheese == spam+eggs) > > Neither :-) I don't expect that to be a real problem. I'm sure the C designers didn't either. You miss the point that looking at the above, it is impossible to tell whether I meant assignment or an equality test. Typos of = for == do happen, even in Python, for whatever reason typos occur. Regardless of whether this makes them more likely or not (I didn't make that claim) once made, it is a bug
Re: [Python-ideas] A "local" pseudo-function
> local { m = re.match(regexp, line) >if m: >print(m.group(0)) > } Or how about making "local" a pseudo-statement of sorts? local (m=re.match(exp, string)) { if m: print(m.group(0)) } The grammar would be as follows: local_stmt = "local" "(" local_assignments [ "," local_assignments ... ] ")" "{" BLOCK "}" local_assignments = NAME "=" EXPR There would be no question about the scope of things in BLOCK - the variables would disappear after the closing "}". I say "pseudo"-statement because I'm wondering if something like this would be legal: things = list(map(lambda m: local (gp1=m.group(1)) { result = gp1 + ''.join(reversed(gp1)) result += gp1.replace('some', 'thing') return result }, re.finditer(exp, string))) I'm thinking specifically about the "lambda m: local (...) {...}". If that was made legal, it would finally allow for full-fledged anonymous functions. Indeed, the "local" (statement?) itself is actually almost equivalent to defining an anonymous function and executing it immediately, i.e. this: (lambda x=5: x*x)() would be equivalent to this: local (x=5) { return x * x } both evaluating to 25. Just some random thoughts! Sincerely, Ken Hilton ; ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
To all of the following, I was talking about the syntax. Which includes that all existing Python-aware editors and IDEs already know how to format it intelligibly. It would be nice if they also colored "local" (or "let") as a keyword, though. For the rest, of course I'm already aware of the _semantic_ trickery. Indeed, that may be too much to bear. But I'm already sympathetic to that too :-) On Sat, Apr 28, 2018 at 9:03 PM, Greg Ewingwrote: > Tim Peters wrote: >> >> To the compiler, it's approximately nothing like "a >> function call". > > > It's nothing like a function call to the user, either, > except in the most superficial of ways. > >> - The explicit parentheses make it impossible to misunderstand where >> the expression begins or ends. > > > Except that you then go and break that rule by saying > that it doesn't apply when you're in the condition of > an "if" statement. > >> I do want to leverage what people "already know". > > > Seems to me this proposal does that in the worst > possible way, by deceiving the user into thinking > it's something familiar when it's not. > > -- > Greg > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
Tim Peters wrote: To the compiler, it's approximately nothing like "a function call". It's nothing like a function call to the user, either, except in the most superficial of ways. - The explicit parentheses make it impossible to misunderstand where the expression begins or ends. Except that you then go and break that rule by saying that it doesn't apply when you're in the condition of an "if" statement. I do want to leverage what people "already know". Seems to me this proposal does that in the worst possible way, by deceiving the user into thinking it's something familiar when it's not. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
let[x](EXPR) x == EXPR let[x](a=1) x == 1 let[x](a=1, EXPR) x == EXPR let[x, y](a=1, EXPR) x == 1 y == EXPR let[x, y](a=1, b=2, EXPR) x == 2 y == EXPR z = let[x, y](a=1, EXPR) x == 1 y == EXPR z == (1, EXPR) Anybody seeing how the above might be useful, and address some of the concerns I've read? I don't recall seeing this suggested prior. I like the idea behind pseudo-function let/local, especially when paired with the explanation of equal sign precedence changes within paren, but I'm having a really hard time getting over the name binding leaking out of the paren. I like this item-ish style because it puts the name itself outside the parentheses while still retaining the benefits in readability. It also allows capturing the entire resultset, or individual parts. On Sat, Apr 28, 2018, 6:00 PM Tim Delaneywrote: > > On Sat, 28 Apr 2018 at 12:41, Tim Peters wrote: > > My big concern here involves the: > > if local(m = re.match(regexp, line)): > print(m.group(0)) > > example. The entire block needs to be implicitly local for that to work - > what happens if I assign a new name in that block? Also, what happens with: > > if local(m = re.match(regexp1, line)) or local(m = re.match(regexp2, > line)): > print(m.group(0)) > > Would the special-casing of local still apply to the block? Or would you > need to do: > > if local(m = re.match(regexp1, line) or re.match(regexp2, line)): > print(m.group(0)) > > This might just be lack of coffee and sleep talking, but maybe new > "scoping delimiters" could be introduced. Yes - I'm suggesting introducing > curly braces for blocks, but with a limited scope (pun intended). Within > a local {} block statements and expressions are evaluated exactly like they > currently are, including branching statements, optional semi-colons, etc. > The value returned from the block is from an explicit return, or the last > evalauted expression. > > a = 1 >> b = 2 >> c = >> >> local(a=3) * local(b=4) >> > > c = local { a=3 } * local { b=4 } > > c = >> >> local(a=3 >> , >> b=4 >> , >> a*b) > > > > c = > > > > local > { > a=3 > ; > b=4 > ; > a*b > } > > > > c = > > > local > { > a = 3 > b = 4 > a * b > > } > > c = local(a=3, >> >> b=local(a=2, a*a), a*b) > > > > c = > > > local > { > a = 3 > b = local(a=2, a*a) > return a * b > > } > > >> >> r1, r2 = local(D = b**2 - 4*a*c, >>sqrtD = math.sqrt(D), >>twoa = 2*a, >>((-b + sqrtD)/twoa, (-b - sqrtD)/twoa)) >> > > > > r1, r2 = local { > D = b**2 - 4*a*c > sqrtD = math.sqrt(D) > twoa = 2*a > return ((-b + sqrtD)/twoa, (-b - sqrtD)/twoa) > } > > >> if local(m = re.match(regexp, line)): >> print(m.group(0)) >> > > > if local { m = re.match(regexp, line) }: > print(m.group(0)) > > And a further implication: > > a = lambda a, b: local(c=4, a*b*c) > > a = lambda a, b: local { > c = 4 > return a * b * c > } > > Tim Delaney > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > On Apr 28, 2018 6:00 PM, "Tim Delaney" wrote: On Sat, 28 Apr 2018 at 12:41, Tim Peters wrote: My big concern here involves the: if local(m = re.match(regexp, line)): print(m.group(0)) example. The entire block needs to be implicitly local for that to work - what happens if I assign a new name in that block? Also, what happens with: if local(m = re.match(regexp1, line)) or local(m = re.match(regexp2, line) ): print(m.group(0)) Would the special-casing of local still apply to the block? Or would you need to do: if local(m = re.match(regexp1, line) or re.match(regexp2, line)): print(m.group(0)) This might just be lack of coffee and sleep talking, but maybe new "scoping delimiters" could be introduced. Yes - I'm suggesting introducing curly braces for blocks, but with a limited scope (pun intended). Within a local {} block statements and expressions are evaluated exactly like they currently are, including branching statements, optional semi-colons, etc. The value returned from the block is from an explicit return, or the last evalauted expression. a = 1 > b = 2 > c = > > local(a=3) * local(b=4) > c = local { a=3 } * local { b=4 } c = > > local(a=3 > , > b=4 > , > a*b) c = local { a=3 ; b=4 ; a*b } c = local { a = 3 b = 4 a * b } c = local(a=3, > > b=local(a=2, a*a), a*b) c = local { a = 3 b = local(a=2, a*a) return a * b } > > r1, r2 = local(D = b**2 - 4*a*c, >sqrtD = math.sqrt(D), >twoa = 2*a, >((-b + sqrtD)/twoa, (-b -
Re: [Python-ideas] A "local" pseudo-function
[Tim Delaney] > My big concern here involves the: > > if local(m = re.match(regexp, line)): > print(m.group(0)) > > example. The entire block needs to be implicitly local for that to work - > what happens if I assign a new name in that block? I really don't know what you're asking there. Can you make it concrete? If, e.g., you're asking what happens if this appeared after the `print`: x = 3.14 then the answer is "the same as what would happen if `local` had not been used". We can't know what that is without context, though. Maybe x is global. Maybe x was declared nonlocal earlier. Maybe it's function-local. While it may be irrelevant to what you're asking, I noted just before: """ Time to note another subtlety: people don't _really_ want "a new scope" in Python. If they did, then _every_ name appearing in a binding context (assignment statement target, `for` target, ...) for the duration would vanish when the new scope ended. What they really want is a new scope with an implied "nonlocal" declaration for every name appearing in a binding context _except_ for the specific names they're effectively trying to declare as being "sublocal" instead. """ If by "new name" you mean that `x` didn't appear in any earlier line, then Python's current analysis would classify `x` as local to the current function (or as global if this is module-level code ...). That wouldn't change. > Also, what happens with: > > if local(m = re.match(regexp1, line)) or local(m = re.match(regexp2, line)): > print(m.group(0)) As is, if `local()` appears in an `if` or `while` expression, the scope extends to the end of the block construct. In that specific case, I'd expect to get a compile-time error, for attempting to initialize the same name more than once in the new scope. If the scope also encompasses associated `elif` statements, I'd instead expect local(m=...) in one of those to be treated as a re-initialization ;-) instead. > Would the special-casing of local still apply to the block? As is, `local()` in an `if` or `while` expressions triggers deeply magical behavior, period. > Or would you need to do: > > if local(m = re.match(regexp1, line) or re.match(regexp2, line)): > print(m.group(0)) Yes, not to trigger magical behavior, but to avoid the compile-time error. > This might just be lack of coffee and sleep talking, but maybe new "scoping > delimiters" could be introduced. Yes - I'm suggesting introducing curly > braces for blocks, but with a limited scope (pun intended). Within a local > {} block statements and expressions are evaluated exactly like they > currently are, including branching statements, optional semi-colons, etc. > The value returned from the block is from an explicit return, or the last > evalauted expression. >> a = 1 >> b = 2 >> c = local(a=3) * local(b=4) > c = local { a=3 } * local { b=4 } >> c = local(a=3 , b=4, a*b) > c = local > { > a = 3 > b = 4 > a * b > } >> c = local(a=3, b=local(a=2, a*a), a*b) > c = > local > { > a = 3 > b = local(a=2, a*a) I expect you wanted b = local{a=2; a*a} there instead (braces instead of parens, and semicolon instead of comma). > return a * b > } >> r1, r2 = local(D = b**2 - 4*a*c, >>sqrtD = math.sqrt(D), >>twoa = 2*a, >>((-b + sqrtD)/twoa, (-b - sqrtD)/twoa)) > r1, r2 = local { > D = b**2 - 4*a*c > sqrtD = math.sqrt(D) > twoa = 2*a > return ((-b + sqrtD)/twoa, (-b - sqrtD)/twoa) > } >> if local(m = re.match(regexp, line)): >> print(m.group(0)) > if local { m = re.match(regexp, line) }: > print(m.group(0)) OK, this is the only case in which you used it in an `if` or `while` expression. All the questions you asked of me at the start can be asked of this spelling too. You seemed to imply at the start that the right curly brace would always mark the end of the new scope. But if that's so, the `m` in `m.group()` has nothing to do with the `m` assigned to in the `local` block - _that_ scope ended before `print` was reached. So if you're not just trying to increase the level of complexity of what can appear in a local block, a fundamental problem still needs solving ;-) I suppose you could solve it like so: local { m = re.match(regexp, line) if m: print(m.group(0)) } but, besides losing the "shortcut", it would also mean something radically different if x = 3.14 appeared after the "print". Right? If a "local block" is taken seriously, then _all_ names bound inside it vanish when the block ends. > And a further implication: > > a = lambda a, b: local(c=4, a*b*c) > > a = lambda a, b: local { > c = 4 > return a * b * c > } If people do want a for-real "new scope" in Python, I certainly agree `local {...}` is far better suited for that purpose. ___ Python-ideas mailing list
Re: [Python-ideas] A "local" pseudo-function
On Sun, Apr 29, 2018 at 06:20:14AM +1000, Chris Angelico wrote: > On Sun, Apr 29, 2018 at 6:04 AM, Ed Kellettwrote: > > What if the names persist until the end of the statement? [...] > Oh, you mean exactly like PEP 572 used to advocate for? :) Indeed. It was a bad idea in revision 1 and it remains a bad idea now :-) -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sat, 28 Apr 2018 at 12:41, Tim Peterswrote: My big concern here involves the: if local(m = re.match(regexp, line)): print(m.group(0)) example. The entire block needs to be implicitly local for that to work - what happens if I assign a new name in that block? Also, what happens with: if local(m = re.match(regexp1, line)) or local(m = re.match(regexp2, line) ): print(m.group(0)) Would the special-casing of local still apply to the block? Or would you need to do: if local(m = re.match(regexp1, line) or re.match(regexp2, line)): print(m.group(0)) This might just be lack of coffee and sleep talking, but maybe new "scoping delimiters" could be introduced. Yes - I'm suggesting introducing curly braces for blocks, but with a limited scope (pun intended). Within a local {} block statements and expressions are evaluated exactly like they currently are, including branching statements, optional semi-colons, etc. The value returned from the block is from an explicit return, or the last evalauted expression. a = 1 > b = 2 > c = > > local(a=3) * local(b=4) > c = local { a=3 } * local { b=4 } c = > > local(a=3 > , > b=4 > , > a*b) c = local { a=3 ; b=4 ; a*b } c = local { a = 3 b = 4 a * b } c = local(a=3, > > b=local(a=2, a*a), a*b) c = local { a = 3 b = local(a=2, a*a) return a * b } > > r1, r2 = local(D = b**2 - 4*a*c, >sqrtD = math.sqrt(D), >twoa = 2*a, >((-b + sqrtD)/twoa, (-b - sqrtD)/twoa)) > r1, r2 = local { D = b**2 - 4*a*c sqrtD = math.sqrt(D) twoa = 2*a return ((-b + sqrtD)/twoa, (-b - sqrtD)/twoa) } > if local(m = re.match(regexp, line)): > print(m.group(0)) > if local { m = re.match(regexp, line) }: print(m.group(0)) And a further implication: a = lambda a, b: local(c=4, a*b*c) a = lambda a, b: local { c = 4 return a * b * c } Tim Delaney ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Chris Angelico] > I'm concerned that there are, in effect, two quite different uses of > the exact same syntax. Yes, the construct implements a profoundly different meaning of "scope" depending on the context it appears in. > 1) In an arbitrary expression, local() creates a scope that is defined > entirely by the parentheses. Yes. > 2) In an 'if' header, the exact same local() call creates a scope that > extends to the corresponding suite. And in a 'while' header, and also possibly (likely) including associated suites (elif/else). So it goes ;-) There is nothing "obvious" you can say inside an "if" or "while" expression that says "and, oh ya, this name also shadows anything of the same name from here on, except when it stops doing so". Even in C, e.g., it's not *obvious* what the scope of `i` is in: for (int i = 0; ...) { } It needs to be learned. Indeed, it's so non-obvious that C and C++ give different answers. The {...} part by itself introduces a new scope in both languages. In C++ the `int i` is viewed as being _part_ of that scope, despite that it's outside the braces. But in C the `int i` is really viewed as being part of a Yet Another new scope _enclosing_ the scope introduced by {...}, but nevertheless ending when the {...} scope ends. Not that it matters much. The practical effect is that, e.g., double i = 3.0; is legal as the first line of the block in C (shadows the `int i`), but illegal in C++ (a conflicting declaration for `i` in a single scope). In either case, it's only "obvious" if you learned it and then stopped thinking too much about it ;-) > For instance: > > a = 1; b = 2 > x = a + local(a = 3, b = 4, a + b) + b > if x == 10: > # Prints "x is 10: 1 2" > print("x is 10: ", a, b) > > This makes reasonable sense. The parentheses completely enclose the > local scope. It's compiler magic, and you cannot explain it as a > function call, but it makes intuitive sense. Yup, it's effectively a function-like spelling of any number of binding constructs widely used in functional languages. I had mostly in mind Haskell's "let" pile-of-bindings "in" expression spelled as "local(" pile-of-bindings "," expression ")" The points to using function-call-like syntax were already covered ("nothing syntactically new to learn there", since the syntax for specifying keyword arguments is already understood, and already groups as intended). > But the same thing inside the if header itself would be much weirder. > I'm actually not even sure what it would do. You think I am? ;-) I don't know that it matters, because intended use cases are far simpler than all the goofy things people _can_ dream up just for the hell of it. They need to be defined, but exactly how isn't of much interest to me. For example, let's put your example in an `if`: a = 1; b = 2 if a + local(a = 3, b = 4, a + b) + b: The rules I sketched pretty clearly imply that would be evaluated as: if 1 + (3+4) + 4: It's the final "4" that's of interest. In your original example the original `b` was restored because ")" ended the new scope, leaving the final "+b" to resolve to "+2". But because it's in an "if" expression here, the new scope doesn't end at ")" anymore. > And you've clearly shown that the local() call can > be anywhere inside the condition, based on these examples: And/or used multiple times, and/or used in nested ways. None of which anyone will actually do ;-) >> ... >> if local(m = re.match(regexp, line)) is not None: >> print(m.group(0)) > > At what point does the name 'm' stop referring to the local? More generally: Probably at the end of the final (if any) `elif` or `else` suite associated with the `if`/`while`, but possibly at the end of the suite associated with the `if`/`while`. Time to note another subtlety: people don't _really_ want "a new scope" in Python. If they did, then _every_ name appearing in a binding context (assignment statement target, `for` target, ...) for the duration would vanish when the new scope ended. What they really want is a new scope with an implied "nonlocal" declaration for every name appearing in a binding context _except_ for the specific names they're effectively trying to declare as being "sublocal" instead. So It's somewhat of a conceptual mess no mater how it's spelled ;) In most other languages this doesn't come up because the existence of a variable in a scope is established by an explicit declaration rather than inferred from examining binding sites. > > if local(m = ...) is not m: > print("Will I ever happen?") No, that `print` can't be reached. > Perhaps it would be better to make this special case *extremely* > special. For instance: > > if_local: 'if' 'local' '(' local_item (',' local_item)* ')' ':' suite > > as the ONLY way to have the local names persist. In other words, if > you tack "is not None" onto the outside of the local() call, it > becomes a regular
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-28 21:40, Tim Peters wrote: > [Ed Kellett] >> How about if you just can't have an expression in a local()? > > See the quadratic equation example in the original post. When > working with expressions, the entire point of the construct is to > define (sub)local names for use in a result expression. > > [snip] > > Requiring "in" requires annoying syntactic repetition in the common > > if local(match=re.match(regexp, line)) in match: > > kinds of cases. I don't mean "in" should be required, but rather that the last thing is always an assignment, and the local() yields the assigned value. So that'd remain: if local(match=re.match(regexp, line)): whereas your quadratic equation might do the "in" thing: r1, r2 = local(D = b**2 - 4*a*c, sqrtD = math.sqrt(D), twoa = 2*a) in ( (-b + sqrtD)/twoa, (-b - sqrtD)/twoa) ... which doesn't look particularly wonderful (I think it's nicer with my option 3), but then I've never thought the quadratic equation was a good motivating case for any version of an assignment expression. In most cases I can imagine, a plain local() would be sufficient, e.g.: if local(match=re.match(regexp, line)) and match[1] == 'frog': > Making "in" optional instead was discussed near the end of the > original post. I agree that your spelling just above is more obvious > than `local(x=1, y=2, x+y)` which is why the original post discussed > making an "in clause" optional. But, overwhelmingly, it appears that > people are more interested in establishing sublocal scopes in `if` > and `while` constructs than in standalone expressions, so I picked a > spelling that's _most_ convenient for the latter's common "just name > a result and test its truthiness" uses. > > [snip] > > Indeed, 3 is the only one I'd consider, but I don't see that it's a > real improvement. It seems to require extra typing every time just > to avoid learning "and it returns the value of the last expression" > once. I agree with pretty much all of this--I was trying to attack the "but you could make the terrible C mistake" problem from another angle. I don't think I mind the last "argument" being *allowed* to be an expression, but if the consensus is that it makes '='/'==' confusion too easy, I think it'd be more straightforward to say they're all assignments (with optional, explicit syntax to yield an expression) than to say "they're all assignments except the last thing" (argh) "except in the special case of one argument" (double argh). signature.asc Description: OpenPGP digital signature ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Ed Kellett] > How about if you just can't have an expression in a local()? See the quadratic equation example in the original post. When working with expressions, the entire point of the construct is to define (sub)local names for use in a result expression. > There are a few obvious alternative possibilities: > > 1. No special case at all: > > local(x=1, y=2, _=x+y) As above. > 2. Co-opt a keyword after local(): > > local(x=1, y=2) in x+y Requiring "in" requires annoying syntactic repetition in the common if local(match=re.match(regexp, line)) in match: kinds of cases. Making "in" optional instead was discussed near the end of the original post. I agree that your spelling just above is more obvious than `local(x=1, y=2, x+y)` which is why the original post discussed making an "in clause" optional. But, overwhelmingly, it appears that people are more interested in establishing sublocal scopes in `if` and `while` constructs than in standalone expressions, so I picked a spelling that's _most_ convenient for the latter's common "just name a result and test its truthiness" uses. > 3. Co-opt a keyword inside local(): > > local(x=1, y=2, return x+y) Why would that be better than local(x=1, y=2, x+y)? That no binding is intended for `x+y` is already obvious in the latter. > I hate the first and wish the _ pattern would die in all its forms, but > it's worth mentioning. I don't think there's much to choose between the > other two, but 2 uses syntax that might have been valid and meant > something else, so 3 is probably less confusing. Indeed, 3 is the only one I'd consider, but I don't see that it's a real improvement. It seems to require extra typing every time just to avoid learning "and it returns the value of the last expression" once. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sun, Apr 29, 2018 at 6:04 AM, Ed Kellettwrote: > On 2018-04-28 18:36, Chris Angelico wrote: >> This makes reasonable sense. The parentheses completely enclose the >> local scope. It's compiler magic, and you cannot explain it as a >> function call, but it makes intuitive sense. But the same thing inside >> the if header itself would be much weirder. I'm actually not even sure >> what it would do. And you've clearly shown that the local() call can >> be anywhere inside the condition, based on these examples: > > What if the names persist until the end of the statement? That covers if > (where the statement lasts until the end of the if...elif...else block) > and regular expressions, though it does introduce a potentially annoying > shadowing thing: > x = 2 (local(x=4, x + 1), x) > (5,4) > Oh, you mean exactly like PEP 572 used to advocate for? :) Can't say I'd be against that, although I'm not enamoured of the function-like syntax. But if you remove the function-like syntax and change the semantics, it isn't exactly Tim's proposal any more. :) ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-28 18:16, Tim Peters wrote: > [Steven D'Aprano] >> But the biggest problem is that this re-introduces exactly the same >> awful C mistake that := was chosen to avoid. Which of the following two >> contains the typo? >> >> local(spam=expression, eggs=expression, cheese = spam+eggs) >> >> local(spam=expression, eggs=expression, cheese == spam+eggs) > > [snip] > > Still, if people are scared of that, a variation of Yury's alternative > avoids it: the last "argument" must be an expression (not a binding). > In that case your first line above is a compile-time error. > > I didn't like that because I really dislike the textual redundancy in the > common > > if local(matchobject=re.match(regexp, line), matchobject): > > compared to > > if local(matchobject=re.match(regexp, line)): > > But I could compromise ;-) > > - There must be at least one argument. > - The first argument must be a binding. > - All but the last argument must also be bindings. > - If there's more than one argument, the last argument must be an expression. How about if you just can't have an expression in a local()? There are a few obvious alternative possibilities: 1. No special case at all: local(x=1, y=2, _=x+y) 2. Co-opt a keyword after local(): local(x=1, y=2) in x+y 3. Co-opt a keyword inside local(): local(x=1, y=2, return x+y) I hate the first and wish the _ pattern would die in all its forms, but it's worth mentioning. I don't think there's much to choose between the other two, but 2 uses syntax that might have been valid and meant something else, so 3 is probably less confusing. signature.asc Description: OpenPGP digital signature ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-28 18:36, Chris Angelico wrote: > This makes reasonable sense. The parentheses completely enclose the > local scope. It's compiler magic, and you cannot explain it as a > function call, but it makes intuitive sense. But the same thing inside > the if header itself would be much weirder. I'm actually not even sure > what it would do. And you've clearly shown that the local() call can > be anywhere inside the condition, based on these examples: What if the names persist until the end of the statement? That covers if (where the statement lasts until the end of the if...elif...else block) and regular expressions, though it does introduce a potentially annoying shadowing thing: >>> x = 2 >>> (local(x=4, x + 1), x) (5,4) signature.asc Description: OpenPGP digital signature ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
[Steven D'Aprano] > Chris' PEP 572 started off with the concept that binding expressions > would create a "sub-local" scope, below function locals. After some > debate on Python-Ideas, Chris, Nick and Guido took the discussion off > list and decided to drop the sub-local scope idea as confusing and hard > to implement. Enormously harder to implement than binding expressions, and the latter (to my eyes) capture many high-value use cases "good enough". I'm not concerned about "confusing". "Sub-local" scopes are ubiquitous in modern languages, and they're aimed at experienced programmers. It was also the case that nesting scopes _at all_ was very controversial in Python's earliest years, and Guido resisted it mightily (with my full support). The only scopes at first were function-local, module-global, and builtin, and while functions could _textually_ nest, they had no access to enclosing local scopes. Cute: to write a recursive nested function, you needed to pass it its own name (because its own name isn't in its _own_ local scope, but in the local scope of the function that contains it). Adding nested local scopes was also "confusing" at the time, and indeed made the scoping rules far harder to explain to newbies, and complicated the implementation. Then again, experienced programmers overwhelmingly (unanimously?) welcomed the change after it was done. Since then, Python has gone down a pretty bizarre path, inventing sublocal scopes on an ad hoc basis by "pure magic" when their absence in some specific context seemed just too unbearable to live with (e.g., in comprehensions). So we already have sublocal scopes, but in no explicit form that can be either exploited or explained. > But the biggest problem is that this re-introduces exactly the same > awful C mistake that := was chosen to avoid. Which of the following two > contains the typo? > > local(spam=expression, eggs=expression, cheese = spam+eggs) > > local(spam=expression, eggs=expression, cheese == spam+eggs) Neither :-) I don't expect that to be a real problem. In C I'm _thinking_ "if a equals b" and type "if (a=b)" by mistake in haste. In a "local" I'm _ thinking_ "I want to create these names with these values" in the former case, and in the latter case also "and I want to to test whether cheese equals spam + eggs". But having already typed "=" to mean "binding" twice in the same line, "but the third time I type it it will mean equality instead" just doesn't seem likely. The original C mistake is exceedingly unlikely on the face of it: if what I'm thinking is "if a equals b", or "while a equals b", I'm not going to use "local()" _at all_. Forcing the programmer to be explicit about that they're trying to create a new scope limits the only possible confusions to cases where they _are_ going out of their way to use a "local()" construct, in which case binding behavior is very much at the top of their mind. Plain old "if a=b" remains a SyntaxError regardless. Still, if people are scared of that, a variation of Yury's alternative avoids it: the last "argument" must be an expression (not a binding). In that case your first line above is a compile-time error. I didn't like that because I really dislike the textual redundancy in the common if local(matchobject=re.match(regexp, line), matchobject): compared to if local(matchobject=re.match(regexp, line)): But I could compromise ;-) - There must be at least one argument. - The first argument must be a binding. - All but the last argument must also be bindings. - If there's more than one argument, the last argument must be an expression. Then your first line above is a compile-time error, the common "just name a result and test its truthiness" doesn't require repetition, and "local(a==b)" is also a compile-time error. > I have other objections, but I'll leave them for now, since I think > these two alone are fatal. I don't. > Once you drop those two flaws, you're basically left with PEP 572 :-) Which is fine by me, but do realize that since PEP 572 dropped any notion of sublocal scopes, that recurring issue remains wholly unaddressed regardless. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
>$(a=7, $(a=a+1, a*2)) I suspect you ALREADY have bash installed on your computer, you don't need Python to emulate it. On Sat, Apr 28, 2018 at 6:22 AM, Zero Piraeuswrote: > : > > On 28 April 2018 at 07:07, Tim Peters wrote: > > [...] For that matter, I'd be fine too with shortening it to "let". In > > fact, I prefer that! Thanks :-) > > If you really wanted to overcome objections that this looks too much > like a function, you could spell it "$". I'm not sure what I think > about > > $(a=7, $(a=a+1, a*2)) > > yet, but it doesn't make me want to run screaming in the way that := does. > > I think I finally worked out why I have such a violent reaction to := > in the end, by the way: it's because it reminds me of Javascript's > "===" (not the meaning, but the fact that it exists). > > -[]z. > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 2018-04-27 11:37 PM, Tim Peters wrote: A brain dump, inspired by various use cases that came up during the binding expression discussions. Idea: introduce a "local" pseudo-function to capture the idea of initialized names with limited scope. As an expression, it's "local" "(" arguments ")" - Because it "looks like" a function call, nobody will expect the targets of named arguments to be fancier than plain names. - `a=12` in the "argument" list will (& helpfully so) mean pretty much the same as "a=12" in a "def" statement. - In a "local call" on its own, the scope of a named argument begins at the start of the next (if any) argument, and ends at the closing ")". For the duration, any variable of the same name in an enclosing scope is shadowed. - The parentheses allow for extending over multiple lines without needing to teach editors (etc) any new tricks (they already know how to format function calls with arglists spilling over multiple lines). - The _value_ of a local "call" is the value of its last "argument". In part, this is a way to sneak in C's comma operator without adding cryptic new line noise syntax. Time for an example. First a useless one: a = 1 b = 2 c = local(a=3) * local(b=4) Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the end: c = local(a=3, b=4, a*b) And just to be obscure, also the same: c = local(a=3, b=local(a=2, a*a), a*b) There the inner `a=2` temporarily shadows the outer `a=3` just long enough to compute `a*a` (4). This is one that little else really handled nicely: r1, r2 = local(D = b**2 - 4*a*c, sqrtD = math.sqrt(D), twoa = 2*a, ((-b + sqrtD)/twoa, (-b - sqrtD)/twoa)) Everyone's favorite: if local(m = re.match(regexp, line)): print(m.group(0)) Here's where it's truly essential that the compiler know everything about "local", because in _that_ context it's required that the new scope extend through the end of the entire block construct (exactly what that means TBD - certainly through the end of the `if` block, but possibly also through the end of its associated (if any) `elif` and `else` blocks - and similarly for while/else constructs). Of course that example could also be written as: if local(m = re.match(regexp, line), m): print(m.group(0)) or more specifically: if local(m = re.match(regexp, line), m is not None): print(m.group(0)) or even: if local(m = re.match(regexp, line)) is not None: print(m.group(0)) A listcomp example, building the squares of integers from an iterable but only when the square is a multiple of 18: squares18 = [i2 for i in iterable if local(i2=i*i) % 18 == 0] That's a bit mind-bending, but becomes clear if you picture the kinda-equivalent nest: for i in iterable: if local(i2=i*i) % 18 == 0: append i2 to the output list That should also make clear that if `iterable` or `i` had been named `i2` instead, no problem. The `i2` created by `local()` is in a wholly enclosed scope. Drawbacks: since this is just a brain dump, absolutely none ;-) Q: Some of those would be clearer if it were the more Haskell-like local(...) "in" expression A: Yup, but for some of the others needing to add "in m" would be annoyingly redundant noise. Making an "in" clause optional doesn't really fly either, because then local(a='z') in 'xyz' would be ambiguous. Is it meant to return `'xyz'`, or evaluate `'z' in 'xyz'`? And any connector other than "in" would make the loose resemblance to Haskell purely imaginary ;-) Q: Didn't you drone on about how assignment expressions with complex targets seemed essentially useless without also introducing a "comma operator" - and now you're sneaking the latter in but _still_ avoiding complex targets?! A. Yes, and yes :-) The syntactic complexity of the fully general assignment statement is just too crushing to _sanely_ shoehorn into any "expression-like" context. Q: What's the value of this? local(a=7, local(a=a+1, a*2)) A: 16. Obviously. Q: Wow - that _is_ obvious! OK, what about this, where there is no `a` in any enclosing scope: local(a) A: I think it should raise NameError, just like a function call would. There is no _intent_ here to allow merely declaring a local variable without supplying an initial value. Q: What about local(2, 4, 5)? A: It should return 5, and introduce no names. I don't see a point to trying to outlaw stupidity ;-) Then again, it would be consistent with the _intent_ to require that all but the last "argument" be of the `name=expression` form. Q: Isn't changing the meaning of scope depending on context wy magical? A: Yup! But in a language with such a strong distinction between statements and expressions, without a bit of deep magic there's no single syntax I can dream up that could work well for both that didn't require _some_ deep magic. The gimmick here is something I expect will be
Re: [Python-ideas] A "local" pseudo-function
I have to say, this idea feels really nice to me. It's far easier to read than := and separates the assignments and the result expression nicely. Others have brought up the same problem of = vs ==. IMO a solution could be to make a requirement that the last argument is NOT an assignment. In other words, this would be illegal: local(a=1) and you would have to do this: local(a=1, a) Now if the user mixes up = and ==, it'd be a "compile-time error". -- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/ On April 27, 2018 9:41:57 PM Tim Peterswrote: A brain dump, inspired by various use cases that came up during the binding expression discussions. Idea: introduce a "local" pseudo-function to capture the idea of initialized names with limited scope. As an expression, it's "local" "(" arguments ")" - Because it "looks like" a function call, nobody will expect the targets of named arguments to be fancier than plain names. - `a=12` in the "argument" list will (& helpfully so) mean pretty much the same as "a=12" in a "def" statement. - In a "local call" on its own, the scope of a named argument begins at the start of the next (if any) argument, and ends at the closing ")". For the duration, any variable of the same name in an enclosing scope is shadowed. - The parentheses allow for extending over multiple lines without needing to teach editors (etc) any new tricks (they already know how to format function calls with arglists spilling over multiple lines). - The _value_ of a local "call" is the value of its last "argument". In part, this is a way to sneak in C's comma operator without adding cryptic new line noise syntax. Time for an example. First a useless one: a = 1 b = 2 c = local(a=3) * local(b=4) Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the end: c = local(a=3, b=4, a*b) And just to be obscure, also the same: c = local(a=3, b=local(a=2, a*a), a*b) There the inner `a=2` temporarily shadows the outer `a=3` just long enough to compute `a*a` (4). This is one that little else really handled nicely: r1, r2 = local(D = b**2 - 4*a*c, sqrtD = math.sqrt(D), twoa = 2*a, ((-b + sqrtD)/twoa, (-b - sqrtD)/twoa)) Everyone's favorite: if local(m = re.match(regexp, line)): print(m.group(0)) Here's where it's truly essential that the compiler know everything about "local", because in _that_ context it's required that the new scope extend through the end of the entire block construct (exactly what that means TBD - certainly through the end of the `if` block, but possibly also through the end of its associated (if any) `elif` and `else` blocks - and similarly for while/else constructs). Of course that example could also be written as: if local(m = re.match(regexp, line), m): print(m.group(0)) or more specifically: if local(m = re.match(regexp, line), m is not None): print(m.group(0)) or even: if local(m = re.match(regexp, line)) is not None: print(m.group(0)) A listcomp example, building the squares of integers from an iterable but only when the square is a multiple of 18: squares18 = [i2 for i in iterable if local(i2=i*i) % 18 == 0] That's a bit mind-bending, but becomes clear if you picture the kinda-equivalent nest: for i in iterable: if local(i2=i*i) % 18 == 0: append i2 to the output list That should also make clear that if `iterable` or `i` had been named `i2` instead, no problem. The `i2` created by `local()` is in a wholly enclosed scope. Drawbacks: since this is just a brain dump, absolutely none ;-) Q: Some of those would be clearer if it were the more Haskell-like local(...) "in" expression A: Yup, but for some of the others needing to add "in m" would be annoyingly redundant noise. Making an "in" clause optional doesn't really fly either, because then local(a='z') in 'xyz' would be ambiguous. Is it meant to return `'xyz'`, or evaluate `'z' in 'xyz'`? And any connector other than "in" would make the loose resemblance to Haskell purely imaginary ;-) Q: Didn't you drone on about how assignment expressions with complex targets seemed essentially useless without also introducing a "comma operator" - and now you're sneaking the latter in but _still_ avoiding complex targets?! A. Yes, and yes :-) The syntactic complexity of the fully general assignment statement is just too crushing to _sanely_ shoehorn into any "expression-like" context. Q: What's the value of this? local(a=7, local(a=a+1, a*2)) A: 16. Obviously. Q: Wow - that _is_ obvious! OK, what about this, where there is no `a` in any enclosing scope: local(a) A: I think it should raise NameError, just like a function call would. There is no _intent_ here to allow merely declaring a local variable without supplying an initial value. Q: What about local(2, 4, 5)? A: It should
Re: [Python-ideas] A "local" pseudo-function
On Sat, Apr 28, 2018 at 8:22 PM, Zero Piraeuswrote: > I think I finally worked out why I have such a violent reaction to := > in the end, by the way: it's because it reminds me of Javascript's > "===" (not the meaning, but the fact that it exists). Out of morbid curiosity, why does it remind you of that? ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
: On 28 April 2018 at 07:07, Tim Peterswrote: > [...] For that matter, I'd be fine too with shortening it to "let". In > fact, I prefer that! Thanks :-) If you really wanted to overcome objections that this looks too much like a function, you could spell it "$". I'm not sure what I think about $(a=7, $(a=a+1, a*2)) yet, but it doesn't make me want to run screaming in the way that := does. I think I finally worked out why I have such a violent reaction to := in the end, by the way: it's because it reminds me of Javascript's "===" (not the meaning, but the fact that it exists). -[]z. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Fri, Apr 27, 2018 at 09:37:53PM -0500, Tim Peters wrote: > A brain dump, inspired by various use cases that came up during the > binding expression discussions. > > Idea: introduce a "local" pseudo-function to capture the idea of > initialized names with limited scope. [...] Chris' PEP 572 started off with the concept that binding expressions would create a "sub-local" scope, below function locals. After some debate on Python-Ideas, Chris, Nick and Guido took the discussion off list and decided to drop the sub-local scope idea as confusing and hard to implement. But the biggest problem is that this re-introduces exactly the same awful C mistake that := was chosen to avoid. Which of the following two contains the typo? local(spam=expression, eggs=expression, cheese = spam+eggs) local(spam=expression, eggs=expression, cheese == spam+eggs) I have other objections, but I'll leave them for now, since I think these two alone are fatal. Once you drop those two flaws, you're basically left with PEP 572 :-) -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On 28 April 2018 at 03:37, Tim Peterswrote: > Idea: introduce a "local" pseudo-function to capture the idea of > initialized names with limited scope. This looks disturbingly good to me. I say "disturbingly" because the amount of magic involved in that "function call" is pretty high, and the more you look at it, the more weird its behaviour seems. I have essentially no real world use cases for this, though (other than ones that have been brought up already in the binding expression debate), so my comments are purely subjective opinion. [...] > Everyone's favorite: > > if local(m = re.match(regexp, line)): > print(m.group(0)) > > Here's where it's truly essential that the compiler know everything > about "local", because in _that_ context it's required that the new > scope extend through the end of the entire block construct (exactly > what that means TBD - certainly through the end of the `if` block, but > possibly also through the end of its associated (if any) `elif` and > `else` blocks - and similarly for while/else constructs). This is where, in my mind, the magic behaviour goes too far. I can see why it's essential that this happens, but I can't come up with a justification for it other than pure expediency. And while I know "practicality beats purity" (look at me, daring to quote the Zen at Tim!!! :-)) this just feels like a step too far to me. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
On Sat, Apr 28, 2018 at 2:07 AM Tim Peterswrote: [...] > Speaking of which, "sublocal" would be a more accurate name, and less > likely to conflict with existing code names, but after people got over > the shock to their sense of purity, they'd appreciate typing the > shorter "local" instead ;-) > For that matter, I'd be fine too with shortening it to "let". In > fact, I prefer that! Thanks :-) Great! :) [...] > So, curiously enough, I'd be fonder of > result_expression "where" name=expression, ... > than of > "let" name=expression, ... My gripe about the "where"-like syntax family (including "as", proposed by many) is that the whole expression needs to be read backwards to make sense of `name*. It's one of the things that annoy me in languages like SQL, where the relevant "AS" keyword can be buried a couple of screens below/above the area of interest. That's why I find the "let" form superior. [..] > instead. In _an expression_, I naturally group the > a = 3, a > part as the unintended > a = (3, a) > When I'm thinking of function calls, though, I naturally use the intended > (a=3), a > grouping. I really don't want to fight with what "everyone already > knows", but build on that to the extent possible. Looking at all of these and other examples in your email I have to agree that the version with parenthesis reads way more clearly. A different (and correct in this case) precedence of "," between parens is pretty much hardwired in our brains. [..] > Why not? A great many objects in Python are _designed_ so that their > __bool__ method does a useful thing in a plain > if object: > test. In these common cases, needing to type > if let name=object, object: Alright. And yes, I agree, surrounding parens are badly needed. Maybe we can replace parens with {}? Although my immediate reaction to that is "it's too ugly to be taken seriously". In any case, the similarity of this new syntax to a function call still bothers me. If I just looked at some Python 3.xx code and saw the new "let(..)" syntax, I would assume that the names it declares are only visible *within* the parens. Parens imply some locality of whatever is happening between them. Even if I googled the docs first to learn some basics about "let", when I saw "if let(name...)" it wouldn't be immediately apparent to me that `name` is set only for its "if" block (contrary to "if let a = 1: print(a)"). Yury ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "local" pseudo-function
Hi Tim, This is interesting. Even "as is" I prefer this to PEP 572. Below are some comments and a slightly different idea inspired by yours (sorry!) On Fri, Apr 27, 2018 at 10:41 PM Tim Peterswrote: [..] > As an expression, it's > "local" "(" arguments ")" > - Because it "looks like" a function call, nobody will expect the targets >of named arguments to be fancier than plain names. [..] > Everyone's favorite: > if local(m = re.match(regexp, line)): > print(m.group(0)) > Here's where it's truly essential that the compiler know everything > about "local", because in _that_ context it's required that the new > scope extend through the end of the entire block construct (exactly It does look like a function call, although it has a slightly different syntax. In regular calls we don't allow positional arguments to go after keyword arguments. Hence the compiler/parser will have to know what 'local(..)' is *regardless* of where it appears. If you don't want to make 'local' a new keyword, we would need to make the compiler/parser to trace the "local()" name to check if it was imported or is otherwise "local". This would add some extra complexity to already complex code. Another problematic case is when one has a big file and someone adds their own "def local()" function to it at some point, which would break things. Therefore, "local" should probably be a keyword. Perhaps added to Python with a corresponding "from __future__" import. The other way would be to depart from the function call syntax by dropping the parens. (And maybe rename "local" to "let" ;)) In this case, the syntax will become less like a function call but still distinct enough. We will be able to unambiguously parse & compile it. The cherry on top is that we can make it work even without a "__future__" import! When we implemented PEP 492 in Python 3.5 we did a little trick in tokenizer to treat "async def" in a special way. Tokenizer would switch to an "async" mode and yield ASYNC and AWAIT tokens instead of NAME tokens. This resulted in async/await syntax available without a __future__ import, while having full backwards compatibility. We can do a similar trick for "local" / "let" syntax, allowing the following: "let" NAME "=" expr ("," NAME = expr)* ["," expr] * "if local(m = re.match(...), m):" becomes "if let m = re.match(...), m:" * "c = local(a=3) * local(b=4)" becomes "c = let a=3, b=4, a*b" or "c = (let a=3, b=4, a*b)" * for i in iterable: if let i2=i*i, i2 % 18 == 0: append i2 to the output list etc. Note that I don't propose this new "let" or "local" to return their last assignment. That should be done explicitly (as in your "local(..)" idea): `let a = 'spam', a`. Potentially we could reuse our function return annotation syntax, changing the last example to `let a = "spam" -> a` but I think it makes the whole thing to look unnecessarily complex. One obvious downside is that "=" would have a different precedence compared to a regular assignment statement. But it already has a different precedent in function calls, so maybe this isn't a big deal, considered that we'll have a keyword before it. I think that "let" was discussed a couple of times recently, but it's really hard to find a definitive reason of why it was rejected (or was it?) in the ocean of emails about assignment expressions. Yury ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] A "local" pseudo-function
A brain dump, inspired by various use cases that came up during the binding expression discussions. Idea: introduce a "local" pseudo-function to capture the idea of initialized names with limited scope. As an expression, it's "local" "(" arguments ")" - Because it "looks like" a function call, nobody will expect the targets of named arguments to be fancier than plain names. - `a=12` in the "argument" list will (& helpfully so) mean pretty much the same as "a=12" in a "def" statement. - In a "local call" on its own, the scope of a named argument begins at the start of the next (if any) argument, and ends at the closing ")". For the duration, any variable of the same name in an enclosing scope is shadowed. - The parentheses allow for extending over multiple lines without needing to teach editors (etc) any new tricks (they already know how to format function calls with arglists spilling over multiple lines). - The _value_ of a local "call" is the value of its last "argument". In part, this is a way to sneak in C's comma operator without adding cryptic new line noise syntax. Time for an example. First a useless one: a = 1 b = 2 c = local(a=3) * local(b=4) Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the end: c = local(a=3, b=4, a*b) And just to be obscure, also the same: c = local(a=3, b=local(a=2, a*a), a*b) There the inner `a=2` temporarily shadows the outer `a=3` just long enough to compute `a*a` (4). This is one that little else really handled nicely: r1, r2 = local(D = b**2 - 4*a*c, sqrtD = math.sqrt(D), twoa = 2*a, ((-b + sqrtD)/twoa, (-b - sqrtD)/twoa)) Everyone's favorite: if local(m = re.match(regexp, line)): print(m.group(0)) Here's where it's truly essential that the compiler know everything about "local", because in _that_ context it's required that the new scope extend through the end of the entire block construct (exactly what that means TBD - certainly through the end of the `if` block, but possibly also through the end of its associated (if any) `elif` and `else` blocks - and similarly for while/else constructs). Of course that example could also be written as: if local(m = re.match(regexp, line), m): print(m.group(0)) or more specifically: if local(m = re.match(regexp, line), m is not None): print(m.group(0)) or even: if local(m = re.match(regexp, line)) is not None: print(m.group(0)) A listcomp example, building the squares of integers from an iterable but only when the square is a multiple of 18: squares18 = [i2 for i in iterable if local(i2=i*i) % 18 == 0] That's a bit mind-bending, but becomes clear if you picture the kinda-equivalent nest: for i in iterable: if local(i2=i*i) % 18 == 0: append i2 to the output list That should also make clear that if `iterable` or `i` had been named `i2` instead, no problem. The `i2` created by `local()` is in a wholly enclosed scope. Drawbacks: since this is just a brain dump, absolutely none ;-) Q: Some of those would be clearer if it were the more Haskell-like local(...) "in" expression A: Yup, but for some of the others needing to add "in m" would be annoyingly redundant noise. Making an "in" clause optional doesn't really fly either, because then local(a='z') in 'xyz' would be ambiguous. Is it meant to return `'xyz'`, or evaluate `'z' in 'xyz'`? And any connector other than "in" would make the loose resemblance to Haskell purely imaginary ;-) Q: Didn't you drone on about how assignment expressions with complex targets seemed essentially useless without also introducing a "comma operator" - and now you're sneaking the latter in but _still_ avoiding complex targets?! A. Yes, and yes :-) The syntactic complexity of the fully general assignment statement is just too crushing to _sanely_ shoehorn into any "expression-like" context. Q: What's the value of this? local(a=7, local(a=a+1, a*2)) A: 16. Obviously. Q: Wow - that _is_ obvious! OK, what about this, where there is no `a` in any enclosing scope: local(a) A: I think it should raise NameError, just like a function call would. There is no _intent_ here to allow merely declaring a local variable without supplying an initial value. Q: What about local(2, 4, 5)? A: It should return 5, and introduce no names. I don't see a point to trying to outlaw stupidity ;-) Then again, it would be consistent with the _intent_ to require that all but the last "argument" be of the `name=expression` form. Q: Isn't changing the meaning of scope depending on context wy magical? A: Yup! But in a language with such a strong distinction between statements and expressions, without a bit of deep magic there's no single syntax I can dream up that could work well for both that didn't require _some_ deep magic. The gimmick here is something I expect will be surprising the first time it's seen, less so the second, and then