[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Wed, Nov 18, 2020 at 7:45 PM Brett Cannon wrote: > > > On Tue, Nov 17, 2020 at 10:16 PM Greg Ewing > wrote: > >> On 18/11/20 4:36 pm, Larry Hastings wrote: >> > >> > But, >> > the thinking went, you'd never want to examine the last value from a >> > list generator, so it was more convenient if it behaved as if it had >> its >> > own scope. >> >> List comprehensions used to leak, but apparently that was considered >> surprising by enough people that it was changed. >> >> Generator expressions are a bit different -- the only sane way to >> implement them was to make the body implicitly a separate function, >> and non-leaking behaviour naturally fell out of that. Making them >> leak would have taken extra work just to get something that nobody >> really had a good use case for. >> >> The desire to make list comprehensions and generator expressions >> behave consistently may have contributed to the decision to change >> list comprehensions to be non-leaking. >> > > It did if I remember correctly. > > For me, generator expressions are small functions like you say (see, > people did get the equivalent of multi-line lambdas, just in a very > specific format ), and all the comprehensions are just passing a > generator expression to the appropriate constructor, e.g. list(), set(), > and dict(). In that regard the scoping is consistent to me. > > -Brett > > If match were to create a scope, would making names in that scope nonlocal by default work? (Obvs we'd then need a local declaration, but given global's long-standing presence in the language and the more recent addition of nonlocal it seems like a fairly natural extension). ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BL5BGPFOJ25FVPC7KI4QPEEHUIR5L2RC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Tue, Nov 17, 2020 at 10:16 PM Greg Ewing wrote: > On 18/11/20 4:36 pm, Larry Hastings wrote: > > > > But, > > the thinking went, you'd never want to examine the last value from a > > list generator, so it was more convenient if it behaved as if it had its > > own scope. > > List comprehensions used to leak, but apparently that was considered > surprising by enough people that it was changed. > > Generator expressions are a bit different -- the only sane way to > implement them was to make the body implicitly a separate function, > and non-leaking behaviour naturally fell out of that. Making them > leak would have taken extra work just to get something that nobody > really had a good use case for. > > The desire to make list comprehensions and generator expressions > behave consistently may have contributed to the decision to change > list comprehensions to be non-leaking. > It did if I remember correctly. For me, generator expressions are small functions like you say (see, people did get the equivalent of multi-line lambdas, just in a very specific format ), and all the comprehensions are just passing a generator expression to the appropriate constructor, e.g. list(), set(), and dict(). In that regard the scoping is consistent to me. -Brett > > So yes, it's all a bit messy, for reasons that are partly historical > and partly pragmatic, but things seem to work out okay in practice > most of the time. > > If there's anything I would change, it would be to have the for > statement create a new binding on each iteration, so that capturing > it with a def or lambda inside the loop works as expected. I even > came up with a way to do that while still allowing the last-bound > value to be seen afterwards, but the idea didn't gain any traction. > > -- > Greg > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/FC7WYLN7C54ZK2FRQSLH5GFA4P2OHBBC/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OI2DAQYQT3PWL2K3WSG2TA2U3TN2FYGB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On 18/11/20 4:36 pm, Larry Hastings wrote: But, the thinking went, you'd never want to examine the last value from a list generator, so it was more convenient if it behaved as if it had its own scope. List comprehensions used to leak, but apparently that was considered surprising by enough people that it was changed. Generator expressions are a bit different -- the only sane way to implement them was to make the body implicitly a separate function, and non-leaking behaviour naturally fell out of that. Making them leak would have taken extra work just to get something that nobody really had a good use case for. The desire to make list comprehensions and generator expressions behave consistently may have contributed to the decision to change list comprehensions to be non-leaking. So yes, it's all a bit messy, for reasons that are partly historical and partly pragmatic, but things seem to work out okay in practice most of the time. If there's anything I would change, it would be to have the for statement create a new binding on each iteration, so that capturing it with a def or lambda inside the loop works as expected. I even came up with a way to do that while still allowing the last-bound value to be seen afterwards, but the idea didn't gain any traction. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FC7WYLN7C54ZK2FRQSLH5GFA4P2OHBBC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On 2020-11-18 03:36, Larry Hastings wrote:
[snip]
But then we get to generator expressions and list/dict/set
comprehensions. I think of those as doing an "assignment", but I have
to remember the assignment "doesn't leak":
x = 3
y = list(x**2 for x in range(5))
print(f"{x=}")
This code prints "x=3".
Why this exception to the rule? IIRC from the discussions back when
they were added, this one of those "it just seemed best at the time"
things. "for" was added to the language long before I got there, but it
"leaking" its assignment was judged useful; if you broke out of the
loop, you could continue to examine the last value from the loop. But,
the thinking went, you'd never want to examine the last value from a
list generator, so it was more convenient if it behaved as if it had its
own scope.
[snip]
I think the reason that generator expressions don't leak is that
experience of list comprehensions, which did leak, showed that it was a
bad idea. List comprehensions were "fixed" in Python 3.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/Y7S6VWQOVWICPBDUE7MS7RHGA67AQULO/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
I think Python's rules about what constitutes a new scope are a touch
inconsistent--or, as the Zen might put it, "practical".
First, simple statements never create their own scope. That's easy.
Second, compound statements that create a new function, "def" and
"lambda", always create a new scope. "class" also creates a new scope,
but that's intuitive--and if you understand the language you know that
under the hood it's also creating a code object like "def" and "lambda"
do. Anyway, none of the other "compound" statements create scopes.
That's true even for compound statements that themselves make
assignments, like "for", "with", and "except". But you'll note they
also don't create new code objects. Finally, we note that modules
always have their own scopes. And again, if you understand the
language, you know they're also implemented using code objects.
If that was the whole list, then the mental model would be super simple:
code objects always create their own scopes, and nothing else does.
But then we get to generator expressions and list/dict/set
comprehensions. I think of those as doing an "assignment", but I have
to remember the assignment "doesn't leak":
x = 3
y = list(x**2 for x in range(5))
print(f"{x=}")
This code prints "x=3".
Why this exception to the rule? IIRC from the discussions back when
they were added, this one of those "it just seemed best at the time"
things. "for" was added to the language long before I got there, but it
"leaking" its assignment was judged useful; if you broke out of the
loop, you could continue to examine the last value from the loop. But,
the thinking went, you'd never want to examine the last value from a
list generator, so it was more convenient if it behaved as if it had its
own scope.
//arry/
p.s. Maybe I should be using the word "binding" instead of
"assignment"? If there's an important distinction between the two terms
I don't know it.
On 11/17/20 1:55 AM, Mark Shannon wrote:
Hi,
I'm wondering why
```
x = "value"
try:
1/0
except Exception as x:
pass
```
does not restore "value" to x after
the `except` block.
There doesn't seem to be an explanation for this behavior in the docs
or PEPs, that I can find.
Nor does there seem to be any good technical reason for doing it this
way.
Anyone know why, or is just one of those unintended consequences like
`True == 1`?
Here's an example of restoring the value of the variable after the
`except` block:
>>> def f(x):
... try:
... 1/0
... except Exception as x:
... pass
... return x
...
>>> f("hi")
'hi'
Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/KGYRLITEPB22ZZO4N7DD4A7QP7FQS6JO/
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/AZR3EUV4E5SV5J6GDSI5G5EG56CRM3BM/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Tue, Nov 17, 2020 at 3:57 AM Mark Shannon wrote:
>
>
> On 17/11/2020 10:22 am, Cameron Simpson wrote:
> > On 17Nov2020 09:55, Mark Shannon wrote:
> >> I'm wondering why
> >> ```
> >> x = "value"
> >> try:
> >> 1/0
> >> except Exception as x:
> >> pass
> >> ```
> >>
> >> does not restore "value" to x after
> >> the `except` block.
> >
> > Because the except is not a new scope. So it is the same "x".
> >
> > Here:
> >
> > https://docs.python.org/3/reference/compound_stmts.html#try
> >
> > it says:
> >
> > When an exception has been assigned using as target, it is cleared
> > at the end of the except clause. This is as if
> >
> > except E as N:
> > foo
> >
> > was translated to
> >
> > except E as N:
> > try:
> > foo
> > finally:
> > del N
> >
> > This means the exception must be assigned to a different name to be
> > able to refer to it after the except clause. Exceptions are cleared
> > because with the traceback attached to them, they form a reference
> > cycle with the stack frame, keeping all locals in that frame alive
> > until the next garbage collection occurs.
> >
>
> Sorry, I should have made it clearer.
>
> I'm not asking what are the semantics of the current version of Python.
> I'm asking why they are that way.
>
>
> >> Here's an example of restoring the value of the variable after the
> >> `except` block:
> >>
> > def f(x):
> >> ... try:
> >> ... 1/0
> >> ... except Exception as x:
> >> ... pass
> >> ... return x
> >> ...
> > f("hi")
> >> 'hi'
> >
> > In the Python 3.8.5 I don't see this:
> >
> > Python 3.8.5 (default, Jul 21 2020, 10:48:26)
> > [Clang 11.0.3 (clang-1103.0.32.62)] on darwin
> > Type "help", "copyright", "credits" or "license" for more
> information.
> > >>> def f(x):
> > ... try:
> > ... 1/0
> > ... except Exception as x:
> > ... pass
> > ... return x
> > ...
> > >>> f(3)
> > Traceback (most recent call last):
> >File "", line 1, in
> >File "", line 6, in f
> > UnboundLocalError: local variable 'x' referenced before assignment
> >
> > and the same outside a function.
> >
>
> But why have we chosen for it do this?
>
Because that solution didn't occur to anyone at the time?
Because we didn't realize anyone would want those semantics?
Because the current semantics evolved from previous semantics that didn't
erase the variable at the end of the except block?
Why do you need this question answered?
> Wouldn't restoring the value of x be a superior option?
>
That depends. It would be more work to implement, which (at the time that
feature was designed, probably more so than today) was a concern.
But "superior" is rather subjective.
--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
I think we've fallen in the trap of assuming we all agree on the meaning of
"scope".
In Python, that word is typically used to refer to a bunch of behaviors
that go together. In particular, "the current scope" is where assignment
creates new variables. (It is also linked to the semantics of "return" and
"yield", and to "global" and "nonlocal".) The paper uses it in this sense.
The problem with using scopes *in this sense* for case blocks is that all
assignments to variables inside a case block would create the new variable
in the scope of that case block. But consider this example:
```
match arg:
case Point(x, y):
p = (x, y)
case (x, y):
p = (x, y)
case _:
raise TypeError
print(p)
```
There are no assignments to p outside the case blocks. Giving each case
block a traditional "scope" would prevent the print(p) call outside the
match statement from seeing the variable p. (Please don't argue that this
should be refactored by putting the print call in a helper function.)
Now, of course we could change the language so that this all works. But it
would require a lot of changes to the compiler, since it gives scopes a
bundle of semantics that would have to be separated out.
We could also agree to keep the language the same and use "scope" with a
different meaning (the meaning that you seem to want). But you can't blame
the paper for using it in the way it is used, since that is the
conventional usage of the term for Python.
On Tue, Nov 17, 2020 at 7:03 AM Mark Shannon wrote:
> Hi,
>
> It turns out that implementing the save and restore semantics in the
> example I gave is not that difficult.
>
> I was motivated to find out by the DLS 2020 paper on pattern matching.
> It claims that introducing small scopes for variables would have to be
> implemented as a function preventing the use of normal control flow.
>
> Since restoring the variable after an except block is a similar problem,
> I thought I'd see how difficult it was.
>
> If anyone's interested, here's a prototype:
>
>
> https://github.com/python/cpython/compare/master...markshannon:fix-exception-scoping
>
> (This only saves globals and function-locals, class-locals and
> non-locals are unchanged. I'd probably want to emit a syntax warning for
> non-locals, as the semantics are a bit weird).
>
> Cheers,
> Mark.
>
> On 17/11/2020 9:55 am, Mark Shannon wrote:
> > Hi,
> >
> > I'm wondering why
> > ```
> > x = "value"
> > try:
> > 1/0
> > except Exception as x:
> > pass
> > ```
> >
> > does not restore "value" to x after
> > the `except` block.
> >
> > There doesn't seem to be an explanation for this behavior in the docs or
> > PEPs, that I can find.
> > Nor does there seem to be any good technical reason for doing it this
> way.
> >
> > Anyone know why, or is just one of those unintended consequences like
> > `True == 1`?
> >
> >
> > Here's an example of restoring the value of the variable after the
> > `except` block:
> >
> > >>> def f(x):
> > ... try:
> > ... 1/0
> > ... except Exception as x:
> > ... pass
> > ... return x
> > ...
> > >>> f("hi")
> > 'hi'
> >
> >
> > Cheers,
> > Mark.
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> >
> https://mail.python.org/archives/list/python-dev@python.org/message/KGYRLITEPB22ZZO4N7DD4A7QP7FQS6JO/
> >
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IAY4XHLOOA572INPMP34WYXZPOSORBYU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On 18Nov2020 08:34, Steven D'Aprano wrote:
>As far as shadowing other variables, if someone has so much code in
>their function, or at the top level, that they are at risk of
>inadvertantly shadowing variables, they have far more serious problems
>than the use of the conventional "e for exception".
>
>To start with, what else are they using "e" for? Surely it would be more
>important to use a better name *there* rather than change the exception
>variable.
Aye, that was my thought too. Avoiding a collision is so easy that
adding a magic special scope to Python seems overkill.
I'm also against lots of scopes. They cause pain.
I wrote a post in (I thought) a Python discussion about adding more
scopes citing what I consider a horrible flaw in Go's scope design which
shadows the common "err" return from various calls (Go doesn't use
exceptions), and provided a code example of how easy it was to
accidentally break one's error handling because of that design.
Annoyingly, I cannot find that post now. But basicly the if-statement
introduces a scrope (which starts at the test, before the {...}
brackets) while shadows an earlier variable, and on exiting the scrope
the common "err" variable is False again, indicating no error. Really
irritating.
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/OFIF36EACAW5F4OBY7H6JXFQRVGZKKN6/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Wed, Nov 18, 2020 at 08:45:10AM +1100, Chris Angelico wrote: > On Wed, Nov 18, 2020 at 8:38 AM Steven D'Aprano wrote: > > To start with, what else are they using "e" for? Surely it would be more > > important to use a better name *there* rather than change the exception > > variable. > > 2.718281828? eulers_number = 2.718281828 eulers_constant = 0.57721566490153 https://en.wikipedia.org/wiki/List_of_things_named_after_Leonhard_Euler#Numbers No chance of confusion there :-) -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UD5SJJL6XJ6C4UI4IBV3PCZOOGQGKPHD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Wed, Nov 18, 2020 at 8:38 AM Steven D'Aprano wrote: > To start with, what else are they using "e" for? Surely it would be more > important to use a better name *there* rather than change the exception > variable. 2.718281828? ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CNXCWVJA7C7POXFV7VHNM64O22NQ54KN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Tue, Nov 17, 2020 at 12:36:31PM -0500, Richard Damon wrote: > My main thought on that variable, is I would expect it to have a good > name that implies it is an exception, Something like "e" or "err" then, which are the two most common choices I have seen in exception handling code. > not something like e, There are some very common conventions for single character names, such as i,j,k for loop variables, n for integers, d for dicts, and e for an exception or error is one of them. If you need nested try blocks, and each needs an error variable that have to co-exist, e1, e2, e3 etc are obvious choices. As far as shadowing other variables, if someone has so much code in their function, or at the top level, that they are at risk of inadvertantly shadowing variables, they have far more serious problems than the use of the conventional "e for exception". To start with, what else are they using "e" for? Surely it would be more important to use a better name *there* rather than change the exception variable. -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WXMRVA34CS6FY2WIU5YLGU3YSK5WXVED/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On 11/17/20 12:16 PM, Chris Angelico wrote: > On Wed, Nov 18, 2020 at 4:08 AM Richard Damon > wrote: >> One comment about having the exception handler 'save state' and restore >> it is that frequently the purpose of the exception handler is TO make >> changes to outer variable to fix things up based on the exception. >> >> I could see the desire of a way to create an internal scope of sorts and >> create names limited to that scope, but wouldn't want such a scope being >> anywhere automatic, and likely what would make more sense is defining a >> particular name (or names) to have a limited scope. > The ONLY variable that would be special is the one named in the > "except Exc as e:" clause. Everything else would follow the normal > rules. This is the only variable that is currently special, and ti'd > be the only one that ever needs to be special (since it has the > refloop that makes memory management harder). > > ChrisA My main thought on that variable, is I would expect it to have a good name that implies it is an exception, not something like e, unless you are reserving e for exceptions, so would be unlikely to shadow unless this is from a try block inside an except block that then return to more processing in the outer except block -- Richard Damon ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WKGWMDXXNYHT3JDBMVDLM5KYTMY4SRL7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Wed, Nov 18, 2020 at 4:08 AM Richard Damon wrote: > > One comment about having the exception handler 'save state' and restore > it is that frequently the purpose of the exception handler is TO make > changes to outer variable to fix things up based on the exception. > > I could see the desire of a way to create an internal scope of sorts and > create names limited to that scope, but wouldn't want such a scope being > anywhere automatic, and likely what would make more sense is defining a > particular name (or names) to have a limited scope. The ONLY variable that would be special is the one named in the "except Exc as e:" clause. Everything else would follow the normal rules. This is the only variable that is currently special, and ti'd be the only one that ever needs to be special (since it has the refloop that makes memory management harder). ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6O2YYUF7M5PZNZQX7EVYU4U7KTJ7QUHB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
One comment about having the exception handler 'save state' and restore it is that frequently the purpose of the exception handler is TO make changes to outer variable to fix things up based on the exception. I could see the desire of a way to create an internal scope of sorts and create names limited to that scope, but wouldn't want such a scope being anywhere automatic, and likely what would make more sense is defining a particular name (or names) to have a limited scope. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/A5LY2IPFQ6DPKIYAZAJTX6YNUBAVH2YL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
Hi,
It turns out that implementing the save and restore semantics in the
example I gave is not that difficult.
I was motivated to find out by the DLS 2020 paper on pattern matching.
It claims that introducing small scopes for variables would have to be
implemented as a function preventing the use of normal control flow.
Since restoring the variable after an except block is a similar problem,
I thought I'd see how difficult it was.
If anyone's interested, here's a prototype:
https://github.com/python/cpython/compare/master...markshannon:fix-exception-scoping
(This only saves globals and function-locals, class-locals and
non-locals are unchanged. I'd probably want to emit a syntax warning for
non-locals, as the semantics are a bit weird).
Cheers,
Mark.
On 17/11/2020 9:55 am, Mark Shannon wrote:
Hi,
I'm wondering why
```
x = "value"
try:
1/0
except Exception as x:
pass
```
does not restore "value" to x after
the `except` block.
There doesn't seem to be an explanation for this behavior in the docs or
PEPs, that I can find.
Nor does there seem to be any good technical reason for doing it this way.
Anyone know why, or is just one of those unintended consequences like
`True == 1`?
Here's an example of restoring the value of the variable after the
`except` block:
>>> def f(x):
... try:
... 1/0
... except Exception as x:
... pass
... return x
...
>>> f("hi")
'hi'
Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/KGYRLITEPB22ZZO4N7DD4A7QP7FQS6JO/
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/IAY4XHLOOA572INPMP34WYXZPOSORBYU/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
17.11.20 11:55, Mark Shannon пише: > I'm wondering why > ``` > x = "value" > try: > 1/0 > except Exception as x: > pass > ``` > > does not restore "value" to x after > the `except` block. > > There doesn't seem to be an explanation for this behavior in the docs or > PEPs, that I can find. > Nor does there seem to be any good technical reason for doing it this way. Others already said that it is because "except" does not create a new scope. But why it does not create a new scope? Because it is a design of Python. In general, since in Python local variables do not have declarations, inner scopes do not work. For example: y = 1 if x: y = 2 If "if" create a new scope, "y" after "if" would be restored to 1. But what if make an exception for "except" and make a special rule for it? First, "Special cases aren't special enough to break the rules". Second, how would you use it? This feature would only encourage to write hard-to-read and errorprone code. This error is so common, that in some other programming languages which support inner scopes shadowing local variable causes compiler warning or even syntax error. So this feature is a misfeature. There is also historical reason. In Python 2 the variable was not deleted after leaving the except block. Now it is deleted, and if the code uses the variable after this it would raise a NameError. It is a clear indication of program error. With your proposition it would have some other value, and program error would not be caught so easily. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LMMNJLNWGCWVMDSFFZHSPYT4RFIEO3ZR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On 17/11/2020 10:22 am, Cameron Simpson wrote:
On 17Nov2020 09:55, Mark Shannon wrote:
I'm wondering why
```
x = "value"
try:
1/0
except Exception as x:
pass
```
does not restore "value" to x after
the `except` block.
Because the except is not a new scope. So it is the same "x".
Here:
https://docs.python.org/3/reference/compound_stmts.html#try
it says:
When an exception has been assigned using as target, it is cleared
at the end of the except clause. This is as if
except E as N:
foo
was translated to
except E as N:
try:
foo
finally:
del N
This means the exception must be assigned to a different name to be
able to refer to it after the except clause. Exceptions are cleared
because with the traceback attached to them, they form a reference
cycle with the stack frame, keeping all locals in that frame alive
until the next garbage collection occurs.
Sorry, I should have made it clearer.
I'm not asking what are the semantics of the current version of Python.
I'm asking why they are that way.
Here's an example of restoring the value of the variable after the
`except` block:
def f(x):
... try:
... 1/0
... except Exception as x:
... pass
... return x
...
f("hi")
'hi'
In the Python 3.8.5 I don't see this:
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(x):
... try:
... 1/0
... except Exception as x:
... pass
... return x
...
>>> f(3)
Traceback (most recent call last):
File "", line 1, in
File "", line 6, in f
UnboundLocalError: local variable 'x' referenced before assignment
and the same outside a function.
But why have we chosen for it do this?
Wouldn't restoring the value of x be a superior option?
Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/3JMEHLXJ7ZF2FN5ZFUIDMZHODNJYTE6A/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
Hi Chris, On 17/11/2020 11:03 am, Chris Angelico wrote: On Tue, Nov 17, 2020 at 9:44 PM Steven D'Aprano wrote: `try...except` is no different. ... The only wrinkle in the case of `try...except` is that the error variable is deleted, even if it wasn't actually used. If you look at the byte-code generated, each compound try...except with an exception variable is followed by the equivalent of: err = None del err There really ought to be a FAQ about this, but it has something to do with the exception object forming a long-lasting reference cycle. To avoid that, the error variable is nuked on leaving the compound block. That's a much bigger wrinkle than it might seem at first, though, and I agree, this is a quite literal frequently-asked-question and should be made clear somewhere. The except clause is special in that, if you want the exception afterwards, you have to reassign it to another variable; but it doesn't ACTUALLY introduce a subscope, despite kinda looking like it does. Interestingly, Python 3.10 has a very odd disassembly: def f(): ... try: g() ... except Exception as e: ... print(e) ... import dis dis.dis(f) 2 0 SETUP_FINALLY 10 (to 12) 2 LOAD_GLOBAL 0 (g) 4 CALL_FUNCTION0 6 POP_TOP 8 POP_BLOCK 10 JUMP_FORWARD44 (to 56) 3 >> 12 DUP_TOP 14 LOAD_GLOBAL 1 (Exception) 16 JUMP_IF_NOT_EXC_MATCH54 18 POP_TOP 20 STORE_FAST 0 (e) 22 POP_TOP 24 SETUP_FINALLY 20 (to 46) 4 26 LOAD_GLOBAL 2 (print) 28 LOAD_FAST0 (e) 30 CALL_FUNCTION1 32 POP_TOP 34 POP_BLOCK 36 POP_EXCEPT 38 LOAD_CONST 0 (None) 40 STORE_FAST 0 (e) 42 DELETE_FAST 0 (e) 44 JUMP_FORWARD10 (to 56) >> 46 LOAD_CONST 0 (None) 48 STORE_FAST 0 (e) 50 DELETE_FAST 0 (e) 52 RERAISE >> 54 RERAISE >> 56 LOAD_CONST 0 (None) 58 RETURN_VALUE Reconstructing approximately equivalent Python code, this would mean it looks something like this: def f(): try: g() except Exception as e: try: print(e) e = None del e raise finally: e = None del e except: raise return None The equivalent Python is closer to this: def f(): try: g() except Exception as e: try: print(e) finally: e = None del e I don't understand why (a) the "e = None; del e" part is duplicated, nor (b) why the RERAISE opcodes are there in two branches, but I guess it works out best to be explicit in there? The reason for the seeming verbosity of the bytecode is that try: body finally: final compiles to roughly: try: body: except: final raise else: final Which is why you see the duplicated sequences. Cheers, Mark. Anyhow. You say that this can't come up very often because people can go a long time without asking, but the trouble is that there are two false interpretations that are both extremely close - either that "except E as e:" is similar to "with E as e:", or that the except clause creates its own scope. It's entirely possible to see supporting evidence for your own wrong assumption and never actually know the truth. Maybe this is going to be the next "Python has call-by-value" vs "Python has call-by-reference" debate? ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6NKGXWLRX3SD4JQDFCOR43TAXREC33GD/ Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FNRZWHGCXOYL7QL5QUDR5NJS76RRTY3H/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Tue, Nov 17, 2020 at 9:44 PM Steven D'Aprano wrote: > `try...except` is no different. > ... > The only wrinkle in the case of `try...except` is that the error > variable is deleted, even if it wasn't actually used. If you look at the > byte-code generated, each compound try...except with an exception > variable is followed by the equivalent of: > > err = None > del err > > > There really ought to be a FAQ about this, but it has something to do > with the exception object forming a long-lasting reference cycle. To > avoid that, the error variable is nuked on leaving the compound block. That's a much bigger wrinkle than it might seem at first, though, and I agree, this is a quite literal frequently-asked-question and should be made clear somewhere. The except clause is special in that, if you want the exception afterwards, you have to reassign it to another variable; but it doesn't ACTUALLY introduce a subscope, despite kinda looking like it does. Interestingly, Python 3.10 has a very odd disassembly: >>> def f(): ... try: g() ... except Exception as e: ... print(e) ... >>> import dis >>> dis.dis(f) 2 0 SETUP_FINALLY 10 (to 12) 2 LOAD_GLOBAL 0 (g) 4 CALL_FUNCTION0 6 POP_TOP 8 POP_BLOCK 10 JUMP_FORWARD44 (to 56) 3 >> 12 DUP_TOP 14 LOAD_GLOBAL 1 (Exception) 16 JUMP_IF_NOT_EXC_MATCH54 18 POP_TOP 20 STORE_FAST 0 (e) 22 POP_TOP 24 SETUP_FINALLY 20 (to 46) 4 26 LOAD_GLOBAL 2 (print) 28 LOAD_FAST0 (e) 30 CALL_FUNCTION1 32 POP_TOP 34 POP_BLOCK 36 POP_EXCEPT 38 LOAD_CONST 0 (None) 40 STORE_FAST 0 (e) 42 DELETE_FAST 0 (e) 44 JUMP_FORWARD10 (to 56) >> 46 LOAD_CONST 0 (None) 48 STORE_FAST 0 (e) 50 DELETE_FAST 0 (e) 52 RERAISE >> 54 RERAISE >> 56 LOAD_CONST 0 (None) 58 RETURN_VALUE >>> Reconstructing approximately equivalent Python code, this would mean it looks something like this: def f(): try: g() except Exception as e: try: print(e) e = None del e raise finally: e = None del e except: raise return None I don't understand why (a) the "e = None; del e" part is duplicated, nor (b) why the RERAISE opcodes are there in two branches, but I guess it works out best to be explicit in there? Anyhow. You say that this can't come up very often because people can go a long time without asking, but the trouble is that there are two false interpretations that are both extremely close - either that "except E as e:" is similar to "with E as e:", or that the except clause creates its own scope. It's entirely possible to see supporting evidence for your own wrong assumption and never actually know the truth. Maybe this is going to be the next "Python has call-by-value" vs "Python has call-by-reference" debate? ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6NKGXWLRX3SD4JQDFCOR43TAXREC33GD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On Tue, Nov 17, 2020 at 09:55:46AM +, Mark Shannon wrote:
> Hi,
>
> I'm wondering why
> ```
> x = "value"
> try:
> 1/0
> except Exception as x:
> pass
> ```
>
> does not restore "value" to x after
> the `except` block.
Because try, if, for, while, with etc. don't create a new scope. Only
two statements create a new scope: `def` and `class`.
I presume you don't also expect these to restore x after the
blocks are ended:
x = "value"
for x in range(100):
pass
with open('filename') as x:
text = x.read()
`try...except` is no different.
If you search the archives on Python-List, going back two or four years,
you will find extensive discussion between those who think that every
statement with an indent should create a new scope, and those who don't.
Apologies in advance to supporters of the "new scope" philosophy who
might feel I am giving the idea short shrift, but my recollection is
that they were unable to give any better justification than "but that's
what C does" and "well, it might be useful someday to use a name
inside a block without clobbering its existing value".
(My response to that is, just use a new variable name, it's not like
there is a shortage.)
The only wrinkle in the case of `try...except` is that the error
variable is deleted, even if it wasn't actually used. If you look at the
byte-code generated, each compound try...except with an exception
variable is followed by the equivalent of:
err = None
del err
There really ought to be a FAQ about this, but it has something to do
with the exception object forming a long-lasting reference cycle. To
avoid that, the error variable is nuked on leaving the compound block.
This is documented here:
https://docs.python.org/3/reference/compound_stmts.html#the-try-statement
although only briefly and without much detail.
> Nor does there seem to be any good technical reason for doing it this
> way.
I expect the reason is the usual cost-benefit of "good enough". The
effort involved in storing the old name binding and then restoring it
afterwards is more than the benefit gained (almost zero).
This has been part of Python since Python 3.0, so if you are only just
noticing it, that gives an idea of how rarely it comes up.
> Here's an example of restoring the value of the variable after the
> `except` block:
>
> >>> def f(x):
> ... try:
> ... 1/0
> ... except Exception as x:
> ... pass
> ... return x
> ...
> >>> f("hi")
> 'hi'
What Python interpreter are you using for that? This is not what happens
in Python 3:
# raises an exception
UnboundLocalError: local variable 'x' referenced before assignment
or Python 2.7:
# returns the exception object
ZeroDivisionError('integer division or modulo by zero',)
So unless you are using some modified or alternative interpreter, I
don't know how you got that output.
--
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/TIXX4P4KOA2OO7JTU3BCLNIGLIWC66M2/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?
On 17Nov2020 09:55, Mark Shannon wrote:
>I'm wondering why
>```
>x = "value"
>try:
>1/0
>except Exception as x:
>pass
>```
>
>does not restore "value" to x after
>the `except` block.
Because the except is not a new scope. So it is the same "x".
Here:
https://docs.python.org/3/reference/compound_stmts.html#try
it says:
When an exception has been assigned using as target, it is cleared
at the end of the except clause. This is as if
except E as N:
foo
was translated to
except E as N:
try:
foo
finally:
del N
This means the exception must be assigned to a different name to be
able to refer to it after the except clause. Exceptions are cleared
because with the traceback attached to them, they form a reference
cycle with the stack frame, keeping all locals in that frame alive
until the next garbage collection occurs.
>Here's an example of restoring the value of the variable after the
>`except` block:
>
def f(x):
>... try:
>... 1/0
>... except Exception as x:
>... pass
>... return x
>...
f("hi")
>'hi'
In the Python 3.8.5 I don't see this:
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(x):
... try:
... 1/0
... except Exception as x:
... pass
... return x
...
>>> f(3)
Traceback (most recent call last):
File "", line 1, in
File "", line 6, in f
UnboundLocalError: local variable 'x' referenced before assignment
and the same outside a function.
Cheers,
Cameron Simpson
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/CTCU2D2EAEVCHNNZK2LROZM5J4IW76JF/
Code of Conduct: http://python.org/psf/codeofconduct/
