[issue42725] PEP 563: Should the behavior change for yield/yield from's

Change by Batuhan Taskaya : -- stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___ Python-bugs-list

[issue42725] PEP 563: Should the behavior change for yield/yield from's

miss-islington added the comment: New changeset 90d584a2ab91cc0f30e90a0a99f8c1447eb51302 by Miss Islington (bot) in branch '3.10': bpo-42725: mention the changes on what's new (GH-25974) https://github.com/python/cpython/commit/90d584a2ab91cc0f30e90a0a99f8c1447eb51302 --

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Change by miss-islington : -- nosy: +miss-islington nosy_count: 8.0 -> 9.0 pull_requests: +24640 pull_request: https://github.com/python/cpython/pull/25988 ___ Python tracker

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Change by Batuhan Taskaya : -- pull_requests: +24630 pull_request: https://github.com/python/cpython/pull/25974 ___ Python tracker ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: > Batuhan, can this issue be closed? No, I still need to take care of what's new entry. -- ___ Python tracker ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Pablo Galindo Salgado added the comment: Batuhan, can this issue be closed? -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: New changeset ad106c68eb00f5e4af2f937107baff6141948cee by Batuhan Taskaya in branch 'master': bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583) https://github.com/python/cpython/commit/ad106c68eb00f5e4af2f937107baff6141948cee

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Change by Batuhan Taskaya : -- keywords: +patch pull_requests: +24303 stage: -> patch review pull_request: https://github.com/python/cpython/pull/25583 ___ Python tracker ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Pablo Galindo Salgado added the comment: > I just thought having a whole “expr-without-walrus-or-yield” subgrammar > would be tedious. Yeah I have experimented with something similar in the past and this becomes out of hand super quickly and leafs to a lot of duplication. Similarly, a flag

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Serhiy Storchaka added the comment: There is no “expr-without-await-or-async-for” subgrammar, but "await" and asynchronous comprehensions are invalid in synchronous functions. There should be similar flag for annotations in symtable.c. -- ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: I don’t really care how you do it, I just thought having a whole “expr-without-walrus-or-yield” subgrammar would be tedious. If there is a way to have the “in-an-annotation” flag set during parsing that may be better, I don’t know. Maybe Lysandros has an

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: > That can be done after the PEG grammar has accepted them But wouldn't we still end up with maintaining a custom flag to see if we are in annotation (e.g a: Something((yield) + 2)) and act upon that which would seem to do a bit messy in the grammar

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: So shouldn't we just rule out some specific bits of syntax in annotations? That can be done after the PEG grammar has accepted them, otherwise we'd end up having a whole new expression-like grammar section that excludes them. I think this is the list we

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Larry Hastings added the comment: Wouldn't it be easier to just throw an exception in the stringizing code? I note that there are dedicated functions to generate walrus, yield, yield from, and await in Python/ast_unparse.c. In theory this is a general-purpose facility, but in practice

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: > This was easy in my 649 branch; walrus adds locals, and yield / yield from > make it a generator. So the code raises an error if the generated > annotations code object has locals or is a generator. I don't think I had to > do anything special to

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Larry Hastings added the comment: I think stringized annotations should prohibit the same things PEP 649 prohibits: walrus, yield / yield from, and await. This was easy in my 649 branch; walrus adds locals, and yield / yield from make it a generator. So the code raises an error if the

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: I've posted an entry on python-dev to collect comments about how we should act (whether just act them as strings on the symbol table or just forbid them completely):

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: I'm not keen on prohibiting nonlocal variable reference in annotations, since I can imagine uses for this (e.g. in tests). I'm not too worried about keeping the cell alive "forever", since with Larry's PEP 649 their lifetime would end when the object

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: > The use of nonlocal variables in an annotation will be a syntax error in > Python 3.10 What is the reasoning for forbidding nonlocal variables (https://bugs.python.org/msg383659), can't we just treat them like regular variables and leave the value to

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: It'll probably be quicker if you leave me out of the loop (feel free to quote me though). I am currently quite overwhelmed with PEP-sized discussions. I expect that the SC can rule on this quickly (just use their tracker to send them the PEP). --

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Mark Shannon added the comment: Draft PEP here https://github.com/markshannon/peps/blob/annotation-syntax-errors/pep-.rst Guido, would you like to be co-author as it was your idea to make these things a syntax error? -- ___ Python tracker

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: Oh, you’re right about walrus. And I don’t actually understand async generator expressions. This suggests that we definitely need a PEP. :-) -- ___ Python tracker

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Serhiy Storchaka added the comment: Does not walrus affect the meaning of variable? And await affects the meaning of generator expression. -- ___ Python tracker ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: I wouldn’t have thought you’d need a PEP for this but if you want to write one that sounds like the right thing. Re: async and walrus: I think those are different, their presence doesn’t affect the meaning of the function like yield. We can’t hope to

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Serhiy Storchaka added the comment: Should not "await" and "async for" (in comprehesions) and ":=" be forbidden too? -- nosy: +serhiy.storchaka ___ Python tracker ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Mark Shannon added the comment: What's the process for making a decision on whether to make 'yield' in an annotation a syntax error? As a language change it should have a PEP, IMO. The PEP will be short, and shouldn't need a long-winded acceptance process. I just think that a PEP is more

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: Yield in an annotation should be a syntax error. -- ___ Python tracker ___ ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Mark Shannon added the comment: I've also opened #42837 which is about fixing the symbol table, so that it is correct w.r.t. to current behavior. I'd like to fix it ASAP as the compiler should be able to rely on the symbol table being correct. Of course, once we have decided what the

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: One thing to note here, currently Pablo and I are trying to bring annotation unparsing from the compiler to the parser in issue 41967. If we do so, the annotations won't cause any side effects on the symbol table generation. -- nosy: +pablogsal

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: I have a patch ready to go, but I'd prefer to block this issue until issue 42737 is resolved/decided. -- ___ Python tracker ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: Okay, let's do it. It should probably a post-parse check (before we invoke the bytecode compiler but after we have the AST in hand). -- ___ Python tracker

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Lysandros Nikolaou added the comment: I concur with Guido. I feel that making this an error is the best alternative we have. -- ___ Python tracker ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: If we simply ignored yield in the annotation, wouldn't we have the problem that def f(a: (yield)): pass silently changes from being a generator (in 3.9) to not being a generator (in 3.10)? That would be bad. I'd rather make this an error still. (But for

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: > So I don't feel as strongly about that. Just to note, since I believe the solution for all this might be the same (not processing annotations at all, since they will be compiled to strings in the later stage). If we go down on that route, it will be

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: The difference is that that just causes a slight inefficiency, while processing 'yield' makes the function a generator. So I don't feel as strongly about that. -- ___ Python tracker

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Batuhan Taskaya added the comment: This is another side effect of processing annotations (at the symbol table construction stage) (and I would assume there are a few more cases like this); def foo(): outer_var = 1 def bar(): inner_var: outer_var = T return bar

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Change by Guido van Rossum : -- nosy: +lys.nikolaou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue42725] PEP 563: Should the behavior change for yield/yield from's

Guido van Rossum added the comment: Ouch. I think we should generate a SyntaxError if yield [from] is used in an annotation. -- ___ Python tracker ___

[issue42725] PEP 563: Should the behavior change for yield/yield from's

New submission from Batuhan Taskaya : Since the annotations are processed just like all other expressions in the symbol table, the generated entries for functions etc. This would result with def foo(): for number in range(5): foo: (yield number) return number foo()