[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[STINNER Victor]

STINNER Victor added the comment:

Yury Selivanov added the comment:
> I don't think we need to. Isn't 3.5 is in security/important bug fix mode? I 
> don't view this change as an important one (it's just a nice thing to have).

Not yet, it still accept bug fixes:
https://docs.python.org/devguide/#status-of-python-branches

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Yury Selivanov]

Yury Selivanov added the comment:

I don't think we need to. Isn't 3.5 is in security/important bug fix mode? I 
don't view this change as an important one (it's just a nice thing to have).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[STINNER Victor]

STINNER Victor added the comment:

Why not backporting the fix to 3.5 as well?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Yury Selivanov]

Changes by Yury Selivanov :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Yury Selivanov]

Yury Selivanov added the comment:


New changeset e89f95bfd0881a9b80c3b1430d154a77bdf5a824 by Yury Selivanov in 
branch '3.6':
[3.6] bpo-30039: Don't run signal handlers while resuming a yield from stack 
(GH-1081) (#1640)
https://github.com/python/cpython/commit/e89f95bfd0881a9b80c3b1430d154a77bdf5a824


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Yury Selivanov]

Changes by Yury Selivanov :


--
pull_requests: +1735

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Yury Selivanov]

Yury Selivanov added the comment:

Yes, I'll do the backport.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[STINNER Victor]

STINNER Victor added the comment:

The change should be backported to 3.5 and 3.6, right? The change seems very 
short and safe. IMHO it's ok to backport.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Yury Selivanov]

Yury Selivanov added the comment:


New changeset ab4413a7e9bda95b6fcd517073e2a51dafaa1624 by Yury Selivanov 
(Nathaniel J. Smith) in branch 'master':
bpo-30039: Don't run signal handlers while resuming a yield from stack (#1081)
https://github.com/python/cpython/commit/ab4413a7e9bda95b6fcd517073e2a51dafaa1624


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
assignee:  -> yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Nathaniel Smith]

Changes by Nathaniel Smith :


--
pull_requests: +1224

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Nathaniel Smith]

New submission from Nathaniel Smith:

If we have a chain of generators/coroutines that are 'yield from'ing each 
other, then resuming the stack works like:

- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the YIELD_FROM opcode
- which calls send() on the next generator
- which enters _PyEval_EvalFrameDefault, which re-executes the YIELD_FROM opcode
- ...etc.

However, every time we enter _PyEval_EvalFrameDefault, the first thing we do is 
to check for pending signals, and if there are any then we run the signal 
handler. And if it raises an exception, then we immediately propagate that 
exception *instead* of starting to execute bytecode. This means that e.g. a 
SIGINT at the wrong moment can "break the chain" – it can be raised in the 
middle of our yield from chain, with the bottom part of the stack abandoned for 
the garbage collector.

The fix is pretty simple: there's already a special case in _PyEval_EvalFrameEx 
where it skips running signal handlers if the next opcode is SETUP_FINALLY. (I 
don't see how this accomplishes anything useful, but that's another story.) If 
we extend this check to also skip running signal handlers when the next opcode 
is YIELD_FROM, then that closes the hole – now the exception can only be raised 
at the innermost stack frame.

This shouldn't have any performance implications, because the opcode check 
happens inside the "slow path" after we've already determined that there's a 
pending signal or something similar for us to process; the vast majority of the 
time this isn't true.

I'll post a PR in a few minutes that has a test case that demonstrates the 
problem and fails on current master, plus the fix.

--
components: Interpreter Core
messages: 291469
nosy: njs, yselivanov
priority: normal
severity: normal
status: open
title: Resuming a 'yield from' stack is broken if a signal arrives in the middle
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com