[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.9 -Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Martin Panter]

Martin Panter added the comment:

I doubt the Gnu Readline library nor Python’s readline module are relevant. The 
input function only uses Readline if sys.stdin is the original stdin terminal; 
I suspect Idle monkey-patches sys.stdin. The readline method reads directly 
from the file object; it never uses the Readline library (despite the 
unfortunate clash in names :).

Louie seems to have added two workarounds: the finish flag, and using 
interrupt_main with a timeout. Both seem racy. What if the flag is altered in 
another thread between when the flag is checked and when it is acted upon? What 
has to happen in the 0.2 s between calling interrupt_main and raising SIGINT?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Antoine Pitrou]

Antoine Pitrou added the comment:

Github links to the aforementioned changesets:

"added PyErr_SetInterrupt(); NT ifdefs"
https://github.com/python/cpython/commit/06d511ddf5fe16468a3abd53344fa283b9981d73

"Add interrupt_main() to thread module."
https://github.com/python/cpython/commit/a11e84613579e2487bcb3967d3a2edbd0665343a

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Antoine Pitrou]

Antoine Pitrou added the comment:

Also note that:

- PyErr_SetInterrupt() has been setting the SIGINT since the beginning in 1995 
(changeset 06d511dd by Guido)
- interrupt_main() has been calling PyErr_SetInterrupt() since the beginning in 
2003 (changeset a11e8461 by Kurt B Kaiser)

so it stands reasonable to keep the SIGINT behaviour.  The documentation is 
simply wrong and/or imprecise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Antoine Pitrou]

Antoine Pitrou added the comment:

Note that, since interrupt_main() claims to interrupt the main thread, it would 
be ok to use pthread_kill() to make sure it's indeed the C syscall running in 
the main thread that gets interruped.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Antoine Pitrou]

Antoine Pitrou added the comment:

Please step back a bit and read the implementation of interrupt_main(): it 
calls PyErr_SetInterrupt() (in signalmodule.c!), which merely sets an internal 
flag saying SIGINT was received.

So, there: PyErr_SetInterrupt() already behaves like SIGINT, *except* that it 
doesn't actually deliver a C signal, it merely sets a flag.  Which is the 
reason that it fails waking up C syscalls like select().

Demonstration:

>>> def handler(signum, frame):
... print("got signal %d!" % (signum,))
... 
>>> signal.signal(signal.SIGINT, handler)

>>> _thread.interrupt_main()
got signal 2!


In the end, making interrupt_main() *actually* deliver a SIGINT instead of 
merely setting the internal flag for it will certainly be true to the original 
intent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Nathaniel Smith]

Nathaniel Smith added the comment:

Then maybe simplest solution is to scale back the claim :-).

The important semantic change would be that right now, interrupt_main() is 
documented to cause KeyboardInterrupt to be raised in the main thread. So if 
you register a custom SIGINT handler that *doesn't* raise KeyboardInterrupt, 
interrupt_main() doesn't care: it doesn't invoke your custom handler, it just 
raises KeyboardInterrupt. My proposal would make it so that interrupt_main() 
does cause custom SIGINT handlers to be called, as well as triggering all of 
the interpreters existing machinery for prompt signal delivery (waking up 
time.sleep etc.), so in that respect it would be much more similar to a "real" 
control-C than it is now. This is a non-trivial semantic change, so it would 
need to be documented, but I'm not attached to the "equivalent to control-C" 
phrasing.

Even on UNIX, a "real" control-C is delivered to all processes in the 
foreground process group (or something like that, ttys are weird), but we would 
want interrupt_main() to only be delivered to the current process, so the 
equivalence wouldn't be exact there either.

> That operation isn't buggy on its own to my knowledge

I spent quite some time trying to make this work in order to test control-C 
handling in trio, and eventually gave up because I just could not make it work 
reliably. One notable effect is that if you call 
GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) in an appveyor test run, then it 
breaks appveyor -- at the end of the run it freezes with "Terminate batch job 
(Y/N)?" and then sits until the test run times out. It's possible to work 
around this with some extremely obscure code (make sure you always spawn a 
subprocess with CREATE_NEW_CONSOLE or maybe CREATE_NEW_CONSOLE_GROUP etc.), but 
(a) we can't tell people that they need to do that before running code that 
uses interrupt_main(), and (b) even when I did that, then I still had bizarre 
issues I couldn't figure out, where sometimes the event just wouldn't be 
delivered unless I ran the interpreter with the -u option for unbuffered stdio 
(?!?!) (see [1]).

I just don't think this is something that CPython can use.

I eventually switched to simulating control-C events for testing by calling 
raise(SIGINT), and that's worked much much better.

> I mentioned the possibility of calling CancelSynchronousIo in order to cancel 
> a console read like Ctrl+C does (but clunkier) -- again, because of the 
> equivalence claim. ERROR_OPERATION_ABORTED would need to be handled like 
> EINTR on Unix. This would entail small changes to a lot of code, so it does 
> need a separate issue if there's any support for this idea.

It still wouldn't be equivalent because control-C only cancels console reads, 
but CancelSynchronousIo would cancel *any* blocking I/O operation that the main 
thread was doing, right? So e.g. if the main thread was blocked doing 
socket.recv, then control-C wouldn't interrupt that, but interrupt_main() / 
PyErr_SetInterrupt() would. I can certainly see an argument that Python's 
C-level signal handler *should* call CancelSynchronousIo in general, and then 
raise(SIGINT) and control-C would be equivalent because they both called 
CancelSynchronousIo, but yeah, that's way beyond the scope of this issue.

[1] 
https://github.com/python-trio/trio/commit/95843654173e3e826c34d70a90b369ba6edf2c23#diff-345cfb6c136028f9514b67ee7bb8e035R11

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Eryk Sun]

Eryk Sun added the comment:

The strong claim that "interrupt_main is now equivalent to the user having hit 
control-C" is the reason I suggested broadcasting CTRL_C_EVENT via 
GenerateConsoleCtrlEvent. That operation isn't buggy on its own to my 
knowledge. There is a glitch due to the CRT. If a non-console process calls 
AllocConsole or AttachConsole, its list of registered handlers gets reset to 
the default handler that calls ExitProcess, and the CRT provides no way to set 
its handler again. 

I mentioned the possibility of calling CancelSynchronousIo in order to cancel a 
console read like Ctrl+C does (but clunkier) -- again, because of the 
equivalence claim. ERROR_OPERATION_ABORTED would need to be handled like EINTR 
on Unix. This would entail small changes to a lot of code, so it does need a 
separate issue if there's any support for this idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Antoine Pitrou]

Antoine Pitrou added the comment:

+1 on having PyErr_SetInterrupt() call raise(SIGINT) or equivalent.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Nathaniel Smith]

Nathaniel Smith added the comment:

> A real Ctrl+C executes the registered control handlers for the process.

Right, but it's *extremely* unusual for a python 3 program to have a control 
handler registered directly via SetConsoleCtrlHandler. This isn't an API that 
the interpreter uses or exposes. The vast majority of programs receive 
control-C notification by letting the C runtime convert the low level console 
event into a SIGINT, and then receiving this via the signal module.

> To emulate this, PyErr_SetInterrupt could try calling 
> GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) to broadcast a Ctrl+C event. 

But as mentioned up thread, this is really flaky - and even when it works it 
tends to kill random other processes, which is *certainly* not what anyone 
expects from calling interrupt_main. You can't really call it experimentally – 
even trying to call it can do stuff like cause appveyor tests to lock up.

Given these two issues, I think that emulating control-C at the signal level 
makes the best tradeoffs. Something that works 100% reliably for 99.99% of 
python programs is way better than something that is 80% reliable for 100% of 
programs.

> One problem is that GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) doesn't cancel 
> a blocking console read like Ctrl+C does. Python's C handler could call 
> CancelSynchronousIo(hMainThread) to address this problem in general. 
> Unfortunately, when a console read is canceled in the client, it isn't 
> canceled in the console itself. The contents of the read will be discarded, 
> but it's a bit clunky that the user has to press enter.

This might be something to address as a separate issue? I'm guessing this 
doesn't affect idle, and it doesn't affect time.sleep, which seem to be the 
main topics of this thread, plus it sounds like any solution would be mostly 
orthogonal.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Terry J. Reedy]

Terry J. Reedy added the comment:

It appears you are right about open('con'...).

However, this issue is about the fact that 
>>> import time; time.sleep(10)
^C
...
KeyboardInterrupt

works in the console but does not work in the IDLE Shell in default mode.  In 
default mode (-n not on the command line) entered code is executed in a 
separate process and and ^C causes a message to be sent to the separate process 
to simulate ^C keypress by calling 'something'.

The indirect process works for 'normal' code like
>>> s = input('Hi')
so input() is not part of this issue, except that it or its interruption should 
not be broken.  Ditto for
>>> while True: pass

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Eryk Sun]

Eryk Sun added the comment:

Terry, I assume you ran IDLE via pyw.exe or pythonw.exe, which won't inherit 
the console of its parent. You have to use py.exe or python.exe. In this case 
you can also use sys.__stdout__.write('spam\n').

If you run via pythonw.exe or pyw.exe from a command prompt, you could use 
ctypes to attach to the console, but it would involve searching back to find 
the cmd.exe or powershell.exe PID to call AttachConsole(pid). More simply you 
can call ctypes.WinDLL('kernel32').AllocConsole() to get a new one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Eryk Sun]

Eryk Sun added the comment:

> interrupt_main is now equivalent to the user having hit control-C. 

That's true on a POSIX system with pthread_kill, but not really for a Windows 
process that's attached to a console.  

A real Ctrl+C executes the registered control handlers for the process. To 
emulate this, PyErr_SetInterrupt could try calling 
GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) to broadcast a Ctrl+C event. If the 
latter fails, it can fall back on raise(SIGINT), such as when the process isn't 
attached to a console.

One problem is that GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) doesn't cancel a 
blocking console read like Ctrl+C does. Python's C handler could call 
CancelSynchronousIo(hMainThread) to address this problem in general. 
Unfortunately, when a console read is canceled in the client, it isn't canceled 
in the console itself. The contents of the read will be discarded, but it's a 
bit clunky that the user has to press enter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Nathaniel Smith]

Nathaniel Smith added the comment:

Sorry, I meant bpo-21895.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Nathaniel Smith]

Nathaniel Smith added the comment:

In terms of general design cleanliness, I think it would be better to make 
`interrupt_main` work reliably than to have IDLE add workarounds for its 
unreliability.

AFAICT the ideal, minimal redundancy solution would be:

- interrupt_main just calls raise(SIGINT)
- bpo-21895 is fixed by adding a few lines to Python's C-level signal handler 
on Unix-likes to check if it's running in the main thread, and if not then use 
pthread_kill to redirect the signal to the main thread.

If trying to fix bpo-28195 is too controversial, then a minimal change would be:

- in interrupt_main, if we're on Windows, call raise(SIGINT); otherwise, call 
pthread_kill(main_thread, SIGINT)

And in either case we'd also have to document that interrupt_main is now 
equivalent to the user having hit control-C. But AFAICT that's what all its 
users really want anyway.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Terry J. Reedy]

Terry J. Reedy added the comment:

On windows, s=input() is cleanly interrupted by ^-C, leaving s unbound, before 
and after the patch.  time.sleep(1) is not interrupted, before and after the 
patch.  Ditto for the socket test.  Louie, what test are you using on *nix?

It still appears that for Windows a patch to signalmodule.c is still needed.  
Eryk and Nathaniel, can you agree on an actual patch?

Eryk: off topic a bit, but
>>> open('con', 'w').write('spam\n')
Traceback (most recent call last):
  File "", line 1, in 
open('con', 'w').write('spam\n')
OSError: [WinError 6] The handle is invalid: 'con'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Louie Lu]

Changes by Louie Lu :


--
assignee:  -> terry.reedy
components: +IDLE -Library (Lib)
title: time.sleep ignores _thread.interrupt_main() -> IDLE: in shell, 
time.sleep ignores _thread.interrupt_main()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com