[issue9504] signal.signal/signal.alarm not working as expected

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> out of date
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[STINNER Victor]

STINNER Victor  added the comment:

https://www.python.org/dev/peps/pep-0475/ changed deeply how Python handles 
signals. Python now tries to restart syscalls interrupted by signals (EINTR). A 
Python function only raises an exception if the Python signal handler raises an 
exception.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Irit Katriel]

Irit Katriel  added the comment:

I'm unable to reproduce this on 3.11 on a Mac. Has it been fixed?

--
nosy: +iritkatriel
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Ronald Oussoren]

Ronald Oussoren added the comment:

Victor: read(2) and recv(2) are defined as returning all available data when 
interrupted by a signal and might therefore not return an error. The same is 
true for write(2): it returns the number of bytes written when there are any 
bytes written (instead of setting errno to SIGINT and returning an error).

I'd expect the C implementation of readall to behave the same as the Python 
implementation: a signal handler that raises an exception can cause data loss 
due to discarding the buffer that's used to collect the data.  The data loss in 
the script isn't worse than it currently is, AFAIK it would currently lose data 
due to raising the exception just after the call to readall in the next loop 
through the eval loop.

If I read the issue correctly the signal handling issue mentioned here is 
basically similar to the one in this scriptlet:

# BEGIN
import signal, sys
import subprocess

def handler(sig, frames):
raise RuntimeError("Alarm!")

signal.signal(signal.SIGALRM, handler)
signal.alarm(5)
b = sys.stdin.buffer.raw.readall()
print(len(b))
signal.alarm(0)
#END

When you run this with a large file on stdin (I've used /dev/zero) the script 
will continue to run indefinitely and cannot be cleanly interrupted at all (on 
my system its eventually killed by the system due to running out of its rlimit 
limitations, otherwise it can only be killed by sending it an uncatchable 
signal like SIGKILL or SIGABRT).

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[STINNER Victor]

STINNER Victor added the comment:

I don't understand this issue.

I would expect that a system call fails with EINTR if it is interrupted by a 
signal. Python explicitly configures the C library to get EINTR errors: 
automatic retry is disabled when Python sets its signal handlers.

If a system call doesn't fail, what shoud we do with the result, especially 
when reading data (read/recv)?

Yes, we _can_ checks to check regulary if we got a C signal by adding checks, 
but it's unclear to me if we must raise the exception immediatly, or later. 
Immedialty if I understand correctly the initial message. So should we check 
*all* system calls, or only system calls well known to block on I/O, like 
read/write, send/recv, select, etc.?

Do we try to workaround design choices by the Linux kernel and the GNU libc?

Right now, my favorite choice would be to do nothing. Yes, sometimes signals 
are delayed. Sometimes, a signal doesn't interrupt immediatly a blocked 
program. It's not perfect. But it's already very complex, I'm not sure that I 
want to make the code even more complex to check for signals in more places.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Martin Panter]

Martin Panter added the comment:

Okay, I tend to agree, and don’t have a problem with rejecting this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Martin Panter]

Martin Panter added the comment:

With Python’s current technique for signal handling, I think there is always 
going to be a window where a signal can arrive at the low level after checking 
the Python flag and before starting a blocking call. If you need to robustly 
handle a signal maybe it is better to use set_signal_fd() and select() or 
something.

I haven’t looked at the patch closely, but it sounds like it could reduce the 
size of the window. This would make interactive signals like SIGINT more 
reliable, so it may be worthwhile. It probably needs properly reviewing in 
light of the recent EINTR (PEP 457) changes though.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Martin Panter]

Martin Panter added the comment:

Sorry, EINTR changes were PEP 475.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
assignee: pitrou - 
stage: patch review - needs patch
versions: +Python 3.3 -Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Nir Aides]

Changes by Nir Aides n...@winpdb.org:


--
nosy: +nirai

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

It seems that my patch for issue10956 also fixes this issue (although it 
doesn't try to fix FileIO). It also has tests and fixes read() as well as 
write().
I'm sorry for not noticing that the two issues were so similar. Suggest closing 
this issue as duplicate.

--
resolution:  - duplicate
superseder:  - file.write and file.read don't handle EINTR

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Sorry, bad analysis. The patch in issue10956 only seemed to work because my 
system is too fast and find / in the script returned in less than 5 seconds. 
Replacing it with yes shows that the two issued are actually independent.

--
resolution: duplicate - 
stage:  - patch review
superseder: file.write and file.read don't handle EINTR - 
versions: +Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Charles-François' analysis seems to be right. Note that the actual issue here 
is that read() always succeeds, returning a partial result (because you're 
executing a command, 'find /', which outputs a lot of data). If read() were 
interrupted before anything could be read, it would return EINTR and the 
handler would get executed immediately.

Anyone wants to propose a patch + tests?

--
versions: +Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Charles-Francois Natali]

Charles-Francois Natali neolo...@free.fr added the comment:

 Antoine Pitrou pit...@free.fr added the comment:

 Charles-François' analysis seems to be right. Note that the actual issue 
 here is that read() always succeeds, returning a partial result (because 
 you're executing a command, 'find /', which outputs a lot of data). If read() 
 were interrupted before anything could be read, it would return EINTR and the 
 handler would get executed immediately.

 Anyone wants to propose a patch + tests?


Attached is a tentative patch: it checks for pending signals inside
_bufferedreader_read_generic, fileio_readall, and rawiobase_readall.
I tested quickly on regular files and on pipes, and handlers are now
called on time. As a an obvious bonus, it's also easier to interrupt a
script doing I/O with a KeyboardInterrupt (SIGINT).
If nothing seems blatantly wrong about this patch, I will go ahead and
try to write some tests for this.

Charles

 --
 versions: +Python 3.2

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue9504
 ___


--
keywords: +patch
Added file: http://bugs.python.org/file20327/io_signal.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___diff -urp A/Modules/_io/bufferedio.c B/Modules/_io/bufferedio.c
--- A/Modules/_io/bufferedio.c  2010-12-21 22:26:09.0 +0100
+++ B/Modules/_io/bufferedio.c  2011-01-09 18:46:01.0 +0100
@@ -1405,6 +1405,10 @@ _bufferedreader_read_generic(buffered *s
 }
 remaining -= r;
 written += r;
+/* check pending signals */
+if (PyErr_CheckSignals()  0) {
+goto error;
+}
 }
 assert(remaining = self-buffer_size);
 self-pos = 0;
@@ -1442,6 +1446,10 @@ _bufferedreader_read_generic(buffered *s
 }
 if (remaining == 0)
 break;
+/* check pending signals */
+if (PyErr_CheckSignals()  0) {
+goto error;
+}
 }
 
 return res;
diff -urp A/Modules/_io/fileio.c B/Modules/_io/fileio.c
--- A/Modules/_io/fileio.c  2010-11-26 09:52:36.0 +0100
+++ B/Modules/_io/fileio.c  2011-01-09 19:26:52.0 +0100
@@ -583,6 +583,11 @@ fileio_readall(fileio *self)
 return NULL;
 }
 total += n;
+/* check pending signals */
+if (PyErr_CheckSignals()  0) {
+Py_DECREF(result);
+return NULL;
+}
 }
 
 if (PyBytes_GET_SIZE(result)  total) {
diff -urp A/Modules/_io/iobase.c B/Modules/_io/iobase.c
--- A/Modules/_io/iobase.c  2010-09-14 20:53:07.0 +0200
+++ B/Modules/_io/iobase.c  2011-01-09 19:27:15.0 +0100
@@ -831,6 +831,11 @@ rawiobase_readall(PyObject *self, PyObje
 Py_DECREF(chunks);
 return NULL;
 }
+/* check pending signals */
+if (PyErr_CheckSignals()  0) {
+Py_DECREF(chunks);
+return NULL;
+}
 }
 result = _PyBytes_Join(_PyIO_empty_bytes, chunks);
 Py_DECREF(chunks);
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 Attached is a tentative patch: it checks for pending signals inside
 _bufferedreader_read_generic, fileio_readall, and rawiobase_readall.
 I tested quickly on regular files and on pipes, and handlers are now
 called on time. As a an obvious bonus, it's also easier to interrupt a
 script doing I/O with a KeyboardInterrupt (SIGINT).
 If nothing seems blatantly wrong about this patch, I will go ahead and
 try to write some tests for this.

A potential downside is that the bytes which have been read simply get
lost. But since a signal can happen at any moment I guess nothing can be
done to protect from it in the general case, anyway.

As for the tests, the SignalsTest class in test_io.py is a natural
recipient.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Charles-Francois Natali]

Charles-Francois Natali neolo...@free.fr added the comment:

It's due to the way the python interpreter handles signals: when the signal is 
received, python runs a stub signal handler that just sets a flag indicating 
that the signal has been received: the actual handler is executed later, 
synchronously, mainly from the main evaluation loop.
The problem here is that since the main process is performing a read until EOF 
is reached (i.e. until the subprocess exits), the C code loops around the 
read(2) call without giving a chance for the handler to run.
A patch for is similar issue has been applied, see 
http://bugs.python.org/issue9617#

But it only fixed writes, not reads.
I think _bufferedreader_read_all and _bufferedreader_read_generic should be 
fixed too to call PyErr_CheckSignals().

--
nosy: +neologix

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Georg Brandl]

Georg Brandl ge...@python.org added the comment:

Antoine?

--
assignee:  - pitrou
nosy: +georg.brandl

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9504] signal.signal/signal.alarm not working as expected

[Alan Wilter]

New submission from Alan Wilter alanwil...@gmail.com:

I have this example code to illustrate a problem I am having with python3. It 
works fine with python 2.6 and 2.7 but does not with python 3.1.

from __future__ import print_function
import os, subprocess, signal

def signal_handler( signum, frame ):

print( PID: %s % pid )
print( Timed out! Process %s killed, max exec time (%ss) exceeded % (pid, 
timeTol ) )
os.kill( int( pid ), 15 )
raise Exception( Taking too long to finish... aborting! )

if __name__ == '__main__':

timeTol = 5

cmd = 'find /'

signal.signal(signal.SIGALRM, signal_handler)
signal.alarm(timeTol)

p = subprocess.Popen(cmd, shell=True, stderr = subprocess.STDOUT, stdout = 
subprocess.PIPE)
pid = p.pid

out = str( p.communicate()[0].decode() )
print(out)


Executing it:

time python3.1 timout.py
PID: 27687
Timed out! Process 27687 killed, max exec time (5s) exceeded
Traceback (most recent call last):
  File timout.py, line 23, in module
out = str( p.communicate()[0].decode() )
  File /sw/lib/python3.1/subprocess.py, line 719, in communicate
stdout = self.stdout.read()
  File timout.py, line 9, in signal_handler
raise Exception( Taking too long to finish... aborting! )
Exception: Taking too long to finish... aborting!
python3.1 timout.py  0.52s user 3.88s system 19% cpu 22.841 total

 It prints essentially the same thing with a *very* *big* difference it 
takes 22 seconds and actually the alarm only works when the whole task ('find 
/') is finished.

--
messages: 112771
nosy: alanwilter
priority: normal
severity: normal
status: open
title: signal.signal/signal.alarm not working as expected
type: behavior
versions: Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9504
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com