[issue5315] signal handler never gets called

[Patrick Fink]

Patrick Fink  added the comment:

A workaround to handle signals reliably that I successfully tested now is to 
execute everything within a subthread and let the main thread just join this 
subthread. Like:

signal.signal(MY_SIGNAL, signal_handler)
threading.Thread(target = my_main_function)
thread.start()
thread.join()

Doing it like this, the main thread should always listen to signals 
disregarding whether the subthread is stuck.

--
nosy: +Patrick Fink

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Terry J. Reedy]

Terry J. Reedy added the comment:

This was turned into a doc issue, with no patch forthcoming, but Devin has 
submitted a bugfix.  Should this be turned back into a bug issue?

--
nosy: +terry.reedy
stage:  - patch review
versions: +Python 3.4, Python 3.5, Python 3.6 -Python 2.6, Python 3.1, Python 
3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Devin Jeanpierre]

Devin Jeanpierre added the comment:

Adding haypo since apparently he's been touching signals stuff a lot lately, 
maybe has some useful thoughts / review? :)

--
nosy: +haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Devin Jeanpierre]

Devin Jeanpierre added the comment:

Agree with Charles-François's second explanation. This makes it very hard to 
reliably handle signals -- basically everyone has to remember to use 
set_wakeup_fd, and most people don't. For example, gunicorn is likely 
vulnerable to this because it doesn't use set_wakeup_fd. I suspect most code 
using select + signals is wrong.

I've attached a patch which fixes the issue for select(), but not any other 
functions. If it's considered a good patch, I can work on the rest of the 
functions in the select module. (Also, tests for the details of the behavior.)

Also the patch is pretty hokey, so I'd appreciate feedback if it's going to go 
in. :)

--
keywords: +patch
nosy: +Devin Jeanpierre
Added file: http://bugs.python.org/file39489/select_select.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[s7v7nislands]

Changes by s7v7nislands s7v7nisla...@gmail.com:


--
nosy: +s7v7nislands

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Ernesto Menéndez]

Changes by Ernesto Menéndez pya...@gmail.com:


--
nosy: +Netto

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Charles-Francois Natali]

Charles-Francois Natali neolo...@free.fr added the comment:

I think two things can trigger this problem, both have to do with how signals 
are handled by the interpreter.
Contrarily to what you may think, when a signal is received, its handler is 
_not_ called. Instead, it's Modules/signalmodule.c signal_handler() that's 
called. This handler stores the reception of the signal inside a table, and 
schedules the execution of the associated handler for later:

signal_handler(int sig_num)
{
[...]
Handlers[sig_num].tripped = 1;
/* Set is_tripped after setting .tripped, as it gets
   cleared in PyErr_CheckSignals() before .tripped. */
is_tripped = 1;
Py_AddPendingCall(checksignals_witharg, NULL);
[...]
}

checksignal_withargs() calls PyErr_CheckSignals(), which in turn calls the 
handler.
The pending calls are checked periodically from the interpreter main loop, in 
Python/ceval.c: when _Py_Ticker reaches 0, then we check for pending calls, and 
if there are any, we run the pending calls, hence checksignals_witharg, and the 
handler.
This is actually a documented behaviour, quoting signal documentation:
Although Python signal handlers are called asynchronously as far as the Python 
user is concerned, they can only occur between the “atomic” instructions of the 
Python interpreter. This means that signals arriving during long calculations 
implemented purely in C (such as regular expression matches on large bodies of 
text) may be delayed for an arbitrary amount of time.

But there's a race, imagine this happens:
- a thread (or a process for that matter) receives a signal
- signal_handler schedules the associated handler
- before _Py_Ticker reaches 0 and is checked from the interpreter main loop, a 
blocking call is made
- since the process is blocked in the call, the main eval loop doesn't run, and 
the handler doesn't get called until the process leaves the call and enters the 
main eval loop again. If the call doesn't return (e.g. select without timeout), 
then the process remains stuck forever.

This problem can also happen even if the signal is sent after select is called:
- the main thread calls select
- the second thread runs, and sends a signal to the process
- the signal is not received by the main thread, but by the second thread
- the second thread schedules execution of the handler
- since the main thread is blocked in select, the handler never gets called

But this case is quite flaky, because the documentation warns you:
Some care must be taken if both signals and threads are used in the same 
program. The fundamental thing to remember in using signals and threads 
simultaneously is: always perform signal() operations in the main thread of 
execution. Any thread can perform an alarm(), getsignal(), pause(), setitimer() 
or getitimer(); only the main thread can set a new signal handler, and the main 
thread will be the only one to receive signals (this is enforced by the Python 
signal module, even if the underlying thread implementation supports sending 
signals to individual threads). This means that signals can’t be used as a 
means of inter-thread communication. Use locks instead.

Sending signals to a process with multiple threads is risky, you should use 
locks.

Finally, I think that the documentation should be rephrased:
and the main thread will be the only one to receive signals (this is enforced 
by the Python signal module, even if the underlying thread implementation 
supports sending signals to individual threads).
It's false. What's guaranteed is that the signal handler will only be executed 
on behalf of the main thread, but any thread can _receive_ a signal.
And comments in Modules/signalmodule.c are misleading:
   We still have the problem that in some implementations signals
   generated by the keyboard (e.g. SIGINT) are delivered to all
   threads (e.g. SGI), while in others (e.g. Solaris) such signals are
   delivered to one random thread (an intermediate possibility would
   be to deliver it to the main thread -- POSIX?).  For now, we have
   a working implementation that works in all three cases -- the
   handler ignores signals if getpid() isn't the same as in the main
   thread.  XXX This is a hack.

Sounds strange. If only a thread other than the main thread receives the signal 
and you ignore it, then it's lost, isn't it ?
Furthermore, under Linux 2.6 and NPTL, getpid() returns the main thread PID 
even from another thread.

Peers ?

--
nosy: +neologix

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Thanks for the detailed analysis, Charles-François.

 Finally, I think that the documentation should be rephrased:

Yes, I think so.

 Furthermore, under Linux 2.6 and NPTL, getpid() returns the main thread
 PID even from another thread.

Yes, those threads belong to the same process.

But as mentioned, signals are a rather fragile inter-process communication 
device; just use a specific file descriptor.
And if you still wanna use signals, there's set_wakeup_fd():
http://docs.python.org/library/signal.html#signal.set_wakeup_fd

--
assignee:  - georg.brandl
components: +Documentation
nosy: +georg.brandl, pitrou, tim_one
priority:  - normal
versions: +Python 2.6, Python 2.7, Python 3.1, Python 3.2 -Python 2.4, Python 
2.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Andrew McNabb]

Andrew McNabb amcn...@mcnabbs.org added the comment:

I'm seeing something very similar to this.  In my case, I have a 
single-threaded program, and select fails to be interrupted by SIGCHLD.  I'm 
still tracking down more details, so I'll report back if I find more 
information.

--
nosy: +amcnabb

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Andrew McNabb]

Andrew McNabb amcn...@mcnabbs.org added the comment:

Sorry for the noise.  It turns out that my problem was unrelated.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5315] signal handler never gets called

[Péter Szabó]

New submission from Péter Szabó pts...@gmail.com:

According to http://docs.python.org/dev/library/signal.html , if I set
up a signal handler in the main thread, and then have the signal
delivered to the process, then the signal handler will be called in the
main thread. The attached Python script I've written, however, doesn't
work that way: sometimes the signal is completely lost, and the signal
handler is not called.

Here is how it should work. The code has two threads: the main thread
and the subthread. There is also a signal handler installed. The main
thread is running select.select(), waiting for a filehandle to become
readable. Then the subthread sends a signal to the process. The signal
handler writes a byte to the pipe. The select wakes up raising
'Interrupted system call' because of the signal.

I'm running Ubuntu Hardy on x86_64. With Python 2.4.5 and Python 2.5.2,
sometimes the signal handler is not called, and the select continues
waiting indefinitely. This is what I get on stdout in Python 2.4.5:

main pid=8555
--- 0
A
B
S
T
U
handler arg1=10 arg2=frame object at 0x79ab40
select got=(4, 'Interrupted system call')
read str='W'
--- 1
A
B
S
T
U

This means that iteration 0 completed successfully: the signal handler
got called, and the select raised 'Interrupted system call'. However,
iteration 1 was stuck: the signal handler was never called, and the
select waits indefinitely.

The script seems to work in Python 2.4.3, but it hangs in iteration
about 6.

--
components: Interpreter Core, Library (Lib)
files: tsig.py
messages: 82472
nosy: pts
severity: normal
status: open
title: signal handler never gets called
type: behavior
versions: Python 2.4, Python 2.5
Added file: http://bugs.python.org/file13138/tsig.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5315
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com