[issue12905] multiple errors in test_socket on OpenBSD
Changes by Charles-François Natali neolo...@free.fr: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Remi Pointel pyt...@xiri.fr added the comment: I think we could close this bug because it's du to the pthread library on OpenBSD and not Python. We are switching to rthreads and I can't reproduce this bug. Thanks for your help and sorry for the delay. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Charles-François Natali neolo...@free.fr added the comment: I think we could close this bug because it's du to the pthread library on OpenBSD and not Python. Indeed. -- resolution: - invalid stage: - committed/rejected type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Charles-François Natali neolo...@free.fr added the comment: Rémi, do you want to submit a patch to skip those tests on OpenBSD? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Charles-François Natali neolo...@free.fr added the comment: It looks like Python cannot do much to workaround OpenBSD issues. IMO the best fix is just to skip these tests on OpenBSD, until OpenBSD handles correctly signals in programs linked to pthread. The same fix can be used for #12903. Agreed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
STINNER Victor victor.stin...@haypocalc.com added the comment: It looks like Python cannot do much to workaround OpenBSD issues. IMO the best fix is just to skip these tests on OpenBSD, until OpenBSD handles correctly signals in programs linked to pthread. The same fix can be used for #12903. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Charles-François Natali neolo...@free.fr added the comment: Using SA_RESTART, read() is not interrupted. But if the program is linked to pthread, read() is always interrupted: with sa_flags=0 or sa_flags=SA_RESTART. Ouch... But OpenBSD's pthread implementation has severe limitations/bugs. rthread doc contains: Future work: Quite simply, signal handling is one the most complicated aspects of threads to get right. (...) This paper dates back to 2005, I was hoping they would have solved this by now... As for the original problem, IIUC you don't reproduce it with your C test code... It might be due to a subtle difference in the way Python is built (like POSIX_SOURCE...), but it's hard to tell... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Charles-François Natali neolo...@free.fr added the comment: I hope that this issue is not related to threads+signals. We got many threads+signals issues on FreeBSD 6. Yep. OpenBSD has a really specific pthread implementation (in user-space, using non-blocking I/O), so it might very well be yet another threads+signals occurrence. @Rémi What happens if you run a code equivalent to test_sendall_interrupted on its own? I mean, if you try something like this: import signal, socket c, s = socket.socketpair() signal.signal(signal.SIGALRM, lambda x,y: 0) signal.alarm(1) c.sendall(bx * (1024**2)) If it works, you could try creating a dummy thread before setting up the signal handler, something like: t = threading.Thread(target=lambda: 0) t.start() t.join() And retry. The problem with all those interruption tests (like issue #12903) is that they'll break on many *BSD if another thread is running. Although in this specific case, I suspect that there are no threads running, and we're really encountering a kernel/libc bug (Victor, remember the funny bug on FreeBSD where kill(getpid(), signal) didn't deliver the signal synchronously after the first thread had been created?). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Remi Pointel pyt...@xiri.fr added the comment: What happens if you run a code equivalent to test_sendall_interrupted on its own? I mean, if you try something like this: Hi, it blocks too: $ gdb -args ./python [...] (gdb) run Starting program: /usr/ports/pobj/Python-3.2.2/Python-3.2.2/python Python 3.2.2 (default, Sep 5 2011, 21:21:34) [GCC 4.2.1 20070719 ] on openbsd5 Type help, copyright, credits or license for more information. import signal, socket c, s = socket.socketpair() signal.signal(signal.SIGALRM, lambda x,y: 0) 0 signal.alarm(1) 0 [New process 28830] c.sendall(bx * (1024**2)) ^C Program received signal SIGINT, Interrupt. Cannot access memory at address 0x98 (gdb) bt #0 0x00020125678a in poll () from /usr/lib/libc.so.60.1 #1 0x00020619b4aa in _thread_kern_poll (wait_reqd=value optimized out) at /usr/src/lib/libpthread/uthread/uthread_kern.c:780 #2 0x00020619c3a8 in _thread_kern_sched (scp=0x0) at /usr/src/lib/libpthread/uthread/uthread_kern.c:382 #3 0x000206190ade in sendto (fd=9, msg=0x2032d6020, len=1044480, flags=0, to=0x0, to_len=0) at /usr/src/lib/libpthread/uthread/uthread_sendto.c:63 #4 0x0002105f670a in sock_sendall () from /usr/ports/pobj/Python-3.2.2/Python-3.2.2/build/lib.openbsd-5.0-amd64-3.2/_socket.so #5 0x00020668b172 in PyEval_EvalFrameEx () from /usr/local/lib/libpython3.2m.so.1.0 #6 0x00020668bf66 in PyEval_EvalCodeEx () from /usr/local/lib/libpython3.2m.so.1.0 #7 0x00020668c22b in PyEval_EvalCode () from /usr/local/lib/libpython3.2m.so.1.0 #8 0x0002066a93d7 in run_mod () from /usr/local/lib/libpython3.2m.so.1.0 #9 0x0002066abcae in PyRun_InteractiveOneFlags () from /usr/local/lib/libpython3.2m.so.1.0 #10 0x0002066abf3e in PyRun_InteractiveLoopFlags () from /usr/local/lib/libpython3.2m.so.1.0 #11 0x0002066ac04c in PyRun_AnyFileExFlags () from /usr/local/lib/libpython3.2m.so.1.0 #12 0x0002066bc9f4 in Py_Main () from /usr/local/lib/libpython3.2m.so.1.0 #13 0x00400e75 in main () Thanks for your help, Remi. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Charles-François Natali neolo...@free.fr added the comment: Hi, it blocks too: Oops, I just realized there was a typo in the sample test. The signal handler should be lambda x,y: 1/0 and not lambda x,y: 0 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
STINNER Victor victor.stin...@haypocalc.com added the comment: I tried the following script on OpenBSD 5 with Python 3.3: --- import signal import sys s = signal.SIGALRM signal.signal(s, lambda x,y: 1/0) signal.alarm(1) signal.siginterrupt(s, True) sys.stdin.read() --- The C signal handler is called, but the system call (read in this case) is not interrupted. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
STINNER Victor victor.stin...@haypocalc.com added the comment:
I tried the following script on OpenBSD 5 with Python 3.3: ...
Bad news: the script doesn't hang if Python is build without threads.
Short C program to test interrupted syscalls:
-
#include signal.h
#include stdio.h
#include pthread.h
void
handler(int signum)
{
printf(HANDLER!\n);
}
void _noop()
{
}
int main()
{
int s = SIGALRM;
char buffer[1024];
int n;
static int dummy = 0;
pthread_t thread1;
pthread_create(thread1, NULL, (void *) _noop, dummy);
pthread_join(thread1, NULL);
signal(s, handler);
siginterrupt(s, 1);
alarm(1);
printf(read...\n);
n = read(0, buffer, 1024);
printf(read-%i\n, n);
return 0;
}
-
read() is interrupted after 1 second, it works.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12905
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
STINNER Victor victor.stin...@haypocalc.com added the comment:
Oh, siginterrupt(SIGALRM, 0) doesn't work in a program linked to pthread.
Example:
#include signal.h
#include stdio.h
void
handler(int signum)
{ printf(HANDLER!\n); }
int main()
{
int s = SIGALRM;
char buffer[1024];
int n;
signal(s, handler);
siginterrupt(s, 0);
alarm(1);
printf(read...\n);
n = read(0, buffer, 1024);
printf(read-%i\n, n);
return 0;
}
This program ends after 1 second with read--1 if it is linked to pthread
(bug!), it hangs if it is not linked to pthread (ok).
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12905
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Charles-François Natali neolo...@free.fr added the comment: The C signal handler is called, but the system call (read in this case) is not interrupted. That's what I thought... Bad news: the script doesn't hang if Python is build without threads. Makes sense. When linked with pthread, all I/O syscalls are actually non-blocking. read() is interrupted after 1 second, it works. Hmmm... Does it still work if you don't a create thread beforehand? Also, one difference is that Python uses sigaction to setup the signal handler. There might be subtle semantics change/bugs between signal/sigaction. Oh, siginterrupt(SIGALRM, 0) doesn't work in a program linked to pthread. You could try with sigaction/SA_RESTART. But OpenBSD's pthread implementation has severe limitations/bugs. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
STINNER Victor victor.stin...@haypocalc.com added the comment: read() is interrupted after 1 second, it works. Does it still work if you don't a create thread beforehand? Yes, the read() is also interrupted as expected if no thread is created. one difference is that Python uses sigaction to setup the signal handler If the handler is installed using the following code, read() is interrupted: -- sigemptyset(sa.sa_mask); sa.sa_handler = handler; sa.sa_flags = 0; sigaction(SIGALRM, sa, NULL); -- Using sa.sa_flags=SA_RESTART, read() hangs (it is not interrupted). Python uses sigaction with flags=0. You could try with sigaction/SA_RESTART. Using SA_RESTART, read() is not interrupted. But if the program is linked to pthread, read() is always interrupted: with sa_flags=0 or sa_flags=SA_RESTART. But OpenBSD's pthread implementation has severe limitations/bugs. rthread doc contains: Future work: Quite simply, signal handling is one the most complicated aspects of threads to get right. (...) http://www.openbsd.org/papers/eurobsd2005/tedu-rthreads.pdf -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
New submission from Remi Pointel pyt...@xiri.fr: Hi, I have few problems in test_socket.py. 1) test_sendall_interrupted blocks on OpenBSD, I must Ctrl+c to interrupt the test. I added a skip to have full regress tests, but I think it's not the proper way. I have run it into gdb, and ctrl+c it: test_sendall_interrupted (__main__.GeneralModuleTests) ... [New process 20677] ^C Program received signal SIGINT, Interrupt. 0x000206d2378a in poll () from /usr/lib/libc.so.60.1 (gdb) bt #0 0x000206d2378a in poll () from /usr/lib/libc.so.60.1 #1 0x000205f4e4aa in _thread_kern_poll (wait_reqd=Variable wait_reqd is not available. ) at /usr/src/lib/libpthread/uthread/uthread_kern.c:780 #2 0x000205f4f3a8 in _thread_kern_sched (scp=0x0) at /usr/src/lib/libpthread/uthread/uthread_kern.c:382 #3 0x000205f43ade in sendto (fd=8, msg=0x2047bb040, len=1044480, flags=0, to=0x0, to_len=0) at /usr/src/lib/libpthread/uthread/uthread_sendto.c:63 #4 0x000210a5b243 in sock_sendall (s=0x20115c948, args=0x2011510d0) at /usr/ports/pobj/Python-3.2.2/Python-3.2.2/Modules/socketmodule.c:2699 #5 0x00468992 in PyEval_EvalFrameEx (f=0x20a281c30, throwflag=Variable throwflag is not available. ) at Python/ceval.c:3921 #6 0x0046a95c in PyEval_EvalCodeEx (_co=0x20fc581b0, globals=Variable globals is not available. ) at Python/ceval.c:3350 #7 0x00466df6 in PyEval_EvalFrameEx (f=0x20a9a3c30, throwflag=Variable throwflag is not available. ) at Python/ceval.c:4019 #8 0x00468c1a in PyEval_EvalFrameEx (f=0x2033af030, throwflag=Variable throwflag is not available. ) at Python/ceval.c:4009 #9 0x0046a95c in PyEval_EvalCodeEx (_co=0x2028a9de8, globals=Variable globals is not available. ) at Python/ceval.c:3350 #10 0x00466df6 in PyEval_EvalFrameEx (f=0x2052f1430, throwflag=Variable throwflag is not available. ) at Python/ceval.c:4019 #11 0x0046a95c in PyEval_EvalCodeEx (_co=0x2028a9f58, globals=Variable globals is not available. ) at Python/ceval.c:3350 #12 0x00511ce3 in function_call (func=0x2050d5620, arg=0x20115d330, kw=0x2052f0030) at Objects/funcobject.c:629 #13 0x004e4a7f in PyObject_Call (func=0x2050d5620, arg=0x20115d330, kw=0x2052f0030) at Objects/abstract.c:2149 #14 0x0046456d in PyEval_EvalFrameEx (f=0x2052f0e30, throwflag=Variable throwflag is not available. ) at Python/ceval.c:4236 #15 0x0046a95c in PyEval_EvalCodeEx (_co=0x2028ad268, globals=Variable globals is not available. ) at Python/ceval.c:3350 #16 0x00511ce3 in function_call (func=0x2050d5790, arg=0x20115d240, kw=0x0) at Objects/funcobject.c:629 #17 0x004e4a7f in PyObject_Call (func=0x2050d5790, arg=0x20115d240, kw=0x0) at Objects/abstract.c:2149 #18 0x004ff349 in method_call (func=0x2050d5790, arg=0x20115a5a0, kw=0x0) at Objects/classobject.c:319 #19 0x004e4a7f in PyObject_Call (func=0x20115d150, arg=0x20115a5a0, kw=0x0) at Objects/abstract.c:2149 #20 0x0042cb66 in slot_tp_call (self=Variable self is not available. ) at Objects/typeobject.c:5044 #21 0x004e4a7f in PyObject_Call (func=0x20113b680, arg=0x20115a5a0, kw=0x0) at Objects/abstract.c:2149 #22 0x004628fb in PyEval_EvalFrameEx (f=0x2071dc030, throwflag=Variable throwflag is not available. ) at Python/ceval.c:4141 #23 0x0046a95c in PyEval_EvalCodeEx (_co=0x2086db6b8, globals=Variable globals is not available. ) at Python/ceval.c:3350 #24 0x00511ce3 in function_call (func=0x2086de1d0, arg=0x20115d1c8, kw=0x2071dd430) at Objects/funcobject.c:629 #25 0x004e4a7f in PyObject_Call (func=0x2086de1d0, arg=0x20115d1c8, kw=0x2071dd430) at Objects/abstract.c:2149 #26 0x0046456d in PyEval_EvalFrameEx (f=0x2071dde30, throwflag=Variable throwflag is not available. ) at Python/ceval.c:4236 #27 0x0046a95c in PyEval_EvalCodeEx (_co=0x2086db490, globals=Variable globals is not available. ) at Python/ceval.c:3350 #28 0x00511ce3 in function_call (func=0x2086de060, arg=0x20113f420, kw=0x0) at Objects/funcobject.c:629 #29 0x004e4a7f in PyObject_Call (func=0x2086de060, arg=0x20113f420, kw=0x0) at Objects/abstract.c:2149 #30 0x004ff349 in method_call (func=0x2086de060, arg=0x2011411b0, kw=0x0) at Objects/classobject.c:319 #31 0x004e4a7f in PyObject_Call (func=0x20115d0d8, arg=0x2011411b0, kw=0x0) at Objects/abstract.c:2149 #32 0x0042cb66 in slot_tp_call (self=Variable self is not available. ) at Objects/typeobject.c:5044 #33 0x004e4a7f in PyObject_Call (func=0x201138920, arg=0x2011411b0, kw=0x0) at Objects/abstract.c:2149 #34 0x004628fb in PyEval_EvalFrameEx (f=0x2071dc430, throwflag=Variable throwflag is not available. ) at Python/ceval.c:4141 #35 0x0046a95c in PyEval_EvalCodeEx (_co=0x2086db6b8, globals=Variable globals is not available. ) at Python/ceval.c:3350 #36
[issue12905] multiple errors in test_socket on OpenBSD
Remi Pointel pyt...@xiri.fr added the comment: I add bt output with gdb 7, to have more informations. -- Added file: http://bugs.python.org/file23105/OpenBSD_test_socket_gdb.log ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +haypo, neologix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
STINNER Victor victor.stin...@haypocalc.com added the comment: I hope that this issue is not related to threads+signals. We got many threads+signals issues on FreeBSD 6. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12905] multiple errors in test_socket on OpenBSD
STINNER Victor victor.stin...@haypocalc.com added the comment: Issue #12903 is similar to this one. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12905 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
