Ryan Leslie <ryle...@gmail.com> added the comment: Yeah, storing pickled queues in the file system makes for some easy IPC :) It wasn't a very original idea, I took the pickling comments in the documentation at face value: http://docs.python.org/library/multiprocessing.html#proxy-objects
So, from what I can tell this issue is related to the mixing of standard python socket I/O with multiprocessing socket I/O, with state not being carried from the former to the latter. In multiprocessing/connection.py, SocketClient() creates a familiar python socket object which, when a default timeout has been set in the module, will be made non-blocking. In addition, the timeout is remembered in the socket object, and when calling socket.recv(), the function internal_select() will use this to perform the expected poll() or select(). However, after a connection is established, SocketClient() will not use python's socket implementation any further, and instead pass its low-level socket descriptor to a multiprocessing Connection object. This object has its own specialized socket I/O implementation, which is not at all aware of the timeout previously associated with the socket. As a result no select/poll occurs and, due to the socket's non-blocking status, recv() may return EAGAIN immediately. I suspect this is what's happening. There might be a number of ways to make SocketClient() more timeout friendly, but possibility could be to simply check if the python socket has a timeout associated, and if so, use connection.poll() in addition to connection.recv(). There may be other places in the code where similar changes would occur. You obviously have more experience with this code base so I'll be curious to see the outcome. ---------- type: -> behavior _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue6056> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
