Ok I've realised my mistake. I put printlines all throughout pyzmq to see where it was hanging, and it was hanging on an import (context.socket()imports zmq.core.socket, though it's already been imported so it's cached in sys.modules).
>From the python docs on >threading<http://docs.python.org/library/threading.html> : "Firstly, other than in the main module, an import should not have the side effect of spawning a new thread and then waiting for that thread in any way. Failing to abide by this restriction can lead to a deadlock if the spawned thread directly or indirectly attempts to import a module." So I'm clearly not allowed to do this and will have to change my module structure a bit (doing initialisation after import via a function call) or live with the guilt of using one little time.sleep(). -Chris On Wed, Oct 3, 2012 at 3:12 PM, Chris Billington <[email protected] > wrote: > Hi guys, > > I'm having a delightfully puzzling issue with pyzmq (pyzmq 2.2.0, zmq > 2.1.11, Ubuntu 12.04 64 bit, Python 2.7.3). > > This code works fine: > > import threading > import time > import zmq > > def f(): > print 'thread started' > context = zmq.Context.instance() > print 'context obtained' > rep_sock = context.socket(zmq.REP) > print 'socket created' > > def connect(): > event = threading.Event() > threading.Thread(target=f).start() > time.sleep(1000) > event.wait() > > connect() > > It gets up to the time.sleep(1000) and then waits for ages like you'd > expect. However, if I put this code in a file b.py, and make a file a.pywhich > has a single line import > b, then running a.py causes a hang on the line rep_sock = > context.socket(zmq.REP) in the thread. > > Furthermore if I remove the event.wait(), then a.py runs fine! This is > the case even though the event.wait() clearly does not get executed. > > What I'm really using an event.wait() for is to wait for the thread to > bind a socket to inproc:// transport before the main thread proceeds to > connect to said socket. This is necessary as inproc transport does not > auto retry (https://zeromq.jira.com/browse/LIBZMQ-6), so > connect-before-bind fails with an exception. Whilst there are obvious > (though non ideal, retry with time.sleep, yuck) ways around this, it has > made me curious. How on earth is a line that isn't even executed affecting > pyzmq's ability to instantiate sockets? Why would it matter whether I'm > importing this code or running it directly? As far as I can tell, imported > code can only tell it's imported because __name__ is not '__main__', and > the only other difference I can think of is that __builtins__ gives you a > module if you're in __main__, whereas it gives you a dictionary in an > imported module (I've never understood the logic behind that decision). > > I consider myself fairly knowledgeable regarding Pythons's internals, and > I don't think I could write code that behaves like this if I wanted to, > short of having it read its own source looking for the event.wait() line > before deciding whether to hang or not. And somehow I don't think that's > what pyzmq is doing. > > Anyway this is very odd, and I suspect there is some deep magic behind it > regarding the implementation of threading.Event, and the fact that the > function is bytecode compiled when imported instead of being interpreted > as when run directly, at which point it can know about the event.wait()in > advance, and pre-emptively modify some state somewhere in a way that > breaks pyzmq. > > If anyone has some insight behind what's gong on, I'd love to hear it. >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
