Hi all, I'm writing a simple daemon to test out a web application that receives messages via HTTP, using urllib2 for the HTTP connections. The program daemonizes nicely and sends the messages just fine, but when it's sending the first message (which takes place 2-12 seconds after starting the program, since the send interval is randomized), the terminal I started the program from prints out:
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec(). Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug. ...repeating it several dozen times. The execution of the program seems to run normally regardless, at least the messages arrive to the HTTP server just fine. I'm running OS X Leopard 10.5.6, the system default Python 2.5.1, and using this daemonizing code: http://code.activestate.com/recipes/278731/ The actual offending bit in my own code is inside __main__, and occurs in the main loop of the program, right after the daemonization has taken place: [...code...] while True: time.sleep(int(random.random() * 10 + 2)) params = urllib.urlencode({'json': json.dumps({'msg': random_items(phrases)[0][1], 'chan': '07101237-20ad-4229-b663-0c3dd46b8706'} ) }) req = urllib2.Request('http://localhost:8000/json/new_msg/', params, headers) try: fd = urllib2.urlopen(req) # THIS PRODUCES THE ERROR response = fd.read() except urllib2.HTTPError, e: sys.exit(1) [...more code...] I commented and gradually uncommented bunches of code until the error disappeared, and I narrowed it down to the urllib2.urlopen call: if it (and the next row that depends on it) are commented out, the error doesn't appear. Googling the error revealed this Leopard release notes document from Apple: http://developer.apple.com/releasenotes/CoreFoundation/CoreFoundation.html Relevant quote: ---- "CoreFoundation and fork() Due to the behavior of fork(), CoreFoundation cannot be used on the child-side of fork(). If you fork(), you must follow that with an exec*() call of some sort, and you should not use CoreFoundation APIs within the child, before the exec*(). The applies to all higher-level APIs which use CoreFoundation, and since you cannot know what those higher-level APIs are doing, and whether they are using CoreFoundation APIs, you should not use any higher-level APIs either. This includes use of the daemon() function. Additionally, per POSIX, only async-cancel-safe functions are safe to use on the child side of fork(), so even use of lower-level libSystem/BSD/UNIX APIs should be kept to a minimum, and ideally to only async-cancel-safe functions. This has always been true, and there have been notes made of this on various Cocoa developer mailling lists in the past. But CoreFoundation is taking some stronger measures now to "enforce" this limitation, so we thought it would be worthwhile to add a release note to call this out as well. A message is written to stderr when something uses API which is definitely known not to be safe in CoreFoundation after fork(). If file descriptor 2 has been closed, however, you will get no message or notice, which is too bad. We tried to make processes terminate in a very recognizable way, and did for a while and that was very handy, but backwards binary compatibility prevented us from doing so." ---- Now, I know nothing about CoreFoundation and just about nothing about the Unix side of things (fork/exec etc.), but it appears that urllib2.urlopen is somehow dependent on some CoreFoundation code and thus doesn't play well with fork(). If one of the os.exec* functions need to be used, like the release notes suggest, I have no idea how and where. Is this a bug that should be reported either to Apple or to the Python people, or do I simply need a different type of forking semantic in a Leopard system? Any advice will be much appreciated. - Jarkko Laiho _______________________________________________ Pythonmac-SIG maillist - Pythonmac-SIG@python.org http://mail.python.org/mailman/listinfo/pythonmac-sig
