On 4/10/10 9:00 AM, Andreas Jung wrote:
> is it save to use the 'multiprocessing' module inside a BFG app?

I don't know think it is any less safe to use it in BFG than it would be in any 
other app.

Multiprocessing does some weird stuff at startup time in its "utils" module. 
For instance:

def _exit_function():
     global _exiting

     info('process shutting down')
     debug('running all "atexit" finalizers with priority >= 0')

     for p in active_children():
         if p._daemonic:
             info('calling terminate() for daemon %s', p.name)

     for p in active_children():
         info('calling join() for process %s', p.name)

     debug('running the remaining "atexit" finalizers')


Library code that registers an atexit hook or mutates module global state for 
purposes other than defining classes, functions, or variables as the result of 
an import is lame, lame, lame, lame.  I'm sure this is the reason you're 
getting this traceback but I don't know how to avoid it.

> In particular I need to maintain a multiprocessing.Pool() instance...
> this is working at the first glance however the BFG app won't shutdown
> in a clean way when stopped through a keyboard interrupt. The apps
> hangs usually inside the following code:
>    File "/opt/python-2.6.4/lib/python2.6/atexit.py", line 24, in
> _run_exitfuncs
>      func(*targs, **kargs)
>    File "/opt/python-2.6.4/lib/python2.6/multiprocessing/util.py", line
> 260, in _exit_function
>      _run_finalizers(0)
>    File "/opt/python-2.6.4/lib/python2.6/multiprocessing/util.py", line
> 235, in _run_finalizers
>      finalizer()
>    File "/opt/python-2.6.4/lib/python2.6/multiprocessing/util.py", line
> 174, in __call__
>      res = self._callback(*self._args, **self._kwargs)
>    File "/opt/python-2.6.4/lib/python2.6/multiprocessing/pool.py", line
> 363, in _terminate_pool
>      cls._help_stuff_finish(inqueue, task_handler, len(pool))
>    File "/opt/python-2.6.4/lib/python2.6/multiprocessing/pool.py", line
> 348, in _help_stuff_finish
>      inqueue._rlock.acquire()
> There is a Pool.terminate() method for closing the pool manually however
> I don't know exactly where to trigger the code.

I think you probably need to ask on the Python-users newsgroup or ask the 
author of multiprocessing.

- C

Repoze-dev mailing list

Reply via email to