New submission from Mike Jarvis <[email protected]>:
I had a function for making a logger proxy that could be safely passed to
multiprocessing workers and log back to the main logger with essentially the
following code:
```
import logging
from multiprocessing.managers import BaseManager
class SimpleGenerator:
def __init__(self, obj): self._obj = obj
def __call__(self): return self._obj
def get_logger_proxy(logger):
class LoggerManager(BaseManager): pass
logger_generator = SimpleGenerator(logger)
LoggerManager.register('logger', callable = logger_generator)
logger_manager = LoggerManager()
logger_manager.start()
logger_proxy = logger_manager.logger()
return logger_proxy
logger = logging.getLogger('test')
logger_proxy = get_logger_proxy(logger)
```
This worked great on python 2.7 through 3.7. I could pass the resulting
logger_proxy to workers and they would log information, which would then be
properly sent back to the main logger.
However, on python 3.8.2 (and 3.8.0) I get the following:
```
Traceback (most recent call last):
File "test_proxy.py", line 20, in <module>
logger_proxy = get_logger_proxy(logger)
File "test_proxy.py", line 13, in get_logger_proxy
logger_manager.start()
File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/managers.py", line
579, in start
self._process.start()
File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/process.py", line
121, in start
self._popen = self._Popen(self)
File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/context.py", line
283, in _Popen
return Popen(process_obj)
File
"/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py",
line 32, in __init__
super().__init__(process_obj)
File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/popen_fork.py",
line 19, in __init__
self._launch(process_obj)
File
"/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py",
line 47, in _launch
reduction.dump(process_obj, fp)
File "/anaconda3/envs/py3.8/lib/python3.8/multiprocessing/reduction.py", line
60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object
'get_logger_proxy.<locals>.LoggerManager'
```
So it seems that something changed about ForkingPickler that makes it unable to
handle the closure in my get_logger_proxy function.
I don't know if this is an intentional change in behavior or an unintentional
regression. If the former, I would appreciate advice on how to modify the
above code to work.
Possibly relevant system details:
```
$ uname -a
Darwin Fife 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar 5 22:24:32 PST 2018;
root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64
$ python --version
Python 3.8.2
$ which python
/anaconda3/envs/py3.8/bin/python
$ conda info
active environment : py3.8
active env location : /anaconda3/envs/py3.8
shell level : 2
user config file : /Users/Mike/.condarc
populated config files : /Users/Mike/.condarc
conda version : 4.8.3
conda-build version : 3.18.5
python version : 3.6.5.final.0
virtual packages : __osx=10.13.4
base environment : /anaconda3 (writable)
channel URLs : https://conda.anaconda.org/conda-forge/osx-64
https://conda.anaconda.org/conda-forge/noarch
https://conda.anaconda.org/astropy/osx-64
https://conda.anaconda.org/astropy/noarch
https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /anaconda3/pkgs
/Users/Mike/.conda/pkgs
envs directories : /anaconda3/envs
/Users/Mike/.conda/envs
platform : osx-64
user-agent : conda/4.8.3 requests/2.23.0 CPython/3.6.5
Darwin/17.5.0 OSX/10.13.4
UID:GID : 501:20
netrc file : /Users/Mike/.netrc
offline mode : False
```
----------
components: Library (Lib)
messages: 371211
nosy: Mike Jarvis
priority: normal
severity: normal
status: open
title: BaseManager cannot start with local manager
type: behavior
versions: Python 3.8
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue40942>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
