[issue20854] multiprocessing.managers.Server: problem with returning proxy of registered object

[Irit Katriel]

Change by Irit Katriel :


--
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20854] multiprocessing.managers.Server: problem with returning proxy of registered object

[Davin Potts]

Davin Potts added the comment:

It appears that the multiple workarounds proposed by the OP (@allista) address 
the original request and that there is no bug or unintended behavior arising 
from multiprocessing itself.  Combined with the lack of activity in this 
discussion, I'm inclined to believe that the workarounds have satisfied the OP 
and this issue should be closed.

--
nosy: +davin
status: open -> pending
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20854] multiprocessing.managers.Server: problem with returning proxy of registered object

[Allis Tauri]

Allis Tauri added the comment:

Thanks for the suggestion. 
method_to_typeid and create_method are documented features, so I don't see why 
not. It does the trick in a cleaner way than my workaround: a metaclass for 
MyClass that just checks the arguments before creating a new instance. It just 
seems to me somewhat counterintuitive.

Another issue that arises in my case is: when I try to pass a proxy of MyClass 
to a subprocess it looses its' _manager during pickling and thus the ability to 
create proxies for children returned by get_child. This is solved by 
reimplementing the (not-working: http://bugs.python.org/issue5862) __reduce__ 
method of BaseManager in MyManager and creating corresponding custom proxy for 
MyClass with __reduce__ method also reimplemented.


So the working solution for the situation is:

either
1.1)
class ReturnProxy(type):
def __call__(cls, *args, **kwargs):
if not kwargs and args and isinstance(args[0], cls):
return args[0]
return super(ReturnProxy, cls).__call__(*args, **kwargs)

class MyClass(object):
__metaclass__ = ReturnProxy
###class body###

or
1.2)
Your solution with the second typeid registration.

2)
class AutoProxyMeta(type):
'''Metaclass that replicates multiprocessing.managers.MakeProxyType
functionality, but allows proxy classes that use it to be pickable'''
def __new__(cls, name, bases, attrs):
dic = {}
for meth in attrs.get('_exposed_', ()):
exec '''def %s(self, *args, **kwds):
return self._callmethod(%r, args, kwds)''' % (meth, meth) in dic
dic.update(attrs)
return super(AutoProxyMeta, cls).__new__(cls, name, bases, dic)

class MyClassProxy(BaseProxy):
__metaclass__ = AutoProxyMeta
_exposed_ = ('get_child',)
_method_to_typeid_ = dict(get_child='MyClass')
#or: _method_to_typeid_ = dict(get_child='_MyClass')

def __reduce__(self):
_unpickle, (cls, token, serializer, kwds) = BaseProxy.__reduce__(self)
kwds['manager'] = self._manager
return _unpickle, (cls, token, serializer, kwds)

class MyClassManager(UManager):
def __reduce__(self):
return (RebuildMyClassManager,
(self._address, None, self._serializer))
WorkCounterManager.register('MyClass', MyClass, MyClassProxy)
#optionally: WorkCounterManager.register('_MyClass', None, MyClassProxy, 
create_method=False)

def RebuildMyClassManager(address, authkey, serializer):
mgr = MyClassManager(address, authkey, serializer)
mgr.connect()
return mgr

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20854
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20854] multiprocessing.managers.Server: problem with returning proxy of registered object

[Richard Oudkerk]

Richard Oudkerk added the comment:

I am not sure method_to_typeid and create_method were really intended to be 
public -- they are only used by Pool proxies.

You can maybe work around the problem by registering a second typeid without 
specifying callable.  That can be used in method_to_typeid:

import multiprocessing.managers

class MyClass(object):
def __init__(self):
self._children = {}
def get_child(self, i):
return self._children.setdefault(i, type(self)())
def __repr__(self):
return 'MyClass %r' % self._children

class MyManager(multiprocessing.managers.BaseManager):
pass

MyManager.register('MyClass', MyClass,
   method_to_typeid = {'get_child': '_MyClass'})
MyManager.register('_MyClass',
   method_to_typeid = {'get_child': '_MyClass'},
   create_method=False)

if __name__ == '__main__':
m = MyManager()
m.start()
try:
a = m.MyClass()
b = a.get_child(1)
c = b.get_child(2)
d = c.get_child(3)
print a  # MyClass {1: MyClass {2: MyClass {3: MyClass {
finally:
m.shutdown()

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20854
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20854] multiprocessing.managers.Server: problem with returning proxy of registered object

[Allis Tauri]

New submission from Allis Tauri:

1. I have a tree-like recursive class MyClass. It's method 'get_child(i)' 
returns an instanse of that same class.
2. I register this class with BaseManager as follows:

class MyManager(BaseManager): pass
MyManager.register('MyClass', MyClass, method_to_typeid={'get_child':'MyClass'})

3. When I call 'get_child' method of AutoProxy[MyClass] object, the exception 
is raised in the '__init__' method of MyClass: it is called with a single 
argument which is the instance of MyClass returned by 'get_child'. This happens 
in the following code of multiprocessing.managers.Server.create method:

373 def create(self, c, typeid, *args, **kwds):
...
382 if callable is None:
383 assert len(args) == 1 and not kwds
384 obj = args[0]
385 else:
386 obj = callable(*args, **kwds) -This line raises the exception

This means that if ANY method registered with a Manager should return a proxy 
for a registered typeid, for which a callable is provided, it will fail, unless 
the callable is capable to handle such unexpected arguments.

--
components: Library (Lib)
messages: 212763
nosy: allista
priority: normal
severity: normal
status: open
title: multiprocessing.managers.Server: problem with returning proxy of 
registered object
versions: Python 2.7, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20854
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20854] multiprocessing.managers.Server: problem with returning proxy of registered object

[Ned Deily]

Changes by Ned Deily n...@acm.org:


--
nosy: +sbt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20854
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com