On Dec 30, 4:56 pm, lars van gemerden <l...@rational-it.com> wrote: > On Dec 30, 12:16 pm, lars van gemerden <l...@rational-it.com> wrote: > > > > > > > > > > > On Dec 29, 8:55 pm, Ian Kelly <ian.g.ke...@gmail.com> wrote: > > > > On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <l...@rational-it.com> > > > wrote: > > > > > Hello, > > > > > Can someone help me with the following: > > > > > I am using metaclasses to make classes and these classes to make > > > > instances. Now I want to use multiprocessing, which needs to pickle > > > > these instances. > > > > > Pickle cannot find the class definitions of the instances. I am trying > > > > to add a line to the __new__ of the metaclass to add the new class > > > > under the right name in the right module/place, so pickle can find > > > > it. > > > > > Is this the right approach? Can anyone explain to me where/how to add > > > > these classes for pickle to find and maybe why? > > > > It sounds like you're trying to do something like this? > > > > >>> class MetaClass(type): > > > > ... pass > > > ...>>> instance = MetaClass('<Anonymous>', (object,), {})() > > > >>> instance > > > > <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle > > > >>> pickle.dumps(instance) > > > > Traceback (most recent call last): > > > File "<stdin>", line 1, in <module> > > > File "c:\python27\lib\pickle.py", line 1374, in dumps > > > Pickler(file, protocol).dump(obj) > > > File "c:\python27\lib\pickle.py", line 224, in dump > > > self.save(obj) > > > File "c:\python27\lib\pickle.py", line 331, in save > > > self.save_reduce(obj=obj, *rv) > > > File "c:\python27\lib\pickle.py", line 401, in save_reduce > > > save(args) > > > File "c:\python27\lib\pickle.py", line 286, in save > > > f(self, obj) # Call unbound method with explicit self > > > File "c:\python27\lib\pickle.py", line 562, in save_tuple > > > save(element) > > > File "c:\python27\lib\pickle.py", line 295, in save > > > self.save_global(obj) > > > File "c:\python27\lib\pickle.py", line 748, in save_global > > > (obj, module, name)) > > > pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>: > > > it's not found as __main__.<Anonymous> > > > > Yeah, pickle's not going to work with anonymous classes. As you > > > suggest, you could dynamically add the classes to the module namespace > > > so that pickle.dumps will find them, but bear in mind that they will > > > also have to exist when calling pickle.loads, so you will need to be > > > able to reconstruct the same anonymous classes before unpickling later > > > on. > > > > Cheers, > > > Ian > > Ian also wrote: > > > ''' > > Actually, I was wrong, you probably don't need to do that. I suggest > > going with Robert Kern's suggestion to either register the class with > > the copy_reg module, or (perhaps better since it won't leak > > registrations) implement a __reduce__ method on the class. For > > example, this seems to work: > > > >>> def reconstructor(*metaclass_args): > > > ... cls = MetaClass.build_class(*metaclass_args) > > ... self = cls.__new__(cls) > > ... return self > > ...>>> class MetaClass(type): > > > ... @classmethod > > ... def build_class(mcs, arg1, arg2, arg3): > > ... # Do something useful with the args... > > ... class _AnonymousClass(object): > > ... __metaclass__ = mcs > > ... def __reduce__(self): > > ... return (reconstructor, ('foo', 'bar', 'baz'), > > self.__dict__) > > ... return _AnonymousClass > > ...>>> instance = MetaClass.build_class('foo', 'bar', 'baz')() > > >>> instance > > > <__main__._AnonymousClass object at 0x011DB410>>>> instance.banana = 42 > > >>> import pickle > > >>> s = pickle.dumps(instance) > > >>> s > > > "c__main__\nreconstructor > > \np0\n(S'foo'\np1\nS'bar'\np2\nS'baz'\np3\ntp4\nRp5\n(dp6\nS'banana'\np7\nI > > 42\nsb.">>> inst2 = pickle.loads(s) > > >>> inst2 > > > <__main__._AnonymousClass object at 0x011DBE90>>>> inst2.banana > > 42 > > >>> inst2.__class__ is instance.__class__ > > > False > > > Cheers, > > Ian > > > ''' > > Interesting, though I cannot say I completely understand this solution > (looked up __reduce__, but still). I am trying to adapt this example > to a situation where the metaclass generated classes are named at > runtime (not anonymous), but cannot figure it out. > > Cheers, Lars
Found a way to name the classes: def reconstructor(*metaclass_args): cls = MetaClass2.build_class(*metaclass_args) self = cls.__new__(cls) return self class MetaClass(type): @classmethod def build_class(mcs, name, arg1, arg2, arg3): return mcs(name, (object,), {"__reduce__": lambda e: (reconstructor2, (name, arg1, arg2, arg3), e.__dict__)}) I still wonder whether it might be easier to add the class to the namespace. Can anyone help me with that? Regards, Lars -- http://mail.python.org/mailman/listinfo/python-list