Ok, I understand the issue. Cell objects potentially can be part in a cycle. Therefore for pickling, the three argument __reduce__ protocol is used, with an initial 'new', followed by a 'setstate' later. Only in the 'setstate' call will the type of the object be changed from the special "_wrap" type. But precisely because a reference cycle is being constructed, a fresh cell is being inserted into a function, before having had its state set, and the function constructor balks at this. I see no way around this using the current wrapped pickle technique except to fix the test in the function object. Once I have created a simple regression test, I will submit a fix.
K From: stackless-boun...@stackless.com [mailto:stackless-boun...@stackless.com] On Behalf Of Crispin Wellington Sent: 12. október 2009 05:27 To: Richard Tew Cc: stackless@stackless.com Subject: Re: [Stackless] exceptions.TypeError: arg 5 (closure) expected cell, found stackless._wrap.cell when unpickling stackless thread OK. I boiled down the code to a small snippet that triggers the bug. It is a bug that is exposed through my use of Twisted. If i create a factory, and then two callbacks that are closures, each callback setting thread level variables. Then I add the callbacks to the factory's deferred, and then I sleep the threadlet forever. Initiate the tasklet, pickle it, then unpickle it, and BAM, you have your error: $ /usr/local/stackless/bin/python break-stackless.py Traceback (most recent call last): File "break-stackless.py", line 47, in <module> test() File "break-stackless.py", line 37, in test obj = pickle.loads(data) File "/usr/local/stackless/lib/python2.6/pickle.py", line 1409, in loads return Unpickler(file).load() File "/usr/local/stackless/lib/python2.6/pickle.py", line 893, in load dispatch[key](self) File "/usr/local/stackless/lib/python2.6/pickle.py", line 1252, in load_build setstate(state) TypeError: arg 5 (closure) expected cell, found stackless._wrap.cell The code is.... ----------------------------- import stackless import pickle from twisted.web import client def tasklet(): # create an example state factory = client.HTTPClientFactory("http://www.google.com/";, agent = "stackless/1.0" ) get_complete = [False] get_failed = [False] def _doFailure(data): get_failed[0] = factory.status def _doSuccess(data): get_complete[0] = factory.status factory.deferred.addCallback(_doSuccess).addErrback(_doFailure) while True: stackless.schedule() def test(): # run the task task = stackless.tasklet(tasklet) task.setup() task.run() # pump once stackless.schedule() # now serialise data = pickle.dumps(task) # now deserialise obj = pickle.loads(data) # resume task.kill() obj.insert() # pump again stackless.schedule() if __name__=="__main__": test() ----------------------- If code formatting is an issue, I've attached the code to the email as a file... Can anyone shed any light on this bug? Kind Regards Crispin On Sat, Oct 10, 2009 at 2:21 AM, Richard Tew <richard.m....@gmail.com<mailto:richard.m....@gmail.com>> wrote: On Fri, Oct 9, 2009 at 10:38 PM, Crispin Wellington <retrogradeor...@gmail.com<mailto:retrogradeor...@gmail.com>> wrote: > I have written a server that contains stackless threads. When the server > shuts down these threads are serialised to disk. This all happens without > incident and I end up with a reasonably sized file per pickled object. > > When I go to deserialise them I hit a problem. I get this... > ------------------- > File > "/home/cwellington/Development/yabi/yabi-be-twisted/trunk/TaskManager/Tasklets.py", > line 54, in load > task = pickle.loads(data) > File "/usr/local/stackless/lib/python2.6/pickle.py", line 1415, in > loads > return Unpickler(file).load() > File "/usr/local/stackless/lib/python2.6/pickle.py", line 898, in load > dispatch[key](self) > File "/usr/local/stackless/lib/python2.6/pickle.py", line 1258, in > load_build > setstate(state) > exceptions.TypeError: arg 5 (closure) expected cell, found > stackless._wrap.cell > -------------------- > > I think this might be a bug. The arguments are being passed back into the > constructor of the stackless._wrap.function and the constructor is throwing > that exception. So in essence, the __reduce__ or __reduce_ex__ functions are > creating instatiation values that do not work on reinstantiation. > > I changed pickle.py to print out some debug. setstate is... <built-in method > __setstate__ of stackless._wrap.function object at 0x21ca0c8> and (state)[4] > is... (<cell at 0x2605f30: empty>, <cell at 0x2940718: list object at > 0x259f518>, <cell at 0x2940670: str object at 0x2940768>, <cell at > 0x2940750: str object at 0x29407d8>, <cell at 0x29407c0: int object at > 0x25cd338>) > > These all seem to be "cell" objects. Why aren't they (stackless._wrap.cell) > recognised as "cells"? > > I can provide a pickletools.dis() dump of the pickle stream if wanted. > > Using Stackless Python 2.6.2 on 64 bit linux (x86_64). > > Can anybody help me decode my objects? or even find a way to encode them so > the are decodable? Is this a bug, or am I doing it wrong? It definitely looks like a bug. If you can provide the smallest possible reproduction case, it would help us look into it and we could add it to the test suite as a regression test. If you want your objects back in the meantime, you might try compiling 2.6 with the relevant code raising these errors commented out. Richard.
_______________________________________________ Stackless mailing list Stackless@stackless.com http://www.stackless.com/mailman/listinfo/stackless
