On 05:46 am, kylerzhan...@gmail.com wrote:
Hi,
I'm a Google Summer of Code intern working on "Deferred Cancellation"
project. I'm recently working on adding cancellation support to
twisted.internet.task.LoopingCall.
However, after I added the canceller to LoopingCall.deferred,
the twisted.test.test_application.TestInternet2.testPickledTimer failed
due
to a PicklingError.
My branch is loopingcall-deferred-cancellation-6656. Here is the diff
of my
code: http://twistedmatrix.com/~diffresource.twistd/6656
[snip]
File "/usr/lib/python2.7/pickle.py", line 748, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <function <lambda> at 0x8f1fb8c>:
it's
not found as twisted.internet.posixbase.<lambda>
Two things to notice about the previous line. One, it is trying to
pickle a function defined using a lambda expression. Two, it is trying
to pickle something from twisted.internet.posixbase - which probably
means it's trying to pickle the reactor.
You can run trial with --debug and it will drop into pdb when it hits
this error. Then you can walk up and down the call stack and inspect
the objects pickle is operating on. You can get some idea of where
things are going wrong this way.
I thought the reason was the circular references. However I searched
about
it and found that pickle could handle the circular reference cases. But
the
only significant change is that after I added the canceller, there is a
circular reference between LoopingCall and LoopingCall.deferred. So I
don't
know what's the problem. How can I fix this?
There are two changes that seem like they could be relevant.
First, LoopingCall now keeps a reference to the Deferred returned by
application code. This means anything reachable from that Deferred is
going to get pickled when LoopingCall is pickled. This jumped out at me
first, but I don't think it's actually causing the problem.
Second, there is now a reference from the Deferred returned by
`LoopingCall` back to the `LoopingCall` instance - via the bound
`_cancel` method. `TimerService` holds on to a reference to this
`Deferred`.
Of course, stepping back, it doesn't make any sense to pickle
LoopingCall - it explicitly refers to the reactor, so it's never
actually going to be pickleable.
I suggest you take a look at TimerService and figure out why pickling
one of those ever tries to pickle a LoopingCall (take a look around
`__getstate__` and `volatile`, I think that's where the problem is). I
think you'll find an existing bug that the unit test previously failed
to reveal but which your changes have revealed.
Jean-Paul
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
