#15156: Pickling with __reduce__() loops creates invalid pickles
-------------------------------------+-------------------------------------
Reporter: vbraun | Owner:
Type: defect | Status: needs_info
Priority: major | Milestone: sage-6.3
Component: misc | Resolution:
Keywords: | Merged in:
Authors: SimonKing, | Reviewers:
jkeitel, novoselt, nbruin | Work issues:
Report Upstream: N/A | Commit:
Branch: | 82ba456b8c828700d28ac36822b300f00d8b1308
u/saraedum/ticket/15156 | Stopgaps:
Dependencies: #15692 |
-------------------------------------+-------------------------------------
Comment (by saraedum):
Replying to [comment:15 nbruin]:
> Replying to [comment:14 saraedum]:
> > Consider the following code, which creates two objects which reference
each other:
> > [...]
> Note that the circular reference wasn't put there in the corresponding
`__new__` call. It was a later change of state that caused the circularity
(this is always necessarily the case). So the circularity should be
recreated during the `__setstate__` phase of unpickling.
Right. But this is hard to do automatically.
> > I have no idea how to fix this. The domain is really needed here
because it determines the class of the result and it is used for a lookup
in a cache. Any ideas?
> Yes, domain and codomain are really part of a homset, so it's not
unreasonable to make them part of the `__new__` parameters. If
circularities arise here, then I'd say the domain and codomain are at
fault referencing a homset in their instantiation phase.
I'm not sure what you mean by "instantiation phase".
> They can't possibly have done that when they were originally created
either!
Sure. But when pickling you do not only want to restore the initial state
of an object but also to some extent the current state.
Here is an idea how to solve the problem with restoring the state. Say we
have `Hom(A,A)` and `A` somehow circles back to `Hom(A,A)` through its
state (e.g. its `__dict__`). The problem is that to unpickle `Hom(A,A)`,
`A` is created, `A`'s state is restored (which fails), and then `Hom(A,A)`
is created. What if `Hom(A,A)` took care of restoring the state of `A`
until `Hom(A,A)` exists?
I implemented a sketch of this idea and it at least solves this problem
for the cases that I had a look at. I changed `Hom`'s `__reduce__` to
{{{
args = (DelayedStatePickle(self._domain),
DelayedStatePickle(self._codomain), self.__category, False)
states = [a.inner_getstate() if isinstance(a,DelayedStatePickle) else None
for a in args]
return unpickle_from_DelayedStatePickles, (Hom, args), states
}}}
and make `__setstate__` set the state of domain and codomain:
{{{
for a,s in zip(self._args_delayed, states):
if s is not None:
a.__setstate__(s)
}}}
With
{{{
def unpickle_from_DelayedStatePickles(callable, args):
ret = callable(*args)
ret._args_delayed = args
return ret
class DelayedStatePickle(object):
def __init__(self, obj):
self.obj = obj
def inner_getstate(self):
reduction = self.obj.__reduce__()
assert all([r is None for r in reduction[3:]])
self.callable, self.args, self.state = reduction[:3]
return self.state
def __reduce__(self):
return self.callable, self.args
}}}
There is of course lots of space for improvement, but what do you think
about this idea?
--
Ticket URL: <http://trac.sagemath.org/ticket/15156#comment:16>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sage-trac.
For more options, visit https://groups.google.com/d/optout.
