On Fri, Oct 27, 2017 at 10:42 AM, Jeroen Demeyer <jdeme...@cage.ugent.be> wrote: > This is a topic which comes up now and then but which hasn't been resolved > so far. > > Sage has a "pickle jar" stored in src/ext/pickle_jar/pickle_jar.tar.bz2 and > there are tests in src/sage/structure/sage_object.pyx which check that every > object in the pickle jar can be unpickled without raising an exception. > > The theory is that this ensures that people having old pickles on their > machines can still use them with newer versions of Sage. However, the pickle > jar does not achieve this goal because > > (1) The doctest only tests that the pickles can be unpickled. There are no > tests that the unpickled objects still function correctly. See > https://trac.sagemath.org/ticket/16311 > > (2) Nobody adds new pickles to the pickle jar. The last time that a pickle > was added was in 2011. So, if anything, it only tests that pickles which are > at least 6 years old still work correctly. > > Since the pickle jar doesn't do what it is meant to do, I suggest to be > pragmatic and remove it completely. > > Having the pickle jar is a burden for development because it prevents > certain refactorings or removals. For example finite_field_ext_pari.py was > deprecated in 2014 but we cannot remove it because it breaks the pickle jar. > I know that I could probably fix this with register_unpickle_override() but > that feels a waste of time because I wonder if anybody really cares. In the > mean time, people need to maintain that unused deprecated file to make it > Python 3 compatible for example. > > I'm sure that some people have suggestions for improving the pickle jar > procedure. But still, the fact remains that many pickles in the current > pickle jar are broken. So there is no point in keeping them.
+1 Plus, while pickling has many valid runtime use-cases, particularly for IPC, and short-term preservation of objects between interpreter sessions, it was *never* intended for long-term data storage, in part precisely because it's directly tied to the source code that was used to produce the pickle file. The only truly "correct" way to restore old pickled objects is to do so with the same version of the software the pickle was created with. If there does not yet exist, and/or is need for better serialization formats for objects in Sage that's something worth talking seriously about (I feel like the MitM work being done is to this effect though). But pickle ain't it! -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
