#10768: Revisit the pickle jar procedure
-------------------------------+-----------------------------
Reporter: nthiery | Owner: was
Type: enhancement | Status: needs_info
Priority: major | Milestone: sage-wishlist
Component: pickling | Resolution:
Keywords: | Merged in:
Authors: | Reviewers:
Report Upstream: N/A | Work issues:
Branch: | Commit:
Dependencies: | Stopgaps:
-------------------------------+-----------------------------
Changes (by vbraun):
* cc: ohanar (added)
* status: new => needs_info
Old description:
> The current pickle jar mechanism has some drawbacks:
>
> - We don't know how old pickles in the pickle jar are
>
> - We may be testing an old pickle, but not a recent one
>
> - Updating specific pickles is a bit tedious
>
> Here is a new proposal:
>
> 1. Pickles will no longer be stored in a `.tar.bz2` file but simply as
> '''files''' within the directory `extcode/pickle_jar/$VERSION`. This
> will likely increase the on-disk space needed for a Sage install, but
> will not have a big influence on Sage distributions, since we have an
> extcode spkg anyway (which is tarred and compressed).
> 1. Pickles will be '''under `hg` control''' (this will now become
> possible).
> 1. The `$VERSION` in the directory name refers to the Sage version used
> to create the pickle. Once a pickle has been made, it will remain in
> place in that directory, even in subsequent Sage versions (so sage-4.7.2
> will contain `pickle_jar/4.7`, `pickle_jar/4.7.1` and
> `pickle_jar/4.7.2`).
> 1. When making a new release, the release manager will unpickle all old
> pickles and repickle them with the new Sage version. Whenever a pickle
> has changed, the new (changed) pickle will be stored in
> `pickle_jar/$NEWVERSION`. The old pickle is kept where it was.
> 1. sage.structure.sage_object.unpickle_all will check all pickles (old
> and new).
> 1. If some day some pickle rots away and it is decided by consensus to
> not support unpickling it anymore, then the patch author would simply `hg
> remove` the old pickle.
New description:
The current pickle jar mechanism has some drawbacks:
- We don't know how old pickles in the pickle jar are
- We may be testing an old pickle, but not a recent one
- Updating specific pickles is a bit tedious
Here is a new proposal:
1. Pickles will no longer be stored in a `.tar.bz2` file but simply as
'''files''' within the directory `extcode/pickle_jar/$VERSION`. This will
likely increase the on-disk space needed for a Sage install, but will not
have a big influence on Sage distributions, since we have an extcode spkg
anyway (which is tarred and compressed).
1. Pickles will be '''under `git` control''' (this will now become
possible).
1. The `$VERSION` in the directory name refers to the Sage version used
to create the pickle. Once a pickle has been made, it will remain in place
in that directory, even in subsequent Sage versions (so sage-4.7.2 will
contain `pickle_jar/4.7`, `pickle_jar/4.7.1` and `pickle_jar/4.7.2`).
1. When making a new release, the release manager will unpickle all old
pickles and repickle them with the new Sage version. Whenever a pickle
has changed, the new (changed) pickle will be stored in
`pickle_jar/$NEWVERSION`. The old pickle is kept where it was.
1. sage.structure.sage_object.unpickle_all will check all pickles (old
and new).
1. If some day some pickle rots away and it is decided by consensus to
not support unpickling it anymore, then the patch author would simply `git
remove` the old pickle.
--
Comment:
Do we really put all that into the git repo? The current (incredibly old)
pickle jar is about 2MB uncompressed. A new one is likely considerably
larger. There are of the order of 10 minor Sage releases every year. I
don't know often the pickle changes, but it seems likely that this'll
generate on the order of 10MB/year that will be with us forever. The whole
git repo is currently <100MB.
--
Ticket URL: <http://trac.sagemath.org/ticket/10768#comment:11>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sage-trac.
For more options, visit https://groups.google.com/groups/opt_out.
