On Wed, Mar 28, 2018 at 1:03 PM, Serhiy Storchaka <storch...@gmail.com> wrote: > 28.03.18 21:39, Antoine Pitrou пише: >> I'd like to submit this PEP for discussion. It is quite specialized >> and the main target audience of the proposed changes is >> users and authors of applications/libraries transferring large amounts >> of data (read: the scientific computing & data science ecosystems). > > Currently I'm working on porting some features from cloudpickle to the > stdlib. For these of them which can't or shouldn't be implemented in the > general purpose library (like serializing local functions by serializing > their code objects, because it is not portable) I want to add hooks that > would allow to implement them in cloudpickle using official API. This would > allow cloudpickle to utilize C implementation of the pickler and unpickler.
There's obviously some tension here between pickle's use as a persistent storage format, and its use as a transient wire format. For the former, you definitely can't store code objects because there's no forwards- or backwards-compatibility guarantee for bytecode. But for the latter, transmitting bytecode is totally fine, because all you care about is whether it can be decoded once, right now, by some peer process whose python version you can control -- that's why cloudpickle exists. Would it make sense to have a special pickle version that the transient wire format users could opt into, that only promises compatibility within a given 3.X release cycle? Like version=-2 or version=pickle.NONPORTABLE or something? (This is orthogonal to Antoine's PEP.) -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
