On 4/21/2014 9:23 AM, Stefan Schwarzer wrote:
Hi,
Recently, I got a request [1] to support pickling of
`FTPHost` instances in my `ftplib` library.
I explained in the ticket why I think it's a bad idea and
now want to make explicit that `FTPHost` objects can't be
pickled. The usual way to do this seems to be defining a
`__getstate__` method and raise an exception there.
Now the question is "which exception?" and my research left
me a bit confused. I didn't find a recommendation for this
on the web, not even in the Python documentation for the
`pickle` module [2]. The only hint is that the documentation
states:
"""
The pickle module defines three exceptions:
exception pickle.PickleError
Common base class for the other pickling exceptions. It
inherits Exception.
exception pickle.PicklingError
Error raised when an unpicklable object is encountered
by Pickler. It inherits PickleError.
I am going to read this as 'unpicklable as determined by Pickler',
possibly due to a bug in the objects methods.
Refer to What can be pickled and unpickled? to learn
what kinds of objects can be pickled.
exception pickle.UnpicklingError
Error raised when there is a problem unpickling an
object, such as a data corruption or a security
violation. It inherits PickleError.
Note that other exceptions may also be raised during
unpickling, including (but not necessarily limited to)
AttributeError, EOFError, ImportError, and IndexError.
"""
This sounds like unpicklable objects should raise a
`PicklingError`. Indeed, if I do this, `pickle.dumps`
gives me (my own) `PicklingError`:
Python 3.3.2 (default, Nov 8 2013, 13:38:57)
[GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> class X:
... def __getstate__(self):
... raise pickle.PicklingError("can't pickle X objects")
...
>>> x = X()
>>> pickle.dumps(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __getstate__
_pickle.PicklingError: can't pickle X objects
What now confuses me is that most, if not all, objects from
the standard library that aren't picklable raise a
`TypeError` when I try to pickle them:
>>> fobj = open("/etc/passwd")
>>> pickle.dumps(fobj)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: cannot serialize '_io.TextIOWrapper' object
>>> import socket
>>> s = socket.socket()
>>> pickle.dumps(s)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.3/socket.py", line 116, in __getstate__
raise TypeError("Cannot serialize socket object")
TypeError: Cannot serialize socket object
I would just copy the socket.__getstate__, with a revised message.
So the documentation for the `pickle` module (to me) implies
I should raise a `PicklingError` while the standard library
usually seems to use a `TypeError`. When I grep through the
library files for `PicklingError`, I get very few hits, most
of them in `pickle.py`:
This suggests that the exception is intended for use by Pickler, and not
by user code. The reason is pretty subtle and not explained in the doc.
See below.
$ find /usr/lib64/python3.3 -name "*.py" -exec grep -H PicklingError {} \;
/usr/lib64/python3.3/site-packages/numpy/numarray/session.py: except
(pickle.PicklingError, TypeError, SystemError):
/usr/lib64/python3.3/pickle.py:__all__ = ["PickleError", "PicklingError",
"UnpicklingError", "Pickler",
/usr/lib64/python3.3/pickle.py:class PicklingError(PickleError):
/usr/lib64/python3.3/pickle.py: raise
PicklingError("Pickler.__init__() was not called by "
/usr/lib64/python3.3/pickle.py: raise PicklingError("Can't
pickle %r object: %r" %
/usr/lib64/python3.3/pickle.py: raise PicklingError("%s must return
string or tuple" % reduce)
/usr/lib64/python3.3/pickle.py: raise PicklingError("Tuple returned by
%s must have "
/usr/lib64/python3.3/pickle.py: raise PicklingError("args from
save_reduce() should be a tuple")
/usr/lib64/python3.3/pickle.py: raise PicklingError("func from
save_reduce() should be callable")
Most of the above look like bugs in code intended to work with Pickle.
In other words, if you user gets a TypeError, the user has made a
mistake by trying to pickle your object. If you tried to make your
object picklable, and users got PicklingError, that would indicate a bug
in your class code, not the users' use of your class.
/usr/lib64/python3.3/pickle.py: raise PicklingError(
/usr/lib64/python3.3/pickle.py: raise PicklingError(
/usr/lib64/python3.3/pickle.py: raise PicklingError(
/usr/lib64/python3.3/pickle.py: raise PicklingError(
/usr/lib64/python3.3/pickle.py: raise PicklingError(
/usr/lib64/python3.3/idlelib/rpc.py: except pickle.PicklingError:
Which exception would you raise for an object that can't be
pickled and why?
TypeError, as explained above.
[1] http://ftputil.sschwarzer.net/trac/ticket/75
[2] https://docs.python.org/3.4/library/pickle.html
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
