[issue6477] Pickling of NoneType raises PicklingError

[Alexandre Vassalotti]

Alexandre Vassalotti added the comment:

I thought it over. I don't think the fix is appropriate for 2.x, as it seems 
closer to being an extra feature than a bug fix.

Thanks Antoine for calling me out on this one.

--
resolution:  - wont fix
stage: patch review - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Roundup Robot]

Roundup Robot added the comment:

New changeset 16eba94d3cfe by Alexandre Vassalotti in branch '3.3':
Issue #6477: Added support for pickling the types of built-in singletons.
http://hg.python.org/cpython/rev/16eba94d3cfe

New changeset ff56f48b3277 by Alexandre Vassalotti in branch 'default':
Issue #6477: Merge with 3.3.
http://hg.python.org/cpython/rev/ff56f48b3277

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Alexandre Vassalotti]

Changes by Alexandre Vassalotti alexan...@peadrop.com:


--
assignee: docs@python - alexandre.vassalotti
resolution:  - fixed
stage: needs patch - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Roundup Robot]

Roundup Robot added the comment:

New changeset fbb97f6eb3b3 by Alexandre Vassalotti in branch '2.7':
Issue #6477: Added pickling support for singletons and their types.
http://hg.python.org/cpython/rev/fbb97f6eb3b3

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Antoine Pitrou]

Antoine Pitrou added the comment:

Uh... Your commits make PyNone_Type and PyNotImplemented_Type public APIs, 
which I don't think is ok, especially not in bugfix releases.

Also, it would be nice to post patches on the tracker before committing (not 
for trivial patches of course, but it's generally nicer).

--
nosy: +pitrou
status: closed - open

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Alexandre Vassalotti]

Alexandre Vassalotti added the comment:

Would you be okay with removing the static declaration of PyNotImplemented_Type 
and PyNone_Type if we prefix their name with an underscore? There isn't any 
other way to fix this without making the types linkable.

I might revert the 2.7 change anyway as it broke test_xpickle and it doesn't 
look easy to fix.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Roundup Robot]

Roundup Robot added the comment:

New changeset d964d7023aa4 by Alexandre Vassalotti in branch '2.7':
Issue #6477: Revert fbb97f6eb3b3 as it broke test_xpickle.
http://hg.python.org/cpython/rev/d964d7023aa4

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Antoine Pitrou]

Antoine Pitrou added the comment:

Yeah, adding an underscore is a fine way of dealing with this if the symbol has 
to be exported.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Roundup Robot]

Roundup Robot added the comment:

New changeset 9fcba15d7685 by Alexandre Vassalotti in branch '3.3':
Issue #6477: Keep PyNotImplemented_Type and PyNone_Type private.
http://hg.python.org/cpython/rev/9fcba15d7685

New changeset 7d6c27fa7f32 by Alexandre Vassalotti in branch 'default':
Issue #6477: Merge with 3.3.
http://hg.python.org/cpython/rev/7d6c27fa7f32

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Alexandre Vassalotti]

Alexandre Vassalotti added the comment:

Antoine, are you okay with applying this fix to 2.7? Or should we just mark 
this as a won't fix?

--
keywords: +patch
priority: low - normal
resolution: fixed - 
stage: committed/rejected - patch review
versions:  -Python 3.2, Python 3.3, Python 3.4
Added file: 
http://bugs.python.org/file32915/backport_pickle_singleton_types_fix.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Ezio Melotti]

Changes by Ezio Melotti ezio.melo...@gmail.com:


--
keywords: +easy
stage:  - needs patch
versions: +Python 3.3, Python 3.4 -Python 2.6, Python 3.0, Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Jason R. Coombs]

Jason R. Coombs jar...@jaraco.com added the comment:

I've encountered a use-case where the need to pickle NoneType is more relevant 
(and difficult to work around).

We have a strongly-type Limited Python language, LimPy, which is based on 
Python (https://bitbucket.org/yougov/limpy). This parser takes, as part of its 
initialization arguments, a type specification (indicating which types are 
allowed and not allowed). In some cases, the return value may be `None`, in 
which case the specification says the type must be `NoneType`.

We're attempting to run this parser in a separate process, using the 
multiprocessing module, which requires that the arguments passed to and from 
the parser be pickleable. Unfortunately, because `NoneType` is in the type 
specification, it cannot be passed to the parser.

Here's an example of one such type specification (from the test suite):

class SomeFunctionNamespace:
@signature([IListType], [], None, ListOfInt)
def listcount(self, l):
return range(len(l))

@signature([IListType], [], None, int)
def listlen(self, l):
return len(l)

# ...

@signature([IIntType], [IStringType], IStringType, NoneType)
def givespec(self, *args):
for a in args:
print a


Note that we can pickle `str` and `int` just fine. Only type(None) fails.

It would be possible to re-write the entire LimPy system (and its child 
projects) to use a different object where currently NoneType is used, though 
NoneType is precisely the right thing to be used here except that it can't be 
pickled.

Since type(None) is a fundamental Python type, it strikes me as a bug that it's 
not pickleable, though I concede that it's also reasonable to interpret this 
issue as a feature request (as it's never been pickleable).

Nick makes some good comments that pickling of NoneType should be done right, 
but other than that, there haven't been any reasons why in principle NoneType 
should not be pickleable.

NoneType is more akin to `str` and `int` which are pickleable than it is to a 
function or type(NotImplemented), and this is evident by the way that LimPy 
uses it (before there was any consideration for pickleability).

Therefore, I propose we reconsider that NoneType should be made pickleable.

--
nosy: +jaraco

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Alexander Belopolsky]

Alexander Belopolsky belopol...@users.sourceforge.net added the comment:

Ellipsis and NotImplemented are different from None because they are not 
pickleable themselves.  The None situation is more similar to that of say 
module level functions:


 def f(): pass
... 
 dumps(f)  # works
b'\x80\x03c__main__\nf\nq\x00.'
 dumps(type(f))  # does not work
Traceback (most recent call last):
  ..
_pickle.PicklingError: Can't pickle class 'function': attribute lookup 
builtins.function failed


I think documentation at 
http://docs.python.org/dev/py3k/library/pickle.html#what-can-be-pickled-and-unpickled

Can be improved in two ways:

1. s/types/objects/ in The following types can be pickled

2. Add a note that type(x) can be pickled does not follow from x can be 
pickled.  The object type(x) must itself fall into one on the listed 
categories.

3. built-in functions defined at the top level of a module should probably be 
built-in classes or functions defined at the top level of the builtins module

4. instances of such classes should be instances of picklable classes

5. __setstate__() is picklable should be output of __getstate__() is 
picklable

--
nosy: +belopolsky

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Nick Coghlan]

Nick Coghlan ncogh...@gmail.com added the comment:

Given that dumps(type(Ellipsis)) and dumps(type(NotImplemented)) don't
work either, I am reclassifying this as a documentation bug.

The thing about the types of these three objects (None, Ellipsis,
NotImplemented) is that they are all designed to be singletons and they
all disallow creation of new instances of them.

 type(None)()
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: cannot create 'NoneType' instances
 type(NotImplemented)()
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: cannot create 'NotImplementedType' instances
 type(Ellipsis)()
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: cannot create 'ellipsis' instances

This does mean None has to be special cased if pickling (object,
type(object)) pairs, but passing types through pickle strikes me as a
dubious enough exercise that it would take a fairly convincing use case
to accept an RFE to make NoneType pickleable (note that any such patch
would have to make sure that doing loads() on such a pickle didn't
manage to create a second copy of NoneType or None).

--
assignee:  - georg.brandl
components: +Documentation -Library (Lib)
nosy: +georg.brandl, ncoghlan

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Alexandre Vassalotti]

Alexandre Vassalotti alexan...@peadrop.com added the comment:

I agree with Nick.

And if you really want to, you could hack a Pickler subclass to support
NoneType:

import io
import pickle

class XPickler(pickle.Pickler):
  def persistent_id(self, obj):
if obj is type(None):
  return NoneType
return None

class XUnpickler(pickle.Unpickler):
  def persistent_load(self, persistent_id):
if persistent_id == NoneType:
  return type(None)

def dumps(obj):
f = io.BytesIO()
XPickler(f).dump(obj)
return f.getvalue()

def loads(s):
return XUnpickler(io.BytesIO(s)).load()

Not super elegant, but it works:

 loads(dumps([type(None), None]))
[type 'NoneType', None]

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Hagen Fürstenau]

Hagen Fürstenau hfuerste...@gmx.net added the comment:

 but I think it is a bug

I think it is either a feature request (make NoneType picklable) or a
documentation issue (document that it's not).

--
nosy: +hagen

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[Alexandre Vassalotti]

Alexandre Vassalotti alexan...@peadrop.com added the comment:

I don't see why you want to pickle NoneType. Do you have a proper
use-case for this, or are you just playing around with pickle?

--
nosy: +alexandre.vassalotti
priority:  - low

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[July Tikhonov]

July Tikhonov july.t...@gmail.com added the comment:

No, my program failed on this. It was not a big problem to manage this, 
but I think it is a bug. And it isn't documented (or I can't find it).

Other built-in types have no such problem. Is there something special 
with NoneType?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6477] Pickling of NoneType raises PicklingError

[July Tikhonov]

New submission from July Tikhonov july.t...@gmail.com:

Python 3.2a0 (py3k:73749M, Jul  1 2009, 23:17:59)
[GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
Type help, copyright, credits or license for more information.
 import pickle
[40072 refs]
 pickle.dumps(type(None))
Traceback (most recent call last):
  File stdin, line 1, in module
  File /usr/local/lib/python3.2/pickle.py, line 1358, in dumps
Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
_pickle.PicklingError: Can't pickle class 'NoneType': attribute 
lookup builtins.NoneType failed
[40137 refs]


--
components: Library (Lib)
messages: 90496
nosy: July
severity: normal
status: open
title: Pickling of NoneType raises PicklingError
type: behavior
versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6477
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com