[issue17941] namedtuple should support fully qualified name for more portable pickling

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
resolution:  -> fixed
status: open -> closed
versions: +Python 3.6 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Roundup Robot]

Roundup Robot added the comment:

New changeset c8851a0ce7ca by Raymond Hettinger in branch 'default':
Issue #17941: Add a *module* parameter to collections.namedtuple()
https://hg.python.org/cpython/rev/c8851a0ce7ca

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 That will not work correctly if the module name has a dot in it.

Pickling qualified names with arbitrary number of dots is supported in 3.4 with 
protocol 4 and in 3.5 with all protocols (backward compatibly).

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[ctismer]

ctismer added the comment:

Ok, I almost forgot about this because I thought my idea
was not considered, and wondered if I should keep that code online.

It is still around, and I could put it into an experimental branch
if someone asks for it.

Missing pull-request on python.org.

--
nosy: +ctismer

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Raymond Hettinger]

Changes by Raymond Hettinger raymond.hettin...@gmail.com:


--
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Mark Lawrence]

Mark Lawrence added the comment:

Just slipped under the radar?

--
nosy: +BreamoreBoy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Josh Rosenberg]

Josh Rosenberg added the comment:

I was already thinking of the same solution Christian.Tismer proposed before I 
reached his post. namedtuples seem cleaner if they naturally act as singletons, 
so (potentially whether or not pickling is involved) declaring a namedtuple 
with the same name and fields twice returns the same class. Removes the need to 
deal with module qualified names, and if pickle can be made to support it, the 
namedtuple class itself could be pickled uniquely in such a way that it could 
be recreated by someone else who didn't even have a local definition of the 
namedtuple, the module that defines it, etc.

--
nosy: +josh.rosenberg

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Raymond Hettinger]

Raymond Hettinger added the comment:

 Just slipped under the radar?

Nope, I'll get to it before long.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Christian Tismer]

Christian Tismer added the comment:

I would like to make an additional suggestion.
(and I implemented this yesterday):

Native namedtuple (not a derived class) can be made much simpler to handle
when no module and class machinery is involved at all.

The way I implemented it has no need for sys._getframes, and it does
not store a reference to the class at all.

The advantage of doing so is that this maximizes the compatibility
with ordinary tuples. Ordinary tuples have no pickling issue at all,
and this way the named tuple should behave as well.

My implementation re-creates the namedtuple classes on the fly by a
function in __reduce_ex__. There is no speed penalty for this because
of caching the classes by their unique name and set of field names.

This is IMHO the way things should work:
A namedtuple replaces a builtin type, so it has the same pickling
behavior: nothing needed.

Rationale:
tuples are used everywhere and dynamically. namedtuple should be as
compatible to that as possible. By having to specify  a module etc., this 
dynamic is partially lost.

Limitation:
When a class is derived from namedtuple, pickling support is no longer
automated. This is compatible with ordinary tuples as well.

Cheers - Chris

--
hgrepos: +198
nosy: +Christian.Tismer

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Stefan Drees]

Changes by Stefan Drees ste...@drees.name:


--
nosy: +dilettant

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Raymond Hettinger]

Changes by Raymond Hettinger raymond.hettin...@gmail.com:


--
keywords: +patch
Added file: http://bugs.python.org/file30243/nt_module.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Guido van Rossum]

Guido van Rossum added the comment:

I'd like to propose one slight tweak to the patch.  (Also to enum.py.)

If no module name was passed and _getframe() fails, can you set the __module__ 
attribute to something that will cause pickling the object to fail?  That would 
be much better than letting it pickle but then being unable to unpickle it.

(TBH I'm not sure how this should be accomplished, and if it can't, forget that 
I said it.  But it would be nice.)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Richard Oudkerk]

Richard Oudkerk added the comment:

When pickling a class (or instance of a class) there is already a check 
that the invariant

 getattr(sys.modules[cls.__module__], cls.__name__) == cls

holds.

  import pickle
  class A: pass
...
  A.__module__ = 'nonexistent'
  pickle.dumps(A())
Traceback (most recent call last):
   File stdin, line 1, in module
_pickle.PicklingError: Can't pickle class 'nonexistent.A': import of 
module 'nonexistent' failed

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Guido van Rossum]

Guido van Rossum added the comment:

Great, forget I said anything then.

LGTM to the patch, feel free to commit (with update to Misc/NEWS please).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

LGTM too.  Needs test and docs.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Eli Bendersky]

Eli Bendersky added the comment:

A question that came up while reviewing the new enum code: module or 
module_name for this extra argument? The former is shorter, but the latter is 
more correct in a way.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Guido van Rossum]

Guido van Rossum added the comment:

Module, please. The class attribute is also called __module__ after all.

On Friday, May 10, 2013, Eli Bendersky wrote:


 Eli Bendersky added the comment:

 A question that came up while reviewing the new enum code: module or
 module_name for this extra argument? The former is shorter, but the
 latter is more correct in a way.

 --

 ___
 Python tracker rep...@bugs.python.org javascript:;
 http://bugs.python.org/issue17941
 ___


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Eli Bendersky]

Eli Bendersky added the comment:

On Fri, May 10, 2013 at 7:02 AM, Guido van Rossum rep...@bugs.python.orgwrote:


 Guido van Rossum added the comment:

 Module, please. The class attribute is also called __module__ after all.


Makes sense. Thanks.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Raymond Hettinger]

Changes by Raymond Hettinger raymond.hettin...@gmail.com:


--
assignee:  - rhettinger
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Richard Oudkerk]

Richard Oudkerk added the comment:

 If the name is a qualified dotted name, it will be split and the first 
 part becomes the __module__.

That will not work correctly if the module name has a dot in it.

--
nosy: +sbt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Eric V. Smith]

Eric V. Smith added the comment:

I agree it should work the same as Enum, and I agree it should be possible to 
supply the module name. But wouldn't it be cleaner as:

Point = namedtuple('Point', 'x y z', module=__name__)

rather than:

Point = namedtuple(__name__ + '.Point', 'x y z')

?

--
nosy: +eric.smith

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Barry A. Warsaw]

Changes by Barry A. Warsaw ba...@python.org:


--
nosy: +barry

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Guido van Rossum]

Guido van Rossum added the comment:

Agreed with Eric. We're already modifying PEP 435 to do it that way.

--
nosy: +gvanrossum

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17941] namedtuple should support fully qualified name for more portable pickling

[Eli Bendersky]

New submission from Eli Bendersky:

[this came up as part of the Enum discussions. Full details in this thread: 
http://mail.python.org/pipermail/python-dev/2013-May/126076.html]

namedtuple currently uses this code to obtain the __module__ for the class it 
creates dynamically so that pickling works:

  result.__module__ = _sys._getframe(1).f_globals.get('__name__', '__main__')

This may not work correctly on all Python implementations, for example 
IronPython.

To support some way to pickle on all implementations, namedtuple should support 
a fully qualified name for the class: 

  Point = namedtuple('mymodule.Point', ['x', 'y'])

If the name is a qualified dotted name, it will be split and the first part 
becomes the __module__.

--
components: Library (Lib)
messages: 188748
nosy: eli.bendersky, ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: namedtuple should support fully qualified name for more portable pickling
type: behavior
versions: Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17941
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com