Re: [Python-Dev] Impact of Namedtuple on startup time

17.07.17 15:43, Antoine Pitrou пише: Cost of creating a namedtuple has been identified as a contributor to Python startup time. Not only Python core and the stdlib, but any third-party library creating namedtuple classes (there are many of them). An issue was created for this:

Re: [Python-Dev] Impact of Namedtuple on startup time

On 07/17/2017 04:45 PM, Guido van Rossum wrote: Raymond agreed to reopen the issue. Everyone who's eager to redesign namedtuple, please go to python-ideas. Python Ideas thread started. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Impact of Namedtuple on startup time

On Jul 17, 2017 5:28 PM, "Steven D'Aprano" wrote: On Mon, Jul 17, 2017 at 09:31:20PM +, Brett Cannon wrote: > As for removing exec() as a goal, I'll back up Christian's point and the > one Steve made at the language summit that removing the use of exec() from > the

Re: [Python-Dev] Impact of Namedtuple on startup time

On Tue, Jul 18, 2017 at 01:17:24AM +0200, Giampaolo Rodola' wrote: > The extra memory overhead is a price I would be happy to pay considering > that collections.namedtuple is considerably slower than a plain tuple. > Other than the additional overhead on startup / import time, instantiation > is

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 8:16 PM, Barry Warsaw wrote: > .. > > class Foo(NamedTuple, fields = 'x,y,z'): > > ... > > > > Then the name is explicit and you get to add methods > > etc. if you want. > > Yes, I like how that reads. > > I would prefer class

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 09:31:20PM +, Brett Cannon wrote: > As for removing exec() as a goal, I'll back up Christian's point and the > one Steve made at the language summit that removing the use of exec() from > the critical path in Python is a laudable goal from a security perspective. I'm

Re: [Python-Dev] Impact of Namedtuple on startup time

On Jul 17, 2017, at 18:27, Greg Ewing wrote: > > Barry Warsaw wrote: >> namedtuple is great and clever, but it’s also a bit clunky. It has a weird >> signature and requires a made up type name. > > Maybe a metaclass could be used to make something > like this

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 6:27 PM, Greg Ewing wrote: > > > Maybe a metaclass could be used to make something > like this possible: > > >class Foo(NamedTuple, fields = 'x,y,z'): > ... > > If you think of it, collection.namedtuple *is* a metaclass. A simple

Re: [Python-Dev] Impact of Namedtuple on startup time

Raymond agreed to reopen the issue. Everyone who's eager to redesign namedtuple, please go to python-ideas. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 11:24 PM, Tim Peters wrote: > [Giampaolo Rodola' ] > > > > To be entirely honest, I'm not even sure why they need to be forcefully > > declared upfront in the first place, instead of just having a first-class > > function

Re: [Python-Dev] Impact of Namedtuple on startup time

On 07/17/2017 03:27 PM, Greg Ewing wrote: Barry Warsaw wrote: namedtuple is great and clever, but it’s also a bit clunky. It has a weird signature and requires a made up type name. Maybe a metaclass could be used to make something like this possible: class Foo(NamedTuple, fields =

Re: [Python-Dev] Impact of Namedtuple on startup time

On 07/17/2017 02:31 PM, Brett Cannon wrote: I vaguely remember some years ago someone proposing a patch that used metaclasses to avoid using exec() (I think it was to benefit PyPy or one of the JIT-backed interpreters). Would that work to remove the need for exec() while keeping the code in

Re: [Python-Dev] Impact of Namedtuple on startup time

On 07/17/2017 02:26 PM, Barry Warsaw wrote: namedtuple is great and clever, but it’s also a bit clunky. It has a weird > signature and requires a made up type name. It’s also rather unPythonic if > you want to support default arguments when creating namedtuple instances. > Maybe as you say,

Re: [Python-Dev] Impact of Namedtuple on startup time

Barry Warsaw wrote: namedtuple is great and clever, but it’s also a bit clunky. It has a weird signature and requires a made up type name. Maybe a metaclass could be used to make something like this possible: class Foo(NamedTuple, fields = 'x,y,z'): ... Then the name is explicit

Re: [Python-Dev] Impact of Namedtuple on startup time

Just for sake of completeness since people are talking about a namedtuple overhaul, I have a couple implementations here - https://github.com/jsbueno/extradict/blob/master/extradict/extratuple.py If any idea there can help inspiring someone, I will be happy. js -><- On 17 July 2017 at

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, 17 Jul 2017 at 13:28 Raymond Hettinger wrote: > > > On Jul 17, 2017, at 8:49 AM, Guido van Rossum wrote: > > > > One of the reasons to be wary of exec()/eval() other than the usual > security concerns is that in some Python implementations

Re: [Python-Dev] Impact of Namedtuple on startup time

[Giampaolo Rodola' ] > > To be entirely honest, I'm not even sure why they need to be forcefully > declared upfront in the first place, instead of just having a first-class > function (builtin?) written in C: > > >>> ntuple(x=1, y=0) > (x=1, y=0) > > ...or even a literal

Re: [Python-Dev] Impact of Namedtuple on startup time

namedtuple is great and clever, but it’s also a bit clunky. It has a weird signature and requires a made up type name. It’s also rather unPythonic if you want to support default arguments when creating namedtuple instances. Maybe as you say, a lot of the typical use cases for namedtuples

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 11:07 PM, Petr Viktorin wrote: > On 07/17/2017 10:31 PM, Giampaolo Rodola' wrote: > >> I completely agree. I love namedtuples but I've never been too happy >> about the additional overhead vs. plain tuples (both for creation and >> attribute access

Re: [Python-Dev] Impact of Namedtuple on startup time

On 07/17/2017 10:31 PM, Giampaolo Rodola' wrote: I completely agree. I love namedtuples but I've never been too happy about the additional overhead vs. plain tuples (both for creation and attribute access times), to the point that I explicitly avoid to use them in certain circumstances (e.g. a

Re: [Python-Dev] Impact of Namedtuple on startup time

On 2017-07-17 21:46, MRAB wrote: On 2017-07-17 21:31, Giampaolo Rodola' wrote: I completely agree. I love namedtuples but I've never been too happy about the additional overhead vs. plain tuples (both for creation and attribute access times), to the point that I explicitly avoid to use them

Re: [Python-Dev] Impact of Namedtuple on startup time

On 2017-07-17 21:31, Giampaolo Rodola' wrote: I completely agree. I love namedtuples but I've never been too happy about the additional overhead vs. plain tuples (both for creation and attribute access times), to the point that I explicitly avoid to use them in certain circumstances (e.g. a

Re: [Python-Dev] Impact of Namedtuple on startup time

My apologies, I misunderstood what had been proposed (and rejected). So it sounds like the _source is a pre-requisite for the current exec-based implementation, but the proposal is to replace with a non-exec-based implementation, meaning _source would no longer be needed for the module to work

Re: [Python-Dev] Impact of Namedtuple on startup time

I completely agree. I love namedtuples but I've never been too happy about the additional overhead vs. plain tuples (both for creation and attribute access times), to the point that I explicitly avoid to use them in certain circumstances (e.g. a busy loop) and only for public end-user APIs

Re: [Python-Dev] Impact of Namedtuple on startup time

> On Jul 17, 2017, at 8:49 AM, Guido van Rossum wrote: > > One of the reasons to be wary of exec()/eval() other than the usual security > concerns is that in some Python implementations they have a high overhead to > initialize the parser and compiler. (Even in CPython it's

Re: [Python-Dev] Impact of Namedtuple on startup time

> On Jul 17, 2017, at 8:49 AM, Guido van Rossum wrote: > > The approach of generating source code and exec()ing it, is a cool > demonstration of Python's expressive power, but it's always been my sense > that whenever we encounter a popular idiom that uses exec() and

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 9:45 GMT-07:00 Steven D'Aprano : > On Mon, Jul 17, 2017 at 02:43:19PM +0200, Antoine Pitrou wrote: > > > > Hello, > > > > Cost of creating a namedtuple has been identified as a contributor to > > Python startup time. Not only Python core and the stdlib, but any >

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 02:43:19PM +0200, Antoine Pitrou wrote: > > Hello, > > Cost of creating a namedtuple has been identified as a contributor to > Python startup time. Not only Python core and the stdlib, but any > third-party library creating namedtuple classes (there are many of > them).

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 18:13 GMT+02:00 Gregory P. Smith : > I get that namedtuple ._source is a public API. We may need to keep it. If > so, that just means revisiting lazily generating it as a property - > issue19640. I agree. Technically speaking, optimizing namedtuple doesn't have to mean

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 8:00 AM Raymond Hettinger < raymond.hettin...@gmail.com> wrote: > > > On Jul 17, 2017, at 6:31 AM, Antoine Pitrou wrote: > > > >> I think I understand well enough to say something intelligent… > >> > >> While actual references to _source are likely

Re: [Python-Dev] Impact of Namedtuple on startup time

I am firmly with Antoine here. The cumulative startup time of large Python programs is a serious problem and namedtuple is one of the major contributors -- especially because it is so convenient that it is ubiquitous. The approach of generating source code and exec()ing it, is a cool demonstration

Re: [Python-Dev] Impact of Namedtuple on startup time

Makes sense. Thanks. S Steve Holden On Mon, Jul 17, 2017 at 4:29 PM, Raymond Hettinger < raymond.hettin...@gmail.com> wrote: > > > On Jul 17, 2017, at 8:22 AM, Steve Holden wrote: > > > > My only question is "what's a variable called _source doing in the > public API?" >

Re: [Python-Dev] Impact of Namedtuple on startup time

> On Jul 17, 2017, at 8:22 AM, Steve Holden wrote: > > My only question is "what's a variable called _source doing in the public > API?" The convention for named tuple hnas been for all the methods and attributes to be prefixed with an underscore so that the names won't

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 3:59 PM, Raymond Hettinger < raymond.hettin...@gmail.com> wrote: > I really don't want to throw away these benefits to save a couple of > milliseconds. As Nick Coghlan recently posted, "Speed isn't everything, > and it certainly isn't adequate justification for breaking

Re: [Python-Dev] Impact of Namedtuple on startup time

On 2017-07-17 14:43, Antoine Pitrou wrote: > So my take is: > > 1) Usage of "_source" in open source code (as per the search above) > seems non-existent. > > 2) If the primary intent of "_source" is to show-case how to write a > tuple subclass, well, why not write a recipe or tutorial somewhere?

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 16:56 GMT+02:00 Facundo Batista : > My experience inside Canonical is that golang stole a lot of "codebase > share" from Python, and (others and mine) talks hit two walls, mainly: > one is memory consumption, and the other is startup time. > > So yes, startup

Re: [Python-Dev] Impact of Namedtuple on startup time

On Jul 17, 2017, at 10:59, Raymond Hettinger wrote: > > ISTM this issue is being pressed by micro-optimizers who are being very > aggressive and not responding to actual user needs (it is more an invented > issue than a real one). Named tuple has been around for

Re: [Python-Dev] Impact of Namedtuple on startup time

> On Jul 17, 2017, at 6:31 AM, Antoine Pitrou wrote: > >> I think I understand well enough to say something intelligent… >> >> While actual references to _source are likely rare (certainly I’ve never >> used it), my understanding is that the way namedtuple works is to >>

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, Jul 17, 2017 at 9:43 AM, Antoine Pitrou wrote: > As for 2), startup time is actually a very important consideration > nowadays, both for small scripts *and* for interactive use with the > now very wide-spread use of Jupyter Notebooks. A 1 ms. cost when > importing a

Re: [Python-Dev] Impact of Namedtuple on startup time

Le 17/07/2017 à 15:26, Isaac Morland a écrit : > > I think I understand well enough to say something intelligent… > > While actual references to _source are likely rare (certainly I’ve never > used it), my understanding is that the way namedtuple works is to > construct _source, and then exec

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, 17 Jul 2017 15:03:26 +0200 Ivan Levkivskyi wrote: > Interesting coincidence, just two days ago I have heard that a team at one > large company completely abandoned namedtuple because of the creation time > problem. > > Concerning _source, why it is not possible to

Re: [Python-Dev] Impact of Namedtuple on startup time

On 17 July 2017 at 08:43, Antoine Pitrou wrote: > > Hello, > > Cost of creating a namedtuple has been identified as a contributor to > Python startup time. Not only Python core and the stdlib, but any > third-party library creating namedtuple classes (there are many of >

Re: [Python-Dev] Impact of Namedtuple on startup time

Interesting coincidence, just two days ago I have heard that a team at one large company completely abandoned namedtuple because of the creation time problem. Concerning _source, why it is not possible to make it a property so that all the string formatting will happen on request, thus saving

Re: [Python-Dev] Impact of Namedtuple on startup time

On Mon, 17 Jul 2017 14:43:19 +0200 Antoine Pitrou wrote: > Hello, > > Cost of creating a namedtuple has been identified as a contributor to > Python startup time. Imprecise wording: that's the cost of creating a namedtuple *class*, i.e. anytime someone writes `MyClass =

[Python-Dev] Impact of Namedtuple on startup time

Hello, Cost of creating a namedtuple has been identified as a contributor to Python startup time. Not only Python core and the stdlib, but any third-party library creating namedtuple classes (there are many of them). An issue was created for this: https://bugs.python.org/issue28638 Raymond