Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

PJ Eby writes: > By "clear", I mean "free of prior assumptions". Ah, well, I guess I've just run into a personal limitation. I can't imagine thinking that is "free of prior assumptions". Not my own, and not by others, either. So, unfortunately, I was left with the conventional opposition in t

Re: [Python-Dev] More compact dictionaries with faster iteration

On 10.12.12 05:30, Raymond Hettinger wrote: On Dec 9, 2012, at 10:03 PM, MRAB mailto:pyt...@mrabarnett.plus.com>> wrote: What happens when a key is deleted? Will that leave an empty slot in the entry array? Yes. See the __delitem__() method in the pure python implemention at http://code.active

Re: [Python-Dev] More compact dictionaries with faster iteration

On 10.12.12 09:48, Christian Heimes wrote: On the other hand every lookup and collision checks needs an additional multiplication, addition and pointer dereferencing: entry = entries_baseaddr + sizeof(PyDictKeyEntry) * idx I think that main slowdown will be in getting index from array with

Re: [Python-Dev] More compact dictionaries with faster iteration

On 10/12/12 01:44, Raymond Hettinger wrote: The current memory layout for dictionaries is unnecessarily inefficient. It has a sparse table of 24-byte entries containing the hash value, key pointer, and value pointer. Instead, the 24-byte entries should be stored in a dense table referenced by a

Re: [Python-Dev] Do more at compile time; less at runtime

Do know my registervm project? I try to emit better bytecode. See the implementation on http://hg.python.org/sandbox/registervm See for example the documentation: hg.python.org/sandbox/registervm/file/tip/REGISTERVM.txt I implemented other optimisation

Re: [Python-Dev] More compact dictionaries with faster iteration

Hi Raymond, On Mon, Dec 10, 2012 at 2:44 AM, Raymond Hettinger wrote: > Instead, the data should be organized as follows: > > indices = [None, 1, None, None, None, 0, None, 2] > entries = [[-9092791511155847987, 'timmy', 'red'], > [-8522787127447073495, 'barry', 'green']

Re: [Python-Dev] More compact dictionaries with faster iteration

Hi! 2012/12/10 Armin Rigo > Hi Raymond, > > On Mon, Dec 10, 2012 at 2:44 AM, Raymond Hettinger > wrote: > > Instead, the data should be organized as follows: > > > > indices = [None, 1, None, None, None, 0, None, 2] > > entries = [[-9092791511155847987, 'timmy', 'red'], > >

Re: [Python-Dev] [Distutils] Is is worth disentangling distutils?

On Mon, Dec 10, 2012 at 2:22 AM, Antonio Cavallo wrote: > Hi, > I wonder if is it worth/if there is any interest in trying to "clean" up > distutils: nothing in terms to add new features, just a *major* cleanup > retaining the exact same interface. > > > I'm not planning anything like *adding feat

Re: [Python-Dev] Conflicts [was Re: Keyword meanings [was: Accept just PEP-0426]]

On Sun, Dec 09, 2012 at 01:51:09PM +1100, Chris Angelico wrote: > On Sun, Dec 9, 2012 at 1:11 PM, Steven D'Aprano wrote: > > Why would a software package called "Spam" install a top-level module called > > "Jam" rather than "Spam"? Isn't the whole point of Python packages to solve > > this namespa

Re: [Python-Dev] Conflicts [was Re: Keyword meanings [was: Accept just PEP-0426]]

On Sun, Dec 09, 2012 at 12:48:45AM -0500, PJ Eby wrote: > > Do any of the distro folks know of a Python project tagged as > conflicting with another for their distro, where the conflict does > *not* involve any files in conflict? > In Fedora we do work to avoid most types of Conflicts (backportin

Re: [Python-Dev] More compact dictionaries with faster iteration

On Dec 10, 2012, at 08:48 AM, Christian Heimes wrote: >It's hard to predict how the extra CPU instructions are going to affect >the performance on different CPUs and scenarios. My guts tell me that >your proposal is going to perform better on modern CPUs with lots of L1 >and L2 cache and in an ave

Re: [Python-Dev] More compact dictionaries with faster iteration

Le Mon, 10 Dec 2012 10:40:30 +0100, Armin Rigo a écrit : > Hi Raymond, > > On Mon, Dec 10, 2012 at 2:44 AM, Raymond Hettinger > wrote: > > Instead, the data should be organized as follows: > > > > indices = [None, 1, None, None, None, 0, None, 2] > > entries = [[-9092791511155847987, '

Re: [Python-Dev] More compact dictionaries with faster iteration

On 10/12/12 20:40, Armin Rigo wrote: As a side note, your suggestion also enables order-preserving dictionaries: iter() would automatically yield items in the order they were inserted, as long as there was no deletion. People will immediately start relying on this "feature"... and be confused

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 10:48 AM, Antoine Pitrou wrote: > Le Mon, 10 Dec 2012 10:40:30 +0100, > Armin Rigo a écrit : >> Hi Raymond, >> >> On Mon, Dec 10, 2012 at 2:44 AM, Raymond Hettinger >> wrote: >> > Instead, the data should be organized as follows: >> > >> > indices = [None, 1, None, N

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

On Sun, Dec 9, 2012 at 10:38 PM, Andrew McNabb wrote: > On Fri, Dec 07, 2012 at 05:02:26PM -0500, PJ Eby wrote: >> If the packages have files in conflict, they won't be both installed. >> If they don't have files in conflict, there's nothing important to be >> informed of. If one is installing pe

Re: [Python-Dev] More compact dictionaries with faster iteration

Le Mon, 10 Dec 2012 11:16:49 -0500, PJ Eby a écrit : > On Mon, Dec 10, 2012 at 10:48 AM, Antoine Pitrou > wrote: > > Le Mon, 10 Dec 2012 10:40:30 +0100, > > Armin Rigo a écrit : > >> Hi Raymond, > >> > >> On Mon, Dec 10, 2012 at 2:44 AM, Raymond Hettinger > >> wrote: > >> > Instead, the data s

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 10:28 AM, Barry Warsaw wrote: > On Dec 10, 2012, at 08:48 AM, Christian Heimes wrote: > > >It's hard to predict how the extra CPU instructions are going to affect > >the performance on different CPUs and scenarios. My guts tell me that > >your proposal is going to perform

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

On Mon, Dec 10, 2012 at 3:27 AM, Stephen J. Turnbull wrote: > PJ Eby writes: > > > By "clear", I mean "free of prior assumptions". > > Ah, well, I guess I've just run into a personal limitation. I can't > imagine thinking that is "free of prior assumptions". Not my > own, and not by others, eit

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

On Fri, Dec 7, 2012 at 10:46 PM, PJ Eby wrote: > > In any case, as I said before, I don't have an issue with the fields > all being declared as being for informational purposes only. My issue > is only with recommendations for automated tool behavior that permit > one project's author to exercis

Re: [Python-Dev] More compact dictionaries with faster iteration

Am 10.12.2012 16:28, schrieb Barry Warsaw: > I'd be interested to see what effect this has on memory constrained platforms > such as many current ARM applications (mostly likely 32bit for now). Python's > memory consumption is an overheard complaint for folks developing for those > platforms. I h

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

On 10 December 2012 16:35, Toshio Kuratomi wrote: > I have no problems with Obsoletes, Conflicts, Requires, and Provides types > of fields are marked informational. In fact, there are many cases where > packages are overzealous in their use of Requires right now that cause > distributions to patc

Re: [Python-Dev] More compact dictionaries with faster iteration

Indeed, I had to change the dict tuning parameters to make dicts behave reasonably on the PS3. Interesting stuff indeed. > -Original Message- > From: Python-Dev [mailto:python-dev- > bounces+kristjan=ccpgames@python.org] On Behalf Of Barry Warsaw > Sent: 10. desember 2012 15:28 > To:

Re: [Python-Dev] More compact dictionaries with faster iteration

On Dec 10, 2012, at 05:39 PM, Christian Heimes wrote: >I had ARM platforms in mind, too. Unfortunately we don't have any ARM >platforms for testing and performance measurement. Even Trent's >Snakebite doesn't have ARM boxes. I've a first generation Raspberry Pi, >that's all. I have a few ARM mach

Re: [Python-Dev] More compact dictionaries with faster iteration

Hi Philip, On Mon, Dec 10, 2012 at 5:16 PM, PJ Eby wrote: > On the other hand, this would also make a fast ordered dictionary > subclass possible, just by not using the free list for additions, > combined with periodic compaction before adds or after deletes. Technically, I could see Python swit

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, 10 Dec 2012 16:06:23 +, krist...@ccpgames.com wrote: > Indeed, I had to change the dict tuning parameters to make dicts > behave reasonably on the PS3. > > Interesting stuff indeed. Out of both curiosity and a possible application of my own for the information, what values did you end

[Python-Dev] Is is worth disentangling distutils?

Hi, I wonder if is it worth/if there is any interest in trying to "clean" up distutils: nothing in terms to add new features, just a *major* cleanup retaining the exact same interface. I'm not planning anything like *adding features* or rewriting rpm/rpmbuild here, simply cleaning up that un

Re: [Python-Dev] Is is worth disentangling distutils?

On Mon, Dec 10, 2012 at 1:22 AM, Antonio Cavallo wrote: > I'm not into the py3 at all so I wonder how possibly it could fit/collide > into the big plan. > > Or I'll be wasting my time? If you're not doing it on Python 3 then you are wasting your time. _

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 1:01 PM, Armin Rigo wrote: > On Mon, Dec 10, 2012 at 5:16 PM, PJ Eby wrote: >> On the other hand, this would also make a fast ordered dictionary >> subclass possible, just by not using the free list for additions, >> combined with periodic compaction before adds or after d

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, 10 Dec 2012 11:53:17 -0500 Barry Warsaw wrote: > On Dec 10, 2012, at 05:39 PM, Christian Heimes wrote: > > >I had ARM platforms in mind, too. Unfortunately we don't have any ARM > >platforms for testing and performance measurement. Even Trent's > >Snakebite doesn't have ARM boxes. I've a

Re: [Python-Dev] Emacs users: hg-tools-grep

> > P.S. Who wants to abuse Jono and Matthew's copyright again and provide a > git version? > Oh, I do! I also feel weird about adding a copyright to this, but how will other people feel comfortable using it if I don't? Also I put it in github, in case people want to fix it: https://github.com/

Re: [Python-Dev] Emacs users: hg-tools-grep

On Dec 10, 2012, at 03:02 PM, Brandon W Maister wrote: >I also feel weird about adding a copyright to this, but how will other >people feel comfortable using it if I don't? Nice! I've been told that Emacs 23 is probably a minimum requirement because locate-dominating-file is missing in <= Emacs

Re: [Python-Dev] More compact dictionaries with faster iteration

On 12/10/2012 1:38 PM, PJ Eby wrote: On Mon, Dec 10, 2012 at 1:01 PM, Armin Rigo wrote: On Mon, Dec 10, 2012 at 5:16 PM, PJ Eby wrote: On the other hand, this would also make a fast ordered dictionary subclass possible, just by not using the free list for additions, combined with periodic com

Re: [Python-Dev] More compact dictionaries with faster iteration

On 11 December 2012 05:01, Armin Rigo wrote: > Hi Philip, > > On Mon, Dec 10, 2012 at 5:16 PM, PJ Eby wrote: > > On the other hand, this would also make a fast ordered dictionary > > subclass possible, just by not using the free list for additions, > > combined with periodic compaction before ad

[Python-Dev] Guido, Dropbox, and Python

For those who have not heard, Guido left Google Friday and starts at Dropbox in January. (I hope you enjoy the break in between ;-). https://twitter.com/gvanrossum/status/277126763295944705 https://tech.dropbox.com/2012/12/welcome-guido/ My question, Guido, is how this will affect Python develo

Re: [Python-Dev] Guido, Dropbox, and Python

On Mon, Dec 10, 2012 at 1:51 PM, Terry Reedy wrote: > For those who have not heard, Guido left Google Friday and starts at Dropbox > in January. (I hope you enjoy the break in between ;-). Thank you, I am already enjoying it! > https://twitter.com/gvanrossum/status/277126763295944705 > https://t

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 11:29 PM, Tim Delaney wrote: > On 11 December 2012 05:01, Armin Rigo wrote: >> >> Hi Philip, >> >> On Mon, Dec 10, 2012 at 5:16 PM, PJ Eby wrote: >> > On the other hand, this would also make a fast ordered dictionary >> > subclass possible, just by not using the free list

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 4:29 PM, Tim Delaney wrote: > Whilst I think Python should not move to ordered dictionaries everywhere, I > would say there is an argument (no pun intended) for making **kwargs a > dictionary that maintains insertion order *if there are no deletions*. Oooh. Me likey. The

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 5:14 PM, Andrew Svetlov wrote: > Please, no. dict and kwargs should be unordered without any assumptions. Why? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: htt

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 10:01 AM, Armin Rigo wrote: > Technically, I could see Python switching to ordered dictionaries > everywhere. Raymond's insight suddenly makes it easy for CPython and > PyPy, and at least Jython could use the LinkedHashMap class (although > this would need checking with Jy

Re: [Python-Dev] More compact dictionaries with faster iteration

On Dec 10, 2012, at 2:48 AM, Christian Heimes wrote: > On the other hand every lookup and collision checks needs an additional > multiplication, addition and pointer dereferencing: > > entry = entries_baseaddr + sizeof(PyDictKeyEntry) * idx Currently, the dict implementation allows alternati

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 3:13 PM, fwierzbi...@gmail.com wrote: > On Mon, Dec 10, 2012 at 10:01 AM, Armin Rigo wrote: >> Technically, I could see Python switching to ordered dictionaries >> everywhere. Raymond's insight suddenly makes it easy for CPython and >> PyPy, and at least Jython could use

Re: [Python-Dev] More compact dictionaries with faster iteration

On Dec 10, 2012, at 4:06 AM, Mark Shannon wrote: >> Instead, the 24-byte entries should be stored in a >> dense table referenced by a sparse table of indices. > > What minimum size and resizing factor do you propose for the entries array? There are many ways to do this. I don't know which is

Re: [Python-Dev] More compact dictionaries with faster iteration

On Dec 10, 2012, at 1:38 PM, PJ Eby wrote: > Without numbers it's hard to say for certain, but the advantage to > keeping ordered dictionaries a distinct type is that the standard > dictionary type can then get that extra bit of speed in exchange for > dropping the ordering requirement. I expec

Re: [Python-Dev] More compact dictionaries with faster iteration

On 10/12/12 23:39, Raymond Hettinger wrote: On Dec 10, 2012, at 4:06 AM, Mark Shannon mailto:m...@hotpy.org>> wrote: Instead, the 24-byte entries should be stored in a dense table referenced by a sparse table of indices. What minimum size and resizing factor do you propose for the entries

Re: [Python-Dev] More compact dictionaries with faster iteration

On Dec 10, 2012, at 7:04 PM, Mark Shannon wrote: >> Another approach is to pre-allocate the two-thirds maximum >> (This is simple and fast but gives the smallest space savings.) > > What do you mean by maximum? A dict with an index table size of 8 gets resized when it is two-thirds full, so th

Re: [Python-Dev] More compact dictionaries with faster iteration

On Mon, Dec 10, 2012 at 08:53:17AM -0800, Barry Warsaw wrote: > On Dec 10, 2012, at 05:39 PM, Christian Heimes wrote: > > >I had ARM platforms in mind, too. Unfortunately we don't have any ARM > >platforms for testing and performance measurement. Even Trent's > >Snakebite doesn't have ARM boxes. I

Re: [Python-Dev] More compact dictionaries with faster iteration

On Dec 10, 2012, at 4:02 AM, Serhiy Storchaka wrote: > On 10.12.12 05:30, Raymond Hettinger wrote: >> On Dec 9, 2012, at 10:03 PM, MRAB > > wrote: >>> What happens when a key is deleted? Will that leave an empty slot in >>> the entry array? >> Yes. See the __d

[Python-Dev] where is the python "import" implemented

Hi, everyone,     I am testing modifying the pyc file when it is imported. As I know, there is three situation:     1、runing in the python.exe eg: python.exe test.pyc     in this situation, I find the source on line 1983 in file pythonrun.c     2、import the pyc from a zip file   

Re: [Python-Dev] cpython: Using 'long double' to force this structure to be worst case aligned is no

On Tue, 11 Dec 2012 03:05:19 +0100 (CET) gregory.p.smith wrote: > Using 'long double' to force this structure to be worst case aligned is no > longer required as of Python 2.5+ when the gc_refs changed from an int (4 > bytes) to a Py_ssize_t (8 bytes) as the minimum size is 16 bytes. > > The us

Re: [Python-Dev] cpython: Using 'long double' to force this structure to be worst case aligned is no

On Tue, 11 Dec 2012 08:16:27 +0100 Antoine Pitrou wrote: > On Tue, 11 Dec 2012 03:05:19 +0100 (CET) > gregory.p.smith wrote: > > Using 'long double' to force this structure to be worst case aligned is no > > longer required as of Python 2.5+ when the gc_refs changed from an int (4 > > bytes) t