Re: [Python-Dev] Tuning Python dicts

2010-04-13 Thread Daniel Stutzbach
On Sat, Apr 10, 2010 at 1:06 PM, Reid Kleckner r...@mit.edu wrote: Looking at dictnotes.txt, I can see that people have experimented with taking advantage of cache locality. I was wondering what benchmarks were used to glean these lessons before I write my own. Python obviously has very

Re: [Python-Dev] Tuning Python dicts

2010-04-13 Thread Reid Kleckner
On Tue, Apr 13, 2010 at 12:12 PM, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: I don't know what benchmarks were used to write dictnotes.txt, but moving forward I would recommend implementing your changes on trunk (i.e., Python 2.x) and running the Unladen Swallow Benchmarks, which

[Python-Dev] Tuning Python dicts

2010-04-10 Thread Reid Kleckner
Hey folks, I was looking at tuning Python dicts for a data structures class final project. I've looked through Object/dictnotes.txt, and obviously there's already a large body of work on this topic. My idea was to alter dict collision resolution as described in the hopscotch hashing paper[1].

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Antoine Pitrou
Reid Kleckner rnk at mit.edu writes: I think you're right about the number of collisions, though. CPython dicts use a pretty low load factor (2/3) to keep collision counts down. One of the major benefits cited in the paper is the ability to maintain performance in the face of higher load