Re: [Python-Dev] Tuning Python dicts

2010-04-13 Thread Reid Kleckner
On Tue, Apr 13, 2010 at 12:12 PM, Daniel Stutzbach wrote: > I don't know what benchmarks were used to write dictnotes.txt, but moving > forward I would recommend implementing your changes on trunk (i.e., Python > 2.x) and running the Unladen Swallow Benchmarks, which you can get from the > link be

Re: [Python-Dev] Tuning Python dicts

2010-04-13 Thread Daniel Stutzbach
On Sat, Apr 10, 2010 at 1:06 PM, Reid Kleckner wrote: > Looking at dictnotes.txt, I can see that people have experimented with > taking advantage of cache locality. I was wondering what benchmarks > were used to glean these lessons before I write my own. Python > obviously has very particular w

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Terry Reedy
I may be missing the point, but ISTM that the assumption of this approach is that there are often collisions in the hash table. I think that assumption is false; at least, I recommend to validate that assumption before proceeding. It's just an experiment for a class, not something I am (yet) s

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Reid Kleckner
On Sat, Apr 10, 2010 at 4:40 PM, Antoine Pitrou wrote: > Reid Kleckner mit.edu> writes: >> >> I think you're right about the number of collisions, though.  CPython >> dicts use a pretty low load factor (2/3) to keep collision counts >> down.  One of the major benefits cited in the paper is the ab

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Antoine Pitrou
Antoine Pitrou pitrou.net> writes: > > Why wouldn't it matter? Hash collisions still involve more CPU work, even though > if you're not access memory a lot. (sorry for the awful grammar in the last sentence) ___ Python-Dev mailing list Python-Dev@py

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Antoine Pitrou
Reid Kleckner mit.edu> writes: > > I think you're right about the number of collisions, though. CPython > dicts use a pretty low load factor (2/3) to keep collision counts > down. One of the major benefits cited in the paper is the ability to > maintain performance in the face of higher load fa

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Reid Kleckner
On Sat, Apr 10, 2010 at 2:57 PM, "Martin v. Löwis" wrote: >> Any other advice would also be helpful. > > I may be missing the point, but ISTM that the assumption of this > approach is that there are often collisions in the hash table. I think > that assumption is false; at least, I recommend to va

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Martin v. Löwis
> Any other advice would also be helpful. I may be missing the point, but ISTM that the assumption of this approach is that there are often collisions in the hash table. I think that assumption is false; at least, I recommend to validate that assumption before proceeding. There are, of course, ca

[Python-Dev] Tuning Python dicts

2010-04-10 Thread Reid Kleckner
Hey folks, I was looking at tuning Python dicts for a data structures class final project. I've looked through Object/dictnotes.txt, and obviously there's already a large body of work on this topic. My idea was to alter dict collision resolution as described in the hopscotch hashing paper[1]. I