Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread John Machin
On Dec 24, 7:04 pm, "Malcolm Greene" wrote: > Hi Roger, > > By very large dictionary, I mean about 25M items per dictionary. Each > item is a simple integer whose value will never exceed 2^15. In Python-speak about dictionaries, an "item" is a tuple (key, value). I presume from the gross differen

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread Terry Reedy
Marc 'BlackJack' Rintsch wrote: On Wed, 24 Dec 2008 03:23:00 -0500, python wrote: collection, I don't see the advantage of using an iterator or a list. I'm sure I'm missing a subtle point here :) `keys()` creates a list in memory, `iterkeys()` does not. With ``set(dict.keys())`` there is a

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread Scott David Daniels
Gabriel Genellina wrote: En Wed, 24 Dec 2008 06:23:00 -0200, escribió: ... k1 = set(dict1.iterkeys()) You've got an excelent explanation from Marc Rintsch. (Note that in Python 3.0 keys() behaves as iterkeys() in previous versions, so the above code is supposed to be written in Python 2.x)

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread bearophileHUGS
Peter Otten: > >>> a = dict(a=1, b=2, c=3) > >>> b = dict(b=2, c=30, d=4) > >>> dict(set(a.iteritems()) ^ set(b.iteritems())) For larger sets this may be better, may avoid the creation of the second set: dict(set(a.iteritems()).symmetric_difference(b.iteritems())) Bye, bearophile -- http://mail.

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread Malcolm Greene
08 07:10:16 -0200 Subject: Re: Strategy for determing difference between 2 very large dictionaries En Wed, 24 Dec 2008 06:23:00 -0200, escribió: > Hi Gabriel, > > Thank you very much for your feedback! > >> k1 = set(dict1.iterkeys()) > > I noticed you suggested .iterkeys

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread python
x27;: 3, 'd': 4} That's very cool! Thanks for sharing that technique. Regards, Malcolm - Original message - From: "Peter Otten" <__pete...@web.de> To: python-list@python.org Date: Wed, 24 Dec 2008 09:46:51 +0100 Subject: Re: Strategy for determing differ

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread Gabriel Genellina
En Wed, 24 Dec 2008 06:23:00 -0200, escribió: Hi Gabriel, Thank you very much for your feedback! k1 = set(dict1.iterkeys()) I noticed you suggested .iterkeys() vs. .keys(). Is there any advantage to using an iterator vs. a list as the basis for creating a set? I You've got an excelent ex

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread python
Hi James, > For the purpose of perpetuating the annoying pedantry that has made > usenet great: > > http://docs.python.org/dev/3.0/whatsnew/3.0.html#views-and-iterators-instead-of-lists Great tip! Thank you! Malcolm -- http://mail.python.org/mailman/listinfo/python-list

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread James Stroud
Marc 'BlackJack' Rintsch wrote: On Wed, 24 Dec 2008 03:23:00 -0500, python wrote: Hi Gabriel, Thank you very much for your feedback! k1 = set(dict1.iterkeys()) I noticed you suggested .iterkeys() vs. .keys(). Is there any advantage to using an iterator vs. a list as the basis for creating a

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread Peter Otten
Gabriel Genellina wrote: > En Wed, 24 Dec 2008 05:16:36 -0200, escribió: [I didn't see the original post] >> I'm looking for suggestions on the best ('Pythonic') way to >> determine the difference between 2 very large dictionaries >> containing simple key/value pairs. >> By difference, I mean a

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread python
explanation. Thank you! Malcolm - Original message - From: "Marc 'BlackJack' Rintsch" To: python-list@python.org Date: 24 Dec 2008 08:30:41 GMT Subject: Re: Strategy for determing difference between 2 very large dictionaries On Wed, 24 Dec 2008 03:23:00 -0

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread Marc 'BlackJack' Rintsch
On Wed, 24 Dec 2008 03:23:00 -0500, python wrote: > Hi Gabriel, > > Thank you very much for your feedback! > >> k1 = set(dict1.iterkeys()) > > I noticed you suggested .iterkeys() vs. .keys(). Is there any advantage > to using an iterator vs. a list as the basis for creating a set? I > understan

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread python
s much better than the list comprehension approach I was struggling with. Your dictionary comprehension statement describes exactly what I wanted to write. Regards, Malcolm - Original message - From: "Gabriel Genellina" To: python-list@python.org Date: Wed, 24 Dec 2008 05:46:04 -0200 S

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-24 Thread Malcolm Greene
ime. Thank you for sharing your thoughts with me. Regards, Malcolm - Original message - From: "Roger Binns" To: python-list@python.org Date: Tue, 23 Dec 2008 23:26:49 -0800 Subject: Re: Strategy for determing difference between 2 very large dictionaries -BEGIN PGP SIGNE

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-23 Thread Chris Rebert
On Tue, Dec 23, 2008 at 11:46 PM, Gabriel Genellina wrote: > > Yes; but isn't a dict comprehension more adequate? > > [key: (dict1[key], dict2[key]) for key in common_keys if > dict1[key]!=dict2[key]} That initial [ should be a { of course. Cheers, Chris -- Follow the path of the Iguana...

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-23 Thread Gabriel Genellina
En Wed, 24 Dec 2008 05:16:36 -0200, escribió: I'm looking for suggestions on the best ('Pythonic') way to determine the difference between 2 very large dictionaries containing simple key/value pairs. By difference, I mean a list of keys that are present in the first dictionary, but not the seco

Re: Strategy for determing difference between 2 very large dictionaries

2008-12-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 pyt...@bdurham.com wrote: > Feedback on my proposed strategies (or better strategies) would be > greatly appreciated. Both strategies will work but I'd recommend the second approach since it uses already tested code written by other people - the chanc

Strategy for determing difference between 2 very large dictionaries

2008-12-23 Thread python
I'm looking for suggestions on the best ('Pythonic') way to determine the difference between 2 very large dictionaries containing simple key/value pairs. By difference, I mean a list of keys that are present in the first dictionary, but not the second. And vice versa. And a list of keys in common b