Re: Uniquifying a list?
In article [EMAIL PROTECTED], Felipe Almeida Lessa [EMAIL PROTECTED] wrote: list(set(x)) is the clear winner with almost O(1) performance. Moral: in an interpreted language, use builtins as much as possible. -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquifying a list?
There is my version too: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/438599 Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquifying a list?
Tim Chase: Is there an obvious/pythonic way to remove duplicates from a list (resulting order doesn't matter, Use a set. http://www.python.org/doc/lib/types-set.html -- René Pijlman -- http://mail.python.org/mailman/listinfo/python-list
Uniquifying a list?
Is there an obvious/pythonic way to remove duplicates from a list (resulting order doesn't matter, or can be sorted postfacto)? My first-pass hack was something of the form myList = [3,1,4,1,5,9,2,6,5,3,5] uniq = dict([k,None for k in myList).keys() or alternatively uniq = list(set(myList)) However, it seems like there's a fair bit of overhead here...creating a dictionary just to extract its keys, or creating a set, just to convert it back to a list. It feels like there's something obvious I'm missing here, but I can't put my finger on it. Thanks... -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquifying a list?
Em Ter, 2006-04-18 às 10:31 -0500, Tim Chase escreveu: Is there an obvious/pythonic way to remove duplicates from a list (resulting order doesn't matter, or can be sorted postfacto)? My first-pass hack was something of the form myList = [3,1,4,1,5,9,2,6,5,3,5] uniq = dict([k,None for k in myList).keys() or alternatively uniq = list(set(myList)) However, it seems like there's a fair bit of overhead here...creating a dictionary just to extract its keys, or creating a set, just to convert it back to a list. It feels like there's something obvious I'm missing here, but I can't put my finger on it. Your list with 11 elements (7 unique): $ python2.4 -mtimeit -s 'x = [3,1,4,1,5,9,2,6,5,3,5]' 'y = dict((k,None) for k in x).keys()' 10 loops, best of 3: 8.01 usec per loop $ python2.4 -mtimeit -s 'x = [3,1,4,1,5,9,2,6,5,3,5]' $'y = []\nfor i in x:\nif i not in y:\ny.append(i)' 10 loops, best of 3: 5.43 usec per loop $ python2.4 -mtimeit -s 'x = [3,1,4,1,5,9,2,6,5,3,5]' 'y = list(set(x))' 10 loops, best of 3: 3.57 usec per loop A list with 100 000 elements (1 000 unique): $ python2.4 -mtimeit -s 'x = range(1000) * 100' $'y = []\nfor i in x:\n if i not in y:\ny.append(i)' 10 loops, best of 3: 2.12 sec per loop $ python2.4 -mtimeit -s 'x = range(1000) * 100' 'y = dict((k,None) for k in x).keys()' 10 loops, best of 3: 32.2 msec per loop $ python2.4 -mtimeit -s 'x = range(1000) * 100' 'y = list(set(x))' 100 loops, best of 3: 6.09 msec per loop list(set(x)) is the clear winner with almost O(1) performance. *However*, can't you always use set or frozenset instead of converting back and forth? HTH, -- Felipe. -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquifying a list?
You could do uniq = [x for x in set(myList)] but that's not really any different than what you already have. This almost works: uniq = [x for x in myList if x not in uniq] except the r-val uniq isn't updated after each iteration. Personally I think list(set(myList)) is as optimal as you'll get. Tim Chase wrote: Is there an obvious/pythonic way to remove duplicates from a list (resulting order doesn't matter, or can be sorted postfacto)? My first-pass hack was something of the form myList = [3,1,4,1,5,9,2,6,5,3,5] uniq = dict([k,None for k in myList).keys() or alternatively uniq = list(set(myList)) However, it seems like there's a fair bit of overhead here...creating a dictionary just to extract its keys, or creating a set, just to convert it back to a list. It feels like there's something obvious I'm missing here, but I can't put my finger on it. Thanks... -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquifying a list?
Tim Chase [EMAIL PROTECTED] writes: Is there an obvious/pythonic way to remove duplicates from a list (resulting order doesn't matter, or can be sorted postfacto)? My first-pass hack was something of the form myList = [3,1,4,1,5,9,2,6,5,3,5] uniq = dict([k,None for k in myList).keys() or alternatively uniq = list(set(myList)) However, it seems like there's a fair bit of overhead here... It seems you haven't timed these to find out which one actually takes longer; the first one gives syntax errors. Rather than guessing about overhead, and things we can't put our finger on, let's actually finger it: $ python2.4 -m timeit -c 'raw = [3,1,4,1,5,9,2,6,5,3,5]; uniq = dict([(k,None) for k in raw]).keys()' 1 loops, best of 3: 45 usec per loop $ python2.4 -m timeit -c 'raw = [3,1,4,1,5,9,2,6,5,3,5]; uniq = list(set(raw))' 1 loops, best of 3: 21 usec per loop The set method is faster, in this example. It also seems to do what you want. Why do you believe you even need to create a list from the set? $ python2.4 -m timeit -c 'raw = [3,1,4,1,5,9,2,6,5,3,5]; uniq = set(raw)' 10 loops, best of 3: 13.1 usec per loop Why not just create a set and use that? -- \ It is forbidden to steal hotel towels. Please if you are not | `\person to do such is please not to read notice. -- Hotel | _o__) sign, Kowloon, Hong Kong | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquifying a list?
On 19/04/2006 1:31 AM, Tim Chase wrote: Is there an obvious/pythonic way to remove duplicates from a list (resulting order doesn't matter, or can be sorted postfacto)? Google is your friend: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560 -- http://mail.python.org/mailman/listinfo/python-list
