On 11 June 2015 at 15:48, Dolph Mathews <[email protected]> wrote: > tl;dr .iteritems() is faster and more memory efficient than .items() in > python2 > > > Using xrange() in python2 instead of range() because it's more memory > efficient and consistent between python 2 and 3... > > # xrange() + .items() > python -m timeit -n 20 for\ i\ in\ > dict(enumerate(xrange(1000000))).items():\ pass > 20 loops, best of 3: 729 msec per loop > peak memory usage: 203 megabytes
This test conflates setup and execution. Better like my example, because otherwise you're not testing iteritems vs items, you're testing dictionary creation time; likewise memory pressure. Your times are meaningless as it stands. To test memory pressure, don't use timeit. Just use the interpreter. $ python Python 2.7.8 (default, Oct 20 2014, 15:05:19) [GCC 4.9.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> d = dict(enumerate(range(1000000))) >>> import os >>> os.getpid() 28345 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 28345 robertc 20 0 127260 104568 4744 S 0.0 0.6 0:00.17 python >>> i = d.items() 28345 robertc 20 0 206524 183560 4744 S 0.0 1.1 0:00.59 python 183560-104568=80M to hold a reference to all 1 million items, which indeed is not as efficient as python3. So *IF* we had a million item dict, and absolutely nothing else around, we should care. But again - where in OpenStack does this matter the slightest? No one has disputed that they are different. The assertion that it matters is what is out of line with our reality. 10000 items: 28399 robertc 20 0 31404 8480 4612 S 0.0 0.1 0:00.01 python 28399 robertc 20 0 32172 9268 4612 S 0.0 0.1 0:00.01 python 9268-8489=0.8M which is indeed 2 orders of magnitude less. And I'd STILL challenge anyone to find a place where 10000 items are being passed around within OpenStack's components without it being a bug today. Optimising away under a M of data when we shouldn't have that many rows/items/whatever in memory in the first place is just entirely missing the point of programming in Python. -Rob -- Robert Collins <[email protected]> Distinguished Technologist HP Converged Cloud __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
