> From: Guido van Rossum <gu...@python.org> > Steve Howell <showel...@yahoo.com> > wrote: > > It seems to me that the goal of keeping lists > > super-compact from a memory standpoint is contradicted by > > the code in list_resize that optimistically preallocates > > extra memory on appends. > > Ah, but that applies for *large* lists. Adding 8 bytes to > each list > object applies to *all* lists. I betcha that many programs > use many > tiny lists. >
Even most tiny-ish lists are wasting memory, though, according to this sequence: 4, 8, 16, 25, ... > If I had a program that was growing and shrinking a list at > both ends, > I'd consider a deque. I'd use a deque on current Python, but I'd use list with the proposed patch. :) > If I had a program that was growing > and > shrinking a list at the front, I'd consider maintaining the > data > backwards. Extensive use of pop(0) (and insert(0, x) I > suppose) is > only defensible if you also need to access the data by > index and > negative indexes are not an option. Think about that. > Maintaining the data backwards creates complications if you are popping off small chunks of the list and then subsequently operating on them. A parser would be a good example. If you reverse the original list, then grab the first four lines, for example, and then want to concatenate the first four lines in the original order, you will need to reverse them again. Having to reverse a list to change its performance just seems backward to me. > > [...] My > > gut says that the trend of the future is more CPU scarcity > > than memory scarcity, but that's just a WAG. > > I'm no expert in this area, but considering the cost of > cache misses, > I'm not so sure about that. Ok, good point. > > Just more food for thought--doing the memory > management within listobject.c is really a workaround to the > limitations of the underlying memory manager. It seems > conceptually possible to me to design a memory manager that > allows you to shrink and extend memory chunks on both sides, > without any extra memory overhead for bookkeeping. I'm > sure the devils in the details, of course. I asked Doug Lea about this, and he just replied to me that the devil is indeed in the details--it would be hard to make memory managers do what I'm proposing in an efficient way. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
