Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-18 Thread Evan Jones
On Thu, 2005-02-17 at 22:38, Tim Peters wrote: Then you allocate a small object, marked 's': bbbsfff Isn't the whole point of obmalloc is that we don't want to allocate s on the heap, since it is small? I guess s could be an object that might potentially

Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-18 Thread Tim Peters
[Tim Peters] ... Then you allocate a small object, marked 's': bbbsfff [Evan Jones] Isn't the whole point of obmalloc No, because it doesn't matter what follows that introduction: obmalloc has several points, including exploiting the GIL, heuristics

Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-18 Thread Evan Jones
On Feb 18, 2005, at 17:51, Tim Peters wrote: grow the list to its final size once, at the start (overestimating if you don't know for sure). Then instead of appending, keep an index to the next free slot, same as you'd do in C. Then the list guts never move, so if that doesn't yield the same

Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-18 Thread Tim Peters
[Tim Peters] grow the list to its final size once, at the start (overestimating if you don't know for sure). Then instead of appending, keep an index to the next free slot, same as you'd do in C. Then the list guts never move, so if that doesn't yield the same kind of speedup without using

RE: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-17 Thread Gfeller Martin
[mailto:[EMAIL PROTECTED] Sent: Thursday, 17 Feb 2005 02:26 To: Python Dev Cc: Gfeller Martin; Martin v. Löwis Subject: Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15% On Feb 16, 2005, at 18:42, Martin v. Löwis wrote: I must admit that I'm surprised. I would have

Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-17 Thread Tim Peters
[Gfeller Martin] what immediately comes to mind are Modules/cPickle.c and Modules/cStringIO.c, which (I believe) are heavily used by ZODB (which in turn is heavily used by the application). I probably guessed right the first time wink: LFH doesn't help with the lists directly, but helps

[Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-16 Thread Gfeller Martin
Dear all, I'm running a large Zope application on a 1x1GHz CPU 1GB mem Window XP Prof machine using Zope 2.7.3 and Py 2.3.4 The application typically builds large lists by appending and extending them. We regularly observed that using a given functionality a second time using the same

Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-16 Thread =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
Gfeller Martin wrote: Nevertheless, I tried to convert the heap used by Python to a Windows Low Fragmentation Heap (available on XP and 2003 Server). This improved the overall run time of a typical CPU-intensive report by about 15% (overall run time is in the 5 minutes range), with the same

Re: [Python-Dev] Windows Low Fragementation Heap yields speedup of ~15%

2005-02-16 Thread Evan Jones
On Feb 16, 2005, at 18:42, Martin v. Löwis wrote: I must admit that I'm surprised. I would have expected that most allocations in Python go through obmalloc, so the heap would only see large allocations. It would be interesting to find out, in your application, why it is still an improvement to