Re: Dynamic array reallocation algorithms

2010-01-14 Thread Michael McCandless
On Wed, Jan 13, 2010 at 9:12 PM, Marvin Humphrey wrote: > On Wed, Jan 13, 2010 at 11:46:50AM -0500, Michael McCandless wrote: >> If forced to pick, in general, I tend to prefer burning CPU not RAM, >> because the CPU is often a one-time burn, whereas RAM ties up storage >> for indefinite amounts o

Re: Dynamic array reallocation algorithms

2010-01-13 Thread Marvin Humphrey
On Wed, Jan 13, 2010 at 11:46:50AM -0500, Michael McCandless wrote: > If forced to pick, in general, I tend to prefer burning CPU not RAM, > because the CPU is often a one-time burn, whereas RAM ties up storage > for indefinite amounts of time. With our dependence on indexes being RAM-resident for

Re: Dynamic array reallocation algorithms

2010-01-13 Thread Marvin Humphrey
On Wed, Jan 13, 2010 at 09:43:12AM -0500, Yonik Seeley wrote: > Yeah, something highly optimized for python in C may not be optimal for Java. It looks like that algo was tuned to address poor reallocation performance under Windows 9x. http://svn.python.org/view/python/trunk/Objects/listobjec

Re: Dynamic array reallocation algorithms

2010-01-13 Thread Michael McCandless
On Wed, Jan 13, 2010 at 10:04 AM, Marvin Humphrey wrote: > Both mild and aggressive over-allocation solve that problem, but aggressive > over-allocation also involves significant RAM costs. Where the best balance > lies depends on how bad the reallocation performance is in relation to the > cost

Re: Dynamic array reallocation algorithms

2010-01-13 Thread Michael McCandless
On Wed, Jan 13, 2010 at 8:53 AM, DM Smith wrote: > The pattern that is stated in the comment only occurs under a very > specific situation: The growing of an array one element at a time > and reallocation only when the current is exceeded. Ahh -- that works -- thanks for clarifying. So if you st

Re: Dynamic array reallocation algorithms

2010-01-13 Thread Michael McCandless
On Wed, Jan 13, 2010 at 10:04 AM, Marvin Humphrey wrote: > On Wed, Jan 13, 2010 at 05:48:08AM -0500, Michael McCandless wrote: >> Have you notified python-dev? > > No, not yet.  Is it kosher with the Python license to have copied-and-pasted > that comment?  It's not credited from what I can see.  

Re: Dynamic array reallocation algorithms

2010-01-13 Thread Marvin Humphrey
On Wed, Jan 13, 2010 at 05:48:08AM -0500, Michael McCandless wrote: > Have you notified python-dev? No, not yet. Is it kosher with the Python license to have copied-and-pasted that comment? It's not credited from what I can see. Small, but we should probably fix that. > Right, and also to stri

Re: Dynamic array reallocation algorithms

2010-01-13 Thread Yonik Seeley
On Tue, Jan 12, 2010 at 6:27 PM, Marvin Humphrey wrote: >    return (targetSize >> 3) + (targetSize < 9 ? 3 : 6) + targetSize; > > For input values of 9 or greater, all that formula does is multiply by 1.125 > and add 6. (Values enumerated below my sig.) > > The comment appears to have originated

Re: Dynamic array reallocation algorithms

2010-01-13 Thread DM Smith
On Jan 13, 2010, at 1:00 AM, Marvin Humphrey wrote: > On Tue, Jan 12, 2010 at 10:46:29PM -0500, DM Smith wrote: > >> So starting at 0, the size is 0. >> 0 => 0 >> 0 + 1 => 4 >> 4 + 1 => 8 >> 8 + 1 => 16 >> 16 + 1 => 25 >> 25 + 1 => 35 >> ... >> >> So I think the copied python comment is correct

Re: Dynamic array reallocation algorithms

2010-01-13 Thread Michael McCandless
Phew, good svn sleuthing Marvin! Responses below... On Tue, Jan 12, 2010 at 6:27 PM, Marvin Humphrey wrote: > Greets, > > I've been trying to understand this comment regarding ArrayUtil.getNextSize(): > >     * The growth pattern is:  0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... > > Maybe I'm missin

Re: Dynamic array reallocation algorithms

2010-01-12 Thread Marvin Humphrey
On Tue, Jan 12, 2010 at 10:46:29PM -0500, DM Smith wrote: > So starting at 0, the size is 0. > 0 => 0 > 0 + 1 => 4 > 4 + 1 => 8 > 8 + 1 => 16 > 16 + 1 => 25 > 25 + 1 => 35 > ... > > So I think the copied python comment is correct but not obviously correct. So those numbers are supposed to be whe

Re: Dynamic array reallocation algorithms

2010-01-12 Thread DM Smith
On Jan 12, 2010, at 6:27 PM, Marvin Humphrey wrote: > Greets, > > I've been trying to understand this comment regarding ArrayUtil.getNextSize(): > > * The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... > > Maybe I'm missing something, but I can't see how the formula yields su

Dynamic array reallocation algorithms

2010-01-12 Thread Marvin Humphrey
Greets, I've been trying to understand this comment regarding ArrayUtil.getNextSize(): * The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... Maybe I'm missing something, but I can't see how the formula yields such a growth pattern: return (targetSize >> 3) + (targetSize <