On Tue, Jan 12, 2010 at 10:46:29PM -0500, DM Smith wrote:
> So starting at 0, the size is 0.
> 0 => 0
> 0 + 1 => 4
> 4 + 1 => 8
> 8 + 1 => 16
> 16 + 1 => 25
> 25 + 1 => 35
> ...
>
> So I think the copied python comment is correct but not obviously correct.
So those numbers are supposed to be where the transitions occur? But that's
not where the jumps are. The jumps happen at...
8, 16, 24, 32, 40, 48...
... as you'd expect when adding (input >> 3), which is after all just a more
obscure way of writing (input / 8).
Sorry for being dense, but I still don't get it.
> The addition of 3 or 6 only helps initially, after some point it is just
> noise. It has the characteristic of being less aggressive with subsequent
> allocations.
Well, I have my doubts about whether this actually helps or not. It doesn't
seem general purpose enough.
For an array of bytes, the desirable behavior is clear -- you really ought
to round up to at least the size of a pointer because you're never going to
return a non-aligned buffer. Feeding 10 into getNextSize() and getting back
17 is weird -- you should get back either 16 or 24.
I also consider it strange that if you ask for 0 you get 3. A lot of the
time, if you're asking for 0 it's because the resource may never need to be
allocated. And if you know that the resource actually is going to be needed,
you're going to write code like this, from TermAttributeImpl.java:
termBuffer = new char[ArrayUtil.getNextSize(newSize < MIN_BUFFER_SIZE ?
MIN_BUFFER_SIZE : newSize)];
(MIN_BUFFER_SIZE is 10.)
But whatever. "3" and "6" look more significant on first read of the code
than they actually are, but they're only strange, not detrimental to
performance. I'm just ticked off because I spent so much time trying to
understand code which turns out to do so little.
> I'm not really up on whether this is best, but it is better than the
> doubling algorithm that it replaced. I think your suggestion that such an
> algorithm might contribute to fragmented memory is interesting. I wonder if
> C, perl and java have different issues regarding that.
It's wherever the memory allocator lives. That could be the JVM, it could be
glibc, it could be your own custom allocator. If you compile Perl with
-DUSE_MY_MALLOC it uses its own allocator, otherwise it uses the system's
malloc. KinoSearch actually has a dedicated allocator it uses for a very
targeted purpose, and this allocator has its own strategy for avoiding
fragmentation. The golden mean issue is relevant to all of those allocators.
Marvin Humphrey
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
