On 1/23/07, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote:
Am 23.01.2007 um 15:42 schrieb Zoran Vasiljevic: > This way the system allocator > is handling those and it can toss free'd blocks > to other threads reducing peak memory usage. > Since allocations of that sizes (and larger) are > pretty rare, we need not worry about locking > contention as it is the case with smaller allocations. More info... Actually that lowers the amount of memory released to the system (as system allocs mostly never do that) but it exremely reduces the peak memory usage. In our app, when we start a job, the VM goes to about 300MB with initial VTmalloc, to 235MB with modified VTmalloc (8K allocs) and to about 225MB with system alloc. At the same time, VTmalloc is up to 20 times faster than system alloc (thats on Mac). Thats a pretty good bargain for us: invest about 10MB more but get far better performance is absolutely acceptable. I haven't tested Linux or Solaris yet, but I guess I will have the same result (apart from the difference between the OS alloc and us). What I would like to do is to lower that usage even more on the cost of more locking contention, as we can tolerate that. Just to see how low and fast we can go.
I thought the goal was less memory fragmentation -- not necessarily speed which was pretty good with zippy -- and fragmentation was solved by using mmap for memory which could be returned to the system. So fragmentation is only a problem for small allocations, in the real world? Also, why did you choose this strategy, using the system malloc for > 8000b, rather than modify the way the per thread mmap pools are released to the OS. This used to be on thread exit, right? If this was causing bloating in long-lived threads, you could flush the pool when some low water mark is reached.
