The threshold would likely be most beneficial if it was based on pagesize (which is constant relative to RAM size). For small allocations (less than several megabytes), a modern malloc implementation typically uses a pool, so growing a allocation (except by a small amount) will probably result in a copy anyways, and no memory reuse. Once malloc switches to direct mmap calls, then it probably makes sense to add pages at a more gradual rate.
On Fri, May 1, 2015 at 11:48 AM Tim Holy <[email protected]> wrote: > On Friday, May 01, 2015 08:03:31 AM Scott Jones wrote: > > Still, same issue as I described above... probably better to increase by > 2x > > up to a point, and then by chunk sizes, where the chunk sizes might > slowly > > get larger... > > I see your point, but it will also break the O(nlogn) scaling. We couldn't > hard-code the cutoff, because some people run julia on machines with 4GB > of RAM > and others with 1TB of RAM. So, we could query the amount of RAM available > and > switch based on that result, but since all this would only make a > difference > for operations that consume between 0.5x and 1x the user's RAM (which to me > seems like a very narrow window, on the log scale), is it really worth the > trouble? > > --Tim > >
