On Jan 19, 2016, at 9:56 AM, Jakob Buchgraber <[email protected]> wrote: > I think there might be an issue with this approach though: > https://github.com/jemalloc/jemalloc/issues/307
I'll try not to sit on this issue indefinitely, but it takes a while to get my head wrapped around the chunk hooks, so it may be a while before I have time to dig into it. So much to do... > It’s deadlocking right now as well, as I am accessing stats from within the > chunk hooks to determine which arenas to purge. I had to replace the malloc > mutexes with recursive mutexes to make it work. Seems fine so far. My memory is now fuzzy on this aspect of chunk hooks, but my vague recollection is that I attempted to drop all locks before calling into application-provided hook functions, mainly to avoid resource starvation in case the hook functions block in system calls. Maybe that's not generally possible, or maybe there's just a bug somewhere. In any case, if allocation is triggered by a call into mallctl(), calling back into mallctl() inside the hooks is going to deadlock, so calling back into jemalloc APIs is not generally safe. > Basically, I am running with lots of main memory (> 1TB). Most of the time > the program will only use a fraction of the available memory but some queries > will require almost all the memory in some random arena. So even if I leave > purging on and set lg_dirty_mult to say 3, some arenas might end up having > cached 10s of GB of physical memory with others running out and the program > will crash. Ideally, I would want BSD’s MADV_FREE on Linux. That patch never > got merged though. So what I am doing is to add some logic that tracks the > amount of committed physical memory and if some threshold is reached, I query > the jemalloc stats and dynamically adjust the purging ratio. Does that make > sense? Yes, dynamically updating the purging ratio is a reasonable approach, and I've done something similar in HHVM (temporarily purge more aggressively if any request threads go idle). By the way, the MADV_FREE will apparently be in the next Linux kernel release. Keep an eye on https://github.com/jemalloc/jemalloc/issues/149, which I'm actively working on. So far it looks like I'm going to be able to implement a time-based purging mechanism that does a much better job of controlling unused dirty pages *without* asynchronous purging threads, as long as the application doesn't go completely idle. I'm planning to get this feature into the 4.1.0 release as an opt-in, and I suspect it's going to prevail as the default mechanism in a subsequent release. Jason _______________________________________________ jemalloc-discuss mailing list [email protected] http://www.canonware.com/mailman/listinfo/jemalloc-discuss
