On Mar 21, 2013, at 7:21 AM, Benjamin Bazso wrote:
> I am using jemalloc for my server application and I have been monitoring the
> memory (rss) for several days and it continues to rise. I have previously
> monitored the application using valgrind for several days and it reported no
> memory leaks.
What version of jemalloc are you using, and on what operating system?
Also, valgrind will report as leaks memory that is unreachable, but your
application can bloat its reachable memory usage without valgrind complaining.
You may find jemalloc's heap profiling support useful in understanding where
your application is actually using memory, and if allocated memory usage is
truly growing, you will be able to tell where that growth is occurring within
your application.
> This leads me to believe that the rise in memory consumption (rss) is due to
> memory fragmentation.
>
> With this in mind, I added to my application the call to malloc_stats_print()
> function and I poll this function every hour and save the results. From what
> I see and understand from this report, I do indeed see the memory consumption
> rising and I also noticed that the number of bigger allocations increases
> (~2.5 MB) over time.
>
> [...]
> How can I use the malloc_stats_print() stats to show that my application is
> suffering from memory fragmentation?
For a summary of external fragmentation, look for the line that looks like:
Allocated: 5352608, active: 5353472, mapped: 285212672
External fragmentation can be computed as 1.0 - (allocated/active). That
doesn't tell the whole RSS picture though; there are internal jemalloc data
structures, dirty unused pages, and thread-cached objects that impose
additional overhead.
> Is there a way to help eliminate fragmentation in jemalloc, perhaps by
> calling an extended facility?
jemalloc would need to be able to compact active allocations via relocation in
order to eliminate fragmentation, and that's simply not generally possible in
C/C++.
> Will using thread cache flushing and forced dirty page purging help reduce
> fragmentation? Is it a good idea?
Yes, this may reduce fragmentation, but there are probably easier and more
effective methods. From a fragmentation perspective, the ideal jemalloc
configuration is one arena, no thread caching, and a very high active:dirty
ratio (prevents accumulation of unused dirty pages). You will probably find
that you can tune one or more of these parameters in a way that reduces
fragmentation without significantly impacting your application's throughput.
Jason_______________________________________________
jemalloc-discuss mailing list
[email protected]
http://www.canonware.com/mailman/listinfo/jemalloc-discuss
