Hi Nick, Here a few notes and thoughts:
* On the build process: The makefiles build `racketcgc' first, which uses the Boehm collector by default (or the slower but more portable SGC collector by request) and compiles the Racket C code as-is. Then, the makefiles run `xform', which is a Racket program, to convert most of the C source of Racket to better cooperate with the GC. Finally, the makefiles compile the converted C source to arrive at `racket3m', which is renamed to `racket' on install. * On how the source is organized: The 3m GC is in the "src/racket/gc2" directory and mostly in the "newgc.c" file, while the Boehm GC is in "src/racket/gc", and SGC is in "src/racket/sgc". (We really should rename some of those directories to make the organization more clear.) The main Racket C source in "src/racket/src" is mostly independent of the GC, although preprocessor directives such as `#ifdef MZ_PRECISE_GC' are sometimes used for 3m-specific code. * On GC logging: Logging a message from the GC can be tricky, because logging triggers allocation. The GC logging that's currently in place works by waiting until after the GC is complete (and the system is back in normal mode) before logging a message. I'm not sure how well it would work to try to log a message on every allocation --- and somehow treat the logger's allocation specially. * Try `dump-memory-stats': In case you haven't discovered it already, `dump-memory-stats' tells you more about how current memory use breaks down into different kinds of values. * Backtraces may help: If you use `configure --enable-backtrace' when building, then `dump-memory-stats' can show you paths to specific types of objects in the heap. Backtraces are useful for tracking down leaks, especially if you have some idea of the kinds of records that are being leaked and you want to know why the GC thinks the records should be retained. Backtraces are less useful for tracking down the source of allocations, though. Enabling backtraces also enables some other support in `dump-memory-stats', including support for walking through all memory. I'm not sure how well that still works, though, since it hasn't been used in a while. * Instead of putting `(current-memory-use)' everywhere, you might consider creating a separate thread that periodically calls `(current-memory-use)' and uses `continuation-marks' on the main thread with `continuation-mark-set->context' to extract the thread's context. That's how the time-oriented profiler works, for example. Hope that helps, Matthew At Fri, 9 Mar 2012 11:48:28 -0800, Nick Sivo wrote: > Hello, > > I'm optimizing some code and have used gc-info logging > to correlate substantial application pauses with GC occurrences. This make > sense. The obvious solution is to allocate less memory, but tracking down > where it's coming from isn't easy. In the short term I plan to inject > (current-memory-use) based logging all over the code I'm optimizing, but > would like to develop and contribute something more generic and re-usable > if there's interest in it. > > My ideal goal is to add logging that reveals which code is allocating > memory (how often, how much), and also, for each GC pass, the origin(s) and > final resting place of what was collected. > > So now I'm trying to learn more about Racket's 3m garbage collector so I > can instrument it. I've scoured the guide and reference, but found nothing > that gets into the details of the runtime/collector interaction and in > which code files that lives. Before I start grepping through the source > and trying to piece it all together on my own, I figured I'd ask. Some > pointers in the right direction, even vague hints of where to start, would > be awesome :) Even a description of the build process - where xform comes > into play - would be infinitely helpful. > > Please also feel free to tell me I'm crazy and point out something > obviously simpler I'm missing. I just want faster code with fewer, shorter > GC pauses - I have no vested interest in how it happens, but would love to > contribute a useful tool if that makes sense. > > Regards, > Nick > _________________________ > Racket Developers list: > http://lists.racket-lang.org/dev _________________________ Racket Developers list: http://lists.racket-lang.org/dev
