Hi, I just checked in the fruits of my labor from the Mono Summit. With the help of Ben Maurer, I identified a number of places where we were doing some stupid things in Beagle, and with some additional info on some of the quirkier aspects of the runtime, I was able to reduce the startup memory usage of the main Beagle process by about 40%.
The quirkier aspects: * Objects of type System.Type are never collected by the GC. That's because these objects represent the low-level type system and have to be stable inside the runtime. The dynamic nature of many aspects of Beagle caused thousands of these objects to be created. * Our default policy has been to run the Mono runtime with --debug. The main Mono JIT guy, Paolo Molaro, made it clear to me that we shouldn't be running this as the default and that there aren't any guarantees this doesn't leak memory. From some quick testing by Ben and myself, we seem to believe that it does in fact leak. The simple first step is to turn off --debug to the Mono runtime. The downside to this is that we will no longer get line numbers in exception logs by default, but at this point Beagle is generally stable enough that I am willing to make this tradeoff. (Besides, it can now be turned back on by passing --mono-debug to the command-line.) The second step was more involved. Beagle dynamically loads a number of classes to make it highly pluggable. These include backends, filters, and the message passing system. To load these classes previously, we would walk all the assemblies loaded into memory (for messages) or load assemblies from well-known locations on disk and then walk those (for backends and filters). This is what created the thousands of types I mentioned before. Now, we have assembly attributes (essentially assembly metadata) which specifies the classes that were previously picked out of thousands. This metadata is usually set in an AssemblyInfo.cs file for the assembly, and is required for these classes to be loaded. Think of it as registering a class. This is necessary for filters, backends, and message passing elements. It's also used in the Thunderbird backend. The memory savings are substantial. With a non-existent ~/.beagle and all 16 backends started: old: Debug: Memory usage: VmSize=46.0 MB, VmRSS=19.8 MB, GC.GetTotalMemory=1499136 new: Debug: Memory usage: VmSize=34.7 MB, VmRSS=12.9 MB, GC.GetTotalMemory=1343488 That's 7 meg reduction in RSS size from the old to the new. From some heap-buddy statistics: old: Filename: outfile.beagled Allocated Bytes: 4.6M Allocated Objects: 80669 GCs: 16 Resizes: 10 Final heap size: 2.1M Distinct Types: 375 Backtraces: 16768 12306 live objects using 466k new: Filename: outfile.beagled Allocated Bytes: 2.8M Allocated Objects: 57490 GCs: 13 Resizes: 9 Final heap size: 1.5M Distinct Types: 370 Backtraces: 12029 8368 live objects using 423k You can see we allocated 1.8 megabytes less, allocated 23k fewer objects, resized the heap one less time, and GCed two times less. At shutdown time, we had 4000 fewer objects -- this number is particularly relevant because nearly all of those 4000 objects were System.Type instances. The other 19,000 objects were probably duplicate instances of the same types. We see similar reductions in the index helper process. I've yet to do any long-term tests, but I'll do a full index on my system overnight and see where we are tomorrow. I'd like to ask people to build and run from CVS and watch for bugs, and to keep an eye out to see to what extent memory usage has improved. Thanks, Joe _______________________________________________ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers