On Mon, Mar 24, 2008 at 9:21 AM, Scott Windsor <[EMAIL PROTECTED]> wrote:
> Right now, my processes aren't gigantic... I'm preparing for a 'worst case' > scenario when I have a extremely large processes or memory usage. This can > easily happen on specific applications such as an image server (using image > magick) or parsing/creating large xml payloads (a large REST server). For > those applications, I may have a large amount of memory used for each > request, which will increase until the GC is run. (*nod*) image magick is a well known bad citizen. Either don't use it at all, or use it in an external process from your web app processes. And if, for whatever reason, you must use it inside of your web app process, and your use case really does create processes so enormous that you can perceive a response lag from a manual GC.start inside of your request processing, then create a custom Rails handler that does it. You can trivially alter it to do whatever GC.foo actions you desire. The code is simple and easy to follow, so just make your own Mongrel::Rails::RailsHandlerWithParanoidGCManagement. > There may be perfectly good reasons to have intermediate object creation > (good encapsulation, usage of a another library/gem you can't modify, large > operations that you need to keep atomic). While ideally you'd fix the > memory usage problem, this doesn't solve all cases. Obviously. It's easy and convenient to ignore the issue, and often the issue doesn't matter for a given piece of code. But if memory usage or execution speed becomes an issue for one's code, going back and taking a look at the throwaway object creation, and addressing it, can net considerable improvements. > Yes, for Ruby (and other GC'd languages), it's much harder to leak memory > such that the GC can never clean it up - but it does (and has) happened. > This case I'm less concerned about as a leak of this magnitude should be > considered a bug and fixed. Oh, I know. That's why I brought it up, though. You were talking about memory leaks, so I wanted to make a distinction. Real leaks, like the Array#shift bug, or leaky continuations, or badly behaved Ruby extensions, aren't affected by GC manipulations. > Inefficient it may be - but it might be just optimizing for a different > problem. For example, take ActiveRecord's association cache and it's query > cache. If you're doing a large number of queries each page load, > ActiveRecord is still going to cache them for each request - this is far > better than further round trips to the database, but may lead to a large > amount of memory consumed per each request. Sure. And if it's optimizing for a different problem, then that's fine, so long as the optimization isn't creating a worse probablem than the issue it's trying to address. But that's also largely irrelevant, I think. I just did a quick test. I created a program that creates 10 million objects. It has a footprint of about a gigabyte of RAM usage. It takes Ruby 0.5 seconds to walk that 10 million objects on my server. If you have a web app that has processes anywhere near that large, you have bigger problems to deal with. And if you have a more reasonably large, million object app, then on my server, the GC cost would be 0.05 seconds. Given the typical speed of Rails apps, an occasional 0.05 second delay is going to be unnoticable. > Agreed. But again, I'd rather it be a constant cost outside of processing a > request than a variable cost inside of processing a request. You're worrying about something that just isn't a problem in the vast, vast majority of cases. Again, testing on my server, even with a very simple, very fast piece of code creating objects, it takes almost 20x as long to create the objects as to GC them. > This can vary from application to application - all the more reason to make > this a configurable option (and not the default). It's still my position that it's not Mongrel's job to be implementing a manual memory management scheme that is almost always going to be a performance loser over just leaving it alone. It's still my position that if one has an application that, through testing, has been shown to have a use case where it can actually benefit from manual GC.foo management, then one can trivially create a mongrel handler that will do this for you. > I still disagree on this point - I doubt that Rails is the only web > framework that would benefit from being able to control when the GC is run. > This is going to be a common problem across frameworks whenever web > applications are consuming then releasing large amounts of memory - I'd say > it can be a pretty common use case for certain types of web applications. My point is that if it is _Rails_ code that is causing the problem, that's a _Rails_ problem. My point is also that manual GC.foo management is going to cause more problems than it helps for the vast majority of applications. GC cycles aren't that slow, especially compared to the speed of a typical Rails app, and certainly not when compared to the speed of a Rails request that makes a lot of objects and does any sort of intensive, time consuming operations. Kirk Haines _______________________________________________ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
