On Fri, Mar 21, 2008 at 1:19 PM, Kirk Haines <[EMAIL PROTECTED]> wrote:
> On Fri, Mar 21, 2008 at 1:23 PM, Scott Windsor <[EMAIL PROTECTED]> wrote: > > > I understand that the GC is quite knowledgeable about when to run > garbage > > collection when examining the heap. But, the GC doesn't know anything > about > > my application or it's state. The fact that when the GC runs everything > > stops is why I'd prefer to limit when the GC will run. I'd rather it > run > > outside of serving a web request rather then when it's right in the > middle > > of serving requests. > > It doesn't matter, if one is looking at overall throughput. And how > long do your GC runs take? If you have a GC invocation that is > noticable on a single request, your processes must be gigantic, which > would suggest to me that there's a more fundamental problem with the > app. Right now, my processes aren't gigantic... I'm preparing for a 'worst case' scenario when I have a extremely large processes or memory usage. This can easily happen on specific applications such as an image server (using image magick) or parsing/creating large xml payloads (a large REST server). For those applications, I may have a large amount of memory used for each request, which will increase until the GC is run. > > I know that the ideal situation is to not need to run the GC, but the > > reality is that I'm using various gems and plugins and not all are well > > behaved and free of memory leaks. Rails itself may also have regular > leaks > > No, it's impractical to never run the GC. The ideal situation, at > least where execution performance and throughput on a high performance > app is concerned, is to just intelligently reduce how often it needs > to run by paying attention to your object creation. In particular, > pay attention to the throwaway object creation. > There may be perfectly good reasons to have intermediate object creation (good encapsulation, usage of a another library/gem you can't modify, large operations that you need to keep atomic). While ideally you'd fix the memory usage problem, this doesn't solve all cases. > > > from time to time and I'd prefer to have my application consistently be > slow > > than randomly (and unexpectedly) be slow. The alternative is to > terminate > > your application after N number of requests and never run the GC, which > I'm > > not a fan of. > > If your goal is to deal with memory leaks, then you really need to > define what that means in a GC'd language like Ruby. > To me, a leak is something that consumes memory in a way that eludes > the GC's ability to track it and reuse it. The fundamental nature of > that sort of thing is that the GC can't help you with it. > Yes, for Ruby (and other GC'd languages), it's much harder to leak memory such that the GC can never clean it up - but it does (and has) happened. This case I'm less concerned about as a leak of this magnitude should be considered a bug and fixed. > > If by leaks, you mean code that just creates a lot of objects that the > GC needs to clean up, then those aren't leaks. It may be inefficient > code, but it's not a memory leak. > Inefficient it may be - but it might be just optimizing for a different problem. For example, take ActiveRecord's association cache and it's query cache. If you're doing a large number of queries each page load, ActiveRecord is still going to cache them for each request - this is far better than further round trips to the database, but may lead to a large amount of memory consumed per each request. > > And in the end, while disabling GC over the course of a request may > result in processing that one request more quickly than it would have > been processed otherwise, the disable/enable dance is going to cost > you something. > Agreed. But again, I'd rather it be a constant cost outside of processing a request than a variable cost inside of processing a request. > > You'll likely either end up using more RAM than you otherwise would > have in between GC calls, resulting in bigger processes, or you end up > calling GC more often than you otherwise would have, reducing your > high performance app's throughput. > > And for the general cases, that's not an advantageous situation. > This can vary from application to application - all the more reason to make this a configurable option (and not the default). > > To be more specific, if excessive RAM usage and GC costs that are > noticable to the user during requests is a common thing for Rails > apps, and the reason for that is bad code in Rails and not just bad > user code, then the Rails folks should be the targets of a > conversation on the matter. Mongrel itself, though, does not need to > be, and should not be playing manual memory management games on the > behalf of a web framework. > > > Kirk Haines > I still disagree on this point - I doubt that Rails is the only web framework that would benefit from being able to control when the GC is run. This is going to be a common problem across frameworks whenever web applications are consuming then releasing large amounts of memory - I'd say it can be a pretty common use case for certain types of web applications. - scott
_______________________________________________ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
