Re: [Mongrel] mongrel garbage collection

2008-03-26 Thread Evan Weaver
You're right, ok. So the memory causing the OOM error isn't actually on the Ruby heap, but it can't get freed until the opaque object gets GC'd. Evan On Tue, Mar 25, 2008 at 1:20 PM, Kirk Haines <[EMAIL PROTECTED]> wrote: > On Tue, Mar 25, 2008 at 11:02 AM, Evan Weaver <[EMAIL PROTECTED]> wrote:

Re: [Mongrel] mongrel garbage collection

2008-03-25 Thread Kirk Haines
On Tue, Mar 25, 2008 at 11:02 AM, Evan Weaver <[EMAIL PROTECTED]> wrote: > > My hunch is that rmagick is allocating large amounts of RAM ouside of > > Ruby. It registers its objects with the interpreter, but the RAM > > usage in rmagick itself doesn't count against GC_MALLOC_LIMIT because > >

Re: [Mongrel] mongrel garbage collection

2008-03-25 Thread James Tucker
On 25 Mar 2008, at 17:05, Steve Midgley wrote: > At 03:41 AM 3/25/2008, [EMAIL PROTECTED] wrote: >> Date: Tue, 25 Mar 2008 10:40:50 + >> From: James Tucker <[EMAIL PROTECTED]> >> Subject: Re: [Mongrel] mongrel garbage collection >> To: mongrel-users@rub

Re: [Mongrel] mongrel garbage collection

2008-03-25 Thread James Tucker
On 25 Mar 2008, at 15:26, Kirk Haines wrote: > On Tue, Mar 25, 2008 at 4:40 AM, James Tucker <[EMAIL PROTECTED]> > wrote: >> Forgive me for not having read the whole thread, however, there is >> one thing >> that seems to be really important, and that is, ruby hardly ever >> runs the >> damn

Re: [Mongrel] mongrel garbage collection

2008-03-25 Thread Steve Midgley
At 03:41 AM 3/25/2008, [EMAIL PROTECTED] wrote: >Date: Tue, 25 Mar 2008 10:40:50 + >From: James Tucker <[EMAIL PROTECTED]> >Subject: Re: [Mongrel] mongrel garbage collection >To: mongrel-users@rubyforge.org >Message-ID: <[EMAIL PROTECTED]> >Content-Type: text/pl

Re: [Mongrel] mongrel garbage collection

2008-03-25 Thread Steve Midgley
At 03:41 AM 3/25/2008, [EMAIL PROTECTED] wrote: >Date: Tue, 25 Mar 2008 10:40:50 + >From: James Tucker <[EMAIL PROTECTED]> >Subject: Re: [Mongrel] mongrel garbage collection >To: mongrel-users@rubyforge.org >Message-ID: <[EMAIL PROTECTED]> >Content-Type: text/pl

Re: [Mongrel] mongrel garbage collection

2008-03-25 Thread Evan Weaver
> My hunch is that rmagick is allocating large amounts of RAM ouside of > Ruby. It registers its objects with the interpreter, but the RAM > usage in rmagick itself doesn't count against GC_MALLOC_LIMIT because > Ruby didn't allocate it, so doesn't know about it. It's allocating opaque object

Re: [Mongrel] mongrel garbage collection

2008-03-25 Thread Kirk Haines
On Tue, Mar 25, 2008 at 4:40 AM, James Tucker <[EMAIL PROTECTED]> wrote: > Forgive me for not having read the whole thread, however, there is one thing > that seems to be really important, and that is, ruby hardly ever runs the > damned GC. It certainly doesn't do full runs nearly often enough (IMO

Re: [Mongrel] mongrel garbage collection

2008-03-25 Thread James Tucker
Forgive me for not having read the whole thread, however, there is one thing that seems to be really important, and that is, ruby hardly ever runs the damned GC. It certainly doesn't do full runs nearly often enough (IMO). Also, implicit OOMEs or GC runs quite often DO NOT affect the exte

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Luis Lavena
On Mon, Mar 24, 2008 at 4:59 PM, Scott Windsor <[EMAIL PROTECTED]> wrote: > On Mon, Mar 24, 2008 at 12:18 PM, Luis Lavena <[EMAIL PROTECTED]> wrote: > > > > > > > On Mon, Mar 24, 2008 at 3:58 PM, Scott Windsor <[EMAIL PROTECTED]> wrote: > > > > > > > > You're using *RMagick*, not ImageMagick direct

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Scott Windsor
On Mon, Mar 24, 2008 at 12:18 PM, Luis Lavena <[EMAIL PROTECTED]> wrote: > On Mon, Mar 24, 2008 at 3:58 PM, Scott Windsor <[EMAIL PROTECTED]> wrote: > > > You're using *RMagick*, not ImageMagick directly. If you used the > later (via system calls) there will no be memory leakage you can worry > ab

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Luis Lavena
On Mon, Mar 24, 2008 at 3:58 PM, Scott Windsor <[EMAIL PROTECTED]> wrote: > > Right now my current deployment configuration for all my rails applications > is using apache + fastcgi. > > With this deployment strategy, if I don't set the garbage collection in my > dispatch.fcgi, any rails applicati

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Scott Windsor
On Tue, Mar 25, 2008 at 11:53 AM, Zed A. Shaw <[EMAIL PROTECTED]> wrote: > On Mon, 24 Mar 2008 08:21:52 -0700 > "Scott Windsor" <[EMAIL PROTECTED]> wrote: > > > Right now, my processes aren't gigantic... I'm preparing for a 'worst > case' > > scenario when I have a extremely large processes or mem

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Zed A. Shaw
On Mon, 24 Mar 2008 08:21:52 -0700 "Scott Windsor" <[EMAIL PROTECTED]> wrote: > Right now, my processes aren't gigantic... I'm preparing for a 'worst case' > scenario when I have a extremely large processes or memory usage. This can > easily happen on specific applications such as an image server

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Kirk Haines
On Mon, Mar 24, 2008 at 9:21 AM, Scott Windsor <[EMAIL PROTECTED]> wrote: > Right now, my processes aren't gigantic... I'm preparing for a 'worst case' > scenario when I have a extremely large processes or memory usage. This can > easily happen on specific applications such as an image server (us

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Steve Midgley
At 08:21 AM 3/24/2008, [EMAIL PROTECTED] wrote: >Message: 7 >Date: Mon, 24 Mar 2008 08:21:52 -0700 >From: "Scott Windsor" <[EMAIL PROTECTED]> >Subject: Re: [Mongrel] mongrel garbage collection >To: mongrel-users@rubyforge.org >Message-ID: > <[

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Scott Windsor
On Sat, Mar 22, 2008 at 3:39 AM, Dave Cheney <[EMAIL PROTECTED]> wrote: > > On 22/03/2008, at 8:19 AM, Steve Midgley wrote: > > If so, it seems like if one were using something like nginx fair proxy, > then the mongrel would be running it's garbage collection AFTER the > client got all its html bu

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Scott Windsor
If you plan on regularly killing your application (for whatever reason), then this is a pretty good option. This is a pretty common practice for apache modules and fastcgi applications as a hold-over from dealing with older leaky C apps. I'd personally prefer for my Ruby web apps to re-run the GC

Re: [Mongrel] mongrel garbage collection

2008-03-24 Thread Scott Windsor
On Fri, Mar 21, 2008 at 1:19 PM, Kirk Haines <[EMAIL PROTECTED]> wrote: > On Fri, Mar 21, 2008 at 1:23 PM, Scott Windsor <[EMAIL PROTECTED]> wrote: > > > I understand that the GC is quite knowledgeable about when to run > garbage > > collection when examining the heap. But, the GC doesn't know an

Re: [Mongrel] mongrel garbage collection

2008-03-22 Thread Dave Cheney
On 22/03/2008, at 8:19 AM, Steve Midgley wrote: If so, it seems like if one were using something like nginx fair proxy, then the mongrel would be running it's garbage collection AFTER the client got all its html but BEFORE any new requests were sent to it. In a fully loaded server it wouldn'

Re: [Mongrel] mongrel garbage collection

2008-03-21 Thread Steve Midgley
At 01:19 PM 3/21/2008, [EMAIL PROTECTED] wrote: >Date: Fri, 21 Mar 2008 14:19:01 -0600 >From: "Kirk Haines" <[EMAIL PROTECTED]> >Subject: Re: [Mongrel] mongrel garbage collection >To: mongrel-users@rubyforge.org >Message-ID: > <[EMAIL PROTECTED]> >

Re: [Mongrel] mongrel garbage collection

2008-03-21 Thread Evan Weaver
> The alternative is to terminate your application after N number of requests > and never run the > GC, which I'm not a fan of. WSGI (Python) can do that, and it's a pretty nice alternative to having Monit kill a leaky app that may have a bunch of requests queued up (Mongrel soft shutdown not wit

Re: [Mongrel] mongrel garbage collection

2008-03-21 Thread Evan Weaver
> You'll likely either end up using more RAM than you otherwise would > have in between GC calls, resulting in bigger processes This is definitely true. Keep in mind that the in-struct mark phase means that the entire process has to lurch out of swap whenever the GC runs. Since the process is now

Re: [Mongrel] mongrel garbage collection

2008-03-21 Thread Kirk Haines
On Fri, Mar 21, 2008 at 1:23 PM, Scott Windsor <[EMAIL PROTECTED]> wrote: > I understand that the GC is quite knowledgeable about when to run garbage > collection when examining the heap. But, the GC doesn't know anything about > my application or it's state. The fact that when the GC runs every

Re: [Mongrel] mongrel garbage collection

2008-03-21 Thread Scott Windsor
On Fri, Mar 21, 2008 at 11:49 AM, Kirk Haines <[EMAIL PROTECTED]> wrote: > On Fri, Mar 21, 2008 at 12:12 PM, Scott Windsor <[EMAIL PROTECTED]> > wrote: > > Sorry, for the re-post, but I'm new to the mailing list and wanted to > bring > > back up and old topic I saw in the archives. > > > > http://

Re: [Mongrel] mongrel garbage collection

2008-03-21 Thread Kirk Haines
On Fri, Mar 21, 2008 at 12:12 PM, Scott Windsor <[EMAIL PROTECTED]> wrote: > Sorry, for the re-post, but I'm new to the mailing list and wanted to bring > back up and old topic I saw in the archives. > > http://rubyforge.org/pipermail/mongrel-users/2008-February/004991.html > > I think a patch to d