On 25 Mar 2008, at 17:05, Steve Midgley wrote:
> At 03:41 AM 3/25/2008, [EMAIL PROTECTED] wrote:
>> Date: Tue, 25 Mar 2008 10:40:50 +0000
>> From: James Tucker <[EMAIL PROTECTED]>
>> Subject: Re: [Mongrel] mongrel garbage collection
>> To: mongrel-users@rubyforge.org
>> Message-ID: <[EMAIL PROTECTED]>
>> Content-Type: text/plain; charset="us-ascii"
>> [snip]
>> Also, implicit OOMEs or GC runs quite often DO NOT affect the
>> extensions correctly. I don't know what rmagick is doing under the
>> hood in this area, but having been generating large portions of
>> country maps with it (and moving away from it very rapidly), I know
>> the GC doesn't do "The Right Thing".
>> [snip]
>
> Hi James,
>
> My understanding with RMagick is that it is hooking the Imagemagick
> libs directly in process. As a result, memory is not always freed when
> you'd expect it to be. I haven't read up on the details, having chosen
> to just use out of process image management, but you might find this
> link interesting - in it, there is a claim that the latest releases of
> RMagick do *not* in fact leak any memory and that running a full GC
> manually will reclaim all memory it uses after the references are out
> of scope.
Thank you for kindly ensuring that I got this. We actually moved
completely away from anything ImageMagick based. There really wasn't
any sensible way to 'fix' it.
Whilst destroy! looks ok, even when doing what we were (high res
tiling, covering around 250 square miles), we found performance was
fine and we could avoid all allocation issues by using the crazy
thread solution (Thread.new { loop { sleep some_time; GC.start } }).
This is all good in most scenarios but then there are times when
running a framework like eventmachine, where threads (yes, even green
ones) can be total performance killers too. Mind you, under rails,
there's always a linear reaction run, so I'm not going to speculate
more on that detail. It's also OT for here, mostly...
> http://rubyforge.org/forum/forum.php?thread_id=1374&forum_id=1618
>
> Steve
Thanks again,
James.
P.S. Personally, if I was coming up against this problem today, I'd
drop out to a separate process, driven by something like background
job if under rails. If under pure ruby, I'd use drb or eventmachine +
a marshalling protocol, depending on specific requirements. The
biggest issue for our old project was hitting swap / page file. Image
rendering, when you're already working on the per-pixel layer, is
really easy to divide up, though, so optimizing for speed is pretty
easy really.
When it comes to background concurrent scheduling, staying away from
ACID can really help, too, but that really is another topic for
another time. Lets just say, allow slack, and life will be easier if
you ever hit a silly scale. I've seen people trying to recover broken
ACID implementations by trawling logs, and my god, tearful.
_______________________________________________
Mongrel-users mailing list
Mongrel-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-users
