Zed Shaw a écrit : > On Sun, 2006-05-28 at 19:38 +0200, François SIMOND wrote: > >> Here is a rudimentary patch against mongrel-0.3.12.4, that basically works ! >> I could not imagine that it would be so easy :) >> > > Here's the things that blow up--even with this setting--unless there's > been some *major* improvements in how Rails' was structured: > Too bad.. i thought that you had already made it but i wanted to try, encouraged by preliminar test results :) > * Classes mysteriously "disappear" or aren't loaded. Ruby's dynamic > class loading and instance_eval is not thread safe, so when multiple > threads load rails actions, models, and helpers the Dispatcher throws > random exceptions. > * Model objects used in one thread show up in another thread. Last time > this was because there were problems with how shared connections were > being used between the different threads. > * Requests mysteriously pick up other thread's request input or output. > No idea why this was. It's really really weird. > * High load causes database connections to explode exponentially into > the thousands. This is caused by use Thread.current[] to store the DB > connection, which causes the rails application to connect one time for > each thread. > * Once connected, the connection doesn't go away, but the thread does. > This means that when each thread is created, connects, and then finishes > it's response, the connection is kept around until the GC kills it. > This causes file descriptor leaks that eventually make the whole Mongrel > server stop functioning because the number of open files is greater than > the 1024 select() limit on most systems. > * Long running Mongrel servers start eating lots of memory or stop > running. This is mostly caused by the connections per thread model not > being cleaned up, but even when this is cleaned out it's still not good > enough. Files, other thread, popen calls, fork, and many other > resources or notoriously poorly managed by nearly every Rails > programmer. > For ActiveRecord related weirdness, i hope that it was adressed in 1.1.x by recent AR allow_concurrency modifications and connection pool patch. I neigher really have ideas for the wrong input or output.. Do you think that these threads were incorrectly finished ? ( like the example in http://groups.google.com/group/comp.lang.ruby/msg/0b79331e7d40e9b8 ) > I started working on fixing these problems way back in the SCGI days, > but the task was so daunting that I just gave up. So, yes, your little > patch is very easy. It probably won't work without a lot of extra > effort. > Were your fixes integrated in rails trunk ? > If you're interested in doing some hack-o-tronic work to make this > function, then you'll have to either rule out the above problems (with > more than one request with a browser) or find a way to implement fixes > outside of rails. Some possible solutions: > I love this word, "hack-o-tronic" :) but i do not know if i'll like to do it..
> 1) Rip out anything left over in a Thread.current[] after the rails > request is done. > Like Thread.current = nil GC.start ? :) > 2) Find a way to mark what request/response/IO objects are assigned to > each thread, and then blow-up if they change during processing. This > would most likely be a debugging option for people suspecting crossover. > Maybe it could be done by re-defining and logging each problematic operation. > 3) Find a way to keep track of what a rails controller opens, and then > make sure it gets closed completely. This'll be a real realy pain in > the ass since people love to open crap at random and not clean it up. > Right. That's the reason why rails team has disabled allow_concurrency by default. > 4) Do a completely pre-loading when a rails application is started in > production mode so that all classes are properly loaded ahead of time > and you don't have to worry about Ruby's lack of thread safety here. > An appropriate script could do that at mongrel rails startup, looking in standard or configured paths, and loading every class first. > If you're up to the challenge, then give it a shot. You'll want to > setup a test harness with your current code that thrashes the living > hell out of a real rails application for a long period of time and then > fix everything that comes up. My primary goal is to find solutions for a rails app that i'm currently developping (a free personnal and social photo gallery). This application need some long-time processings and uploads, and will be installed in shared hosting too. I'll maybe use BackgrounDRb for the image resize operations, but I think that i can build it better if i'm not much restricted in terms of time of concurrential requests, so I'll try ! Do you have testcases that I could use as a quickstart ? And thank you Zed for a such detailled response ! -- Francois Simond _______________________________________________ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
