[Mongrel] Configuring Mongrel logging
Is there any way to configure Mongrel's logging? Specifically, I'd like to get it to output to syslog. Our Rails apps already do this (via SyslogLogger) - is there any way to persuade Mongrel to do the same? Thanks! -- paul.butcher-msgCount++ Snetterton, Castle Combe, Cadwell Park... Who says I have a one track mind? MSN: [EMAIL PROTECTED] AIM: paulrabutcher Skype: paulrabutcher LinkedIn: https://www.linkedin.com/in/paulbutcher ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Recommentation: Sessions and PStore
Zed wrote: * Using the mysql.rb instead of installing the actual mysql compiled gem. The mysql.rb is just for development, and it sucks horribly. Please forgive the stupid question, but how does one tell whether one is using the mysql compiled gem or not? I've installed it, apparently successfully, but I'd feel happier if I could be certain that Rails was using it... Thanks! -- paul.butcher-msgCount++ Snetterton, Castle Combe, Cadwell Park... Who says I have a one track mind? MSN: [EMAIL PROTECTED] AIM: paulrabutcher Skype: paulrabutcher LinkedIn: https://www.linkedin.com/in/paulbutcher ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
[Mongrel] Why Rails + mongrel_cluster + load balancing doesn't work for us and the beginning of a solution
We have been searching for a Rails deployment architecture which works for
us for some time. We've recently moved from Apache 1.3 + FastCGI to Apache
2.2 + mod_proxy_balancer + mongrel_cluster, and it's a significant
improvement. But it still exhibits serious performance problems.
We have the beginnings of a fix that we would like to share.
To illustrate the problem, imagine a 2 element mongrel cluster running a
Rails app containing the following simple controller:
class HomeController ApplicationController
def fast
sleep 1
render :text = I'm fast
end
def slow
sleep 10
render :text = I'm slow
end
end
and the following test app
#!/usr/bin/env ruby
require File.dirname(__FILE__) + '/config/boot'
require File.dirname(__FILE__) + '/config/environment'
end_time = 1.minute.from_now
fast_count = 0
slow_count = 0
fastthread = Thread.start do
while Time.now end_time do
Net::HTTP.get 'localhost', '/home/fast'
fast_count += 1
end
end
slowthread = Thread.start do
while Time.now end_time do
Net::HTTP.get 'localhost', '/home/slow'
slow_count += 1
end
end
fastthread.join
slowthread.join
puts Fast: #{fast_count}
puts Slow: #{slow_count}
In this scenario, there will be two requests outstanding at any time, one
fast and one slow. You would expect approximately 60 fast and 6 slow
GETs to complete over the course of a minute. This is not what happens;
approximately 12 fast and 6 slow GETs complete per minute.
The reason is that mod_proxy_balancer assumes that it can send multiple
requests to each mongrel and fast requests end up waiting for slow requests,
even if there is an idle mongrel server available.
We've experimented with various different configurations for
mod_proxy_balancer without successfully solving this issue. As far as we can
tell, all other popular load balancers (Pound, Pen, balance) behave in
roughly the same way.
This is causing us real problems. Our user interface is very time-sensitive.
For common user actions, a page refresh delay of more than a couple of
seconds is unacceptable. What we're finding is that if we have (say) a
reporting page which takes 10 seconds to display (an entirely acceptable
delay for a rarely-used report) then our users are seeing similar delays on
pages which should be virtually instantaneous (and would be, if their
requests were directed to idle servers). Worse, we're occasionally seeing
unnecessary timeouts because requests are queuing up on one server.
The real solution to the problem would be to remove Rails' inability to
handle more than one thread. In the absence of that solution, however, we've
implemented (in Ruby) what might be the world's smallest load-balancer. It
only ever sends a single request to each member of the cluster at a time.
It's called HighWire and is available on RubyForge (no Gem yet - it's on the
list of things to do!):
svn checkout svn://rubyforge.org/var/svn/highwire
Using this instead of mod_proxy_balancer, and running the same test script
above, we see approximately 54 fast and 6 slow requests per minute.
HighWire is very young and has a way to go. It's not had any serious
optimization or testing, and there are a bunch of things that need doing
before it can really be considered production ready. But it does work for
us, and does produce a significant performance improvement.
Please check it out and let us know what you think.
--
paul.butcher-msgCount++
Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?
MSN: [EMAIL PROTECTED]
AIM: paulrabutcher
Skype: paulrabutcher
LinkedIn: https://www.linkedin.com/in/paulbutcher
___
Mongrel-users mailing list
Mongrel-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Why Rails + mongrel_cluster + load balancing doesn't work for us and the beginning of a solution
Zed Shaw zedshaw at zedshaw.com wrote But, if your interface is time sensitive then why does it have actions that take too much time? See the conundrum there? I was kinda expecting that response, Zed, but I didn't want to rob you of the pleasure of saying it ;-) We are in fact using backgroundrb for a number of long-running actions. It's fantastic, but it isn't ever going to solve the problem for us completely. There are a whole bunch of reasons, which may be specific to our particular application, but I doubt it. First the less convincing arguments: 1) We're lazy (in a good way). Our application is used by hundreds of our employees, 24 hours a day. We, correctly, invest a great deal of time making sure that the things that they do over and over again are as efficient as they possibly can be. We don't, however, believe that it's appropriate for us to spend time optimising things which are only used by one or two guys once or twice a day. It's not as if we have a lack of things to spend our time on, and we'd rather spend that time on things that really matter, rather than optimising things which only need to be optimised because the performance characteristics of Rails mean that one poorly performing action results in *all* actions performing poorly. 2) We're not infallible. Sometimes we screw up and release a version of the software where one of our actions takes longer than it should. This is bad if it only affects the action we screwed up, but if that's all it does then it's not a disaster. If one slow action screws up *all* the actions, however, that is a disaster. 3) Even if we spend all of the time we possibly can optimising actions, different actions will always take different amounts of time to execute. It's not as if there's one true execution time for an action. The bad effects become apparent whenever you have any two actions which take noticeably different amounts of time to execute. Yes, in the example that I gave it was 1 second and 10 seconds. But the same basic effect would be there if we were talking about 1ms and 10ms. Those arguments would be enough, I believe. But in our case there's one much stronger argument which trumps all the above IMHO: 4) It's not possible to predict the time that an action, even a heavily optimised one, will take. The vagaries of cacheing, swapping etc mean that this is simply out of our hands. In our case, for example, many of our actions involve fulltext searches of the database (we have no choice about this - it's fundamental to the nature of what we do). The performance of a fulltext search is *extremely* unpredictable (in MySQL anyway). There can be occasions where the first time a particular search is done, it takes 10 seconds. The second time, because the database has got the idea, it can take a few milliseconds. Backgroundrb cannot solve this problem - if we sent all of these actions to backgroundrb, pretty much *every* action would end up being sent to backgroundrb and then we end up with a very similar load balancing problem - it just becomes a problem of load balancing to backgroundrb instead of to mongrel. Make sense? -- paul.butcher-msgCount++ Snetterton, Castle Combe, Cadwell Park... Who says I have a one track mind? MSN: [EMAIL PROTECTED] AIM: paulrabutcher Skype: paulrabutcher LinkedIn: https://www.linkedin.com/in/paulbutcher ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Why Rails + mongrel_cluster + load balancing doesn'twork for us and the beginning of a solution
Have you ever use haproxy http://haproxy.1wt.eu/ ?! In a word, no :-) He have the following feature which can help you: Thanks - sounds interesting. We'll give it a go! -- paul.butcher-msgCount++ Snetterton, Castle Combe, Cadwell Park... Who says I have a one track mind? MSN: [EMAIL PROTECTED] AIM: paulrabutcher Skype: paulrabutcher LinkedIn: https://www.linkedin.com/in/paulbutcher ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Design flaw? - num_processors, accept/close
On 16 Oct 2007, at 06:49, Zed A. Shaw wrote: On Mon, 15 Oct 2007 16:43:34 -0700 Brian Williams [EMAIL PROTECTED] wrote: We recently ran into exactly this issue. Some rails requests were making external requests that were taking 5 minutes (networking issues out of our control). Now that's a design flaw. If you're expecting the UI user to wait for a backend request that takes 5 minutes then you need to redesign the workflow and interface. Do it like asynchronous email where the use sends a request, awaits a reply, reads the reply, and doesn't deal with the backend processing chain of events. Zed, you're being obtuse. Of course that isn't what Brian means. What he's doing is giving a pathological example to illustrate just how badly the mod_proxy_balancer/mongrel/rails combination behaves when things go wrong. Yes, you can mask the problem to some extent by mucking about with your application (and in fact that's what we've done here), but that's missing the point. It is not unreasonable to expect that some actions performed by an application are fast and some are slow. It's further not unreasonable to expect a very large difference between the fastest and the slowest actions (if one action takes 10ms and another takes 1s, that's not unreasonable - but it is a 2 order of magnitude difference). With the obvious setup, fast actions will be delayed behind slow actions. This is a Bad Thing. Furthermore, people are fallible. If I happen to accidentally introduce an action into my system which takes 10s, yes I've screwed up and should fix it. But is it reasonable for the fact that I have a single (possibly very rare) action which takes 10s to mean that all the other fast actions are affected? Even when most of my mongrels are idle? Of course, this isn't really a problem with Mongrel. It's a problem with Ruby (which doesn't know what the word thread means) and Rails (which doesn't even manage to successfully make use of the brain-dead version of threading which Ruby does support). -- paul.butcher-msgCount++ Snetterton, Castle Combe, Cadwell Park... Who says I have a one track mind? http://www.paulbutcher.com/ LinkedIn: https://www.linkedin.com/in/paulbutcher MSN: [EMAIL PROTECTED] AIM: paulrabutcher Skype: paulrabutcher ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Workaround found for request queuing vs. num_processors, accept/close
On 15 Oct 2007, at 21:52, Robert Mela wrote: I've discovered a setting in mod_proxy_balancer that prevents the Mongrel/Rails request queuing vs. accept/close problem from ever being reached. Thanks for that, Robert. We've hit exactly the same issue in the past, but have been unable to find a way to persuade mod_proxy_balancer to do the right thing. I posted about this issue here a year or so ago: http://rubyforge.org/pipermail/mongrel-users/2006-September/ 001653.html But was unable to get anyone to take it seriously :-( -- paul.butcher-msgCount++ Snetterton, Castle Combe, Cadwell Park... Who says I have a one track mind? http://www.paulbutcher.com/ LinkedIn: https://www.linkedin.com/in/paulbutcher MSN: [EMAIL PROTECTED] AIM: paulrabutcher Skype: paulrabutcher ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
