[Mongrel] Problem getting Mongrel to start at startup
Hi,I'm running CentOS 4.3 and the latest Mongrel (0.3.13.3) and mongrel_cluster (0.2.0). I have set up a cluster, following the instructions at http://mongrel.rubyforge.org/docs/mongrel_cluster.html . I created a symbolic link in the directory /etc/mongrel_cluster . When I run mongrel_rails cluster::start -c test.yml, the mongrel servers start up. But at boot, the command '/sbin/service mongrel_cluster start' gets run, when I run it I get the error: Starting all mongrel_clusters.../usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27: command not found: mongrel_rails cluster::start -c test.ymlmongrel_rails cluster::start returned an error. I have tried many things, but nothing works. I hope somebody can help me out. Thanks.Kind regards,Nick ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Problem getting Mongrel to start at startup
Hi,I found the problem. I had ruby installed from source and it ended up in /usr/local/bin, so I had to create the following extra symbolic links to get it working.ln -s /usr/local/bin/ruby /usr/bin/ruby ln -s /usr/local/bin/mongrel_cluster_ctl /usr/bin/mongrel_cluster_ctlln -s /usr/local/bin/mongrel_rails /usr/bin/mongrel_railsKind regards,NickOn 8/22/06, Nick Snels [EMAIL PROTECTED] wrote: Hi,I'm running CentOS 4.3 and the latest Mongrel (0.3.13.3) and mongrel_cluster (0.2.0). I have set up a cluster, following the instructions at http://mongrel.rubyforge.org/docs/mongrel_cluster.html . I created a symbolic link in the directory /etc/mongrel_cluster . When I run mongrel_rails cluster::start -c test.yml, the mongrel servers start up. But at boot, the command '/sbin/service mongrel_cluster start' gets run, when I run it I get the error: Starting all mongrel_clusters.../usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27: command not found: mongrel_rails cluster::start -c test.ymlmongrel_rails cluster::start returned an error. I have tried many things, but nothing works. I hope somebody can help me out. Thanks.Kind regards,Nick ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
[Mongrel] Mongrel and Service behave differently
I have a rails application running under the latest version of Mogrel Win32. Seems to run just fine. However, when I attempt to run it under the Win32 Mongrel service, it appears to hand after hitting certain controller actions. I'm sure there is a bug in my code that's causing this - but I just am not sure what it is yet. Is there anything about the mongrel-service that would cause an application to behave differently? Besides the -B option to turn on debugging, are there any other tricks for me to get visibility to what is going on underneath the covers? If I can't track down this issue, is there any to using mongrel_rails from a scheduler instead of the service?Thanks!-- -- Jay TurpinOne test is worth a thousand expert opinions. - Bill Nye (The Science Guy) ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
[Mongrel] Multiple mongrels for one app
Hi mongrel-users, This is my first post, so I'm not sure if it's been asked before, but I can't find an answer anywhere. If I have one rails application running, one processor I'm running it on, and mongrel is multi-threaded, why should I have more than one mongrel running? Everyone seems to agree on 3-5 mongrels per rails app, but why? I must be missing something, sorry if it's obvious. TIA --hax ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
This is my first post, so I'm not sure if it's been asked before, but I can't find an answer anywhere. If I have one rails application running, one processor I'm running it on, and mongrel is multi-threaded, why should I have more than one mongrel running? Everyone seems to agree on 3-5 mongrels per rails app, but why? Performance... somewhere Zed has an article on how to benchmark things.. something along the lines of benchmark a blank static html page. That's as fast as your server will go. Then create a simple action that renders Hello as text say, with one mongrel. Benchmark that. Keep increasing the number of mongrels until you don't get any more performance improvements. For us, it was 4. -philip ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] IIS and response timeout
Are you using SQL Server? I think the connection times-out after 30s, so you're probably seeing a SQL Timeout error. On 8/19/06, Jay Turpin [EMAIL PROTECTED] wrote: I apologize if this has been covered before, I checked the archives and didn't see anything over the last few months. I am running IIS6 with IsapiRewriter and a single mongrel service for each Rails application. One of my apps does a long query to the database to produce a CSV file. After the request has gone beyond 30 seconds, the user get a 500 Internal Server Error. I've looked around for ways to adjust the response timeout, either in IIS or mongrel, but haven't found anything yet. Any ideas? -- -- Jay Turpin One test is worth a thousand expert opinions. - Bill Nye (The Science Guy) ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
Rails is not threadsafe. http://rubyforge.org/pipermail/mongrel-users/2006-August/000930.html //jarkko So what happens when the mongrel gets more than one request at a time? When I run httpref against it, the mongrel process can serve up large number of rails pages before it gets bogged down. And when it does break, it says mongrel timed out this thread: too many open files. So it must be doing some threading, even if rails isn't threadsafe. If it was 1 request per mongrel process, 1 process shouldn't be able to serve up very many per second. httpref tells me its prettty fast though. Thoughts? --hax ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
So what happens when the mongrel gets more than one request at a time? When I run httpref against it, the mongrel process can serve up large number of rails pages before it gets bogged down. And when it does break, it says mongrel timed out this thread: too many open files. So it must be doing some threading, even if rails isn't threadsafe. If it was 1 request per mongrel process, 1 process shouldn't be able to serve up very many per second. httpref tells me its prettty fast though. Thoughts? It will accept multiple connections but it will only feed one connection at a time through the rails handler, because as others have stated rails is not thread safe. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
On 8/22/06, hax [EMAIL PROTECTED] wrote: So what happens when the mongrel gets more than one request at a time? When I run httpref against it, the mongrel process can serve up large number of rails pages before it gets bogged down. And when it does break, it says mongrel timed out this thread: too many open files. So it must be doing some threading, even if rails isn't threadsafe. If it was 1 request per mongrel process, 1 process shouldn't be able to serve up very many per second. httpref tells me its prettty fast though. Take a look at the Rails handler. It's pretty easy code to read. There is a guard mutex that makes sure that the Rails handler only processes a single request through Rails at a time. So, Mongrel receives your requests and Mongrel multithreads them, but the Rails handler only eats them one at a time. If Rails could safely multithread, then the guard mutex and it's synchronization (which you can turn off already if you really want to) wouldn't be required, and overall performance on a single Mongrel, for Rails, should be better. Kirk Haines ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
http://rubyforge.org/pipermail/mongrel-users/2006-August/000930.html //jarkko So what happens when the mongrel gets more than one request at a time? When I run httpref against it, the mongrel process can serve up large number of rails pages before it gets bogged down. And when it does break, it says mongrel timed out this thread: too many open files. So it must be doing some threading, even if rails isn't threadsafe. If it was 1 request per mongrel process, 1 process shouldn't be able to serve up very many per second. httpref tells me its prettty fast though. Search the lists... I don't remember the exact details but basically mongrel is threaded... right up till it hands the request off to rails, then it locks, does the rails request, gets the response, and goes back into threaded mode. So technically you can't process more than 1 rails request at a time, but since many requests are way under a second you get the appearance that you can. Or something like that :) ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
[Mongrel] Mongrel crashes - bad doggie
Hello List, I have a production machine with very low ram (Xen Virtual Server) that runs mysql and lighty-pound-mongrel (2 instances). About once a day, one of the mongrels just locks up, leaving that in its log files: *** glibc detected *** /usr/bin/ruby: free(): invalid next size (fast): 0x0a744780 *** That process (mongrel) will stay locked, not answer any connections anymore and just generally behave bad; for instance kill -s SIGKILL PID will give me a 'Permission denied', even though I am impersonating root. The only real solution I have had is to reboot the server, which is obviously very bad. Has this behavior been observed before? If not, what can I do to help tracking down this bug? Is it even a mongrel bug? I mean it could be that mongrel gets back at me because I feed it not enough RAM. But even so, It should remain sigkillable, should it not? Thank you, kaspar ruby 1.8.4 (2005-12-24) [i386-linux] mongrel (0.3.13.3) Rails 1.1.6 ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
Hi I'll have a shot ;) If I have one rails application running, one processor I'm running it on, and mongrel is multi-threaded, why should I have more than one mongrel running? Mongrel might be internally multi threaded, but it only serves one request at any given moment. So all your web users will have to be serialized through this rather small .. tube. By setting up load balancing via pound to a few mongrels, you can take better advantage of your CPU, running things in parallel. Large-scale rails applications are deployed this way, mostly on more than one machine even to distribute the load across many CPU's and disks. Hope that explains... kaspar neotrivium.com - the swiss ruby shop ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
On Tue, 2006-08-22 at 11:31 -0500, Philip Hallstrom wrote: This is my first post, so I'm not sure if it's been asked before, but I can't find an answer anywhere. If I have one rails application running, one processor I'm running it on, and mongrel is multi-threaded, why should I have more than one mongrel running? Everyone seems to agree on 3-5 mongrels per rails app, but why? Performance... somewhere Zed has an article on how to benchmark things.. something along the lines of benchmark a blank static html page. That's as fast as your server will go. Then create a simple action that renders Hello as text say, with one mongrel. Benchmark that. Keep increasing the number of mongrels until you don't get any more performance improvements. For us, it was 4. Wow, someone listened. Nice job Philip. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
On Tue, 2006-08-22 at 11:47 -0400, hax wrote: Hi mongrel-users, This is my first post, so I'm not sure if it's been asked before, but I can't find an answer anywhere. If I have one rails application running, one processor I'm running it on, and mongrel is multi-threaded, why should I have more than one mongrel running? Everyone seems to agree on 3-5 mongrels per rails app, but why? I must be missing something, sorry if it's obvious. You've got three things you're dealing with when you do this: 1) Ruby uses a select() function to make it seem like it has threads. This is called green threads in the Java world. When you put about 2-4 mongrels per CPU on your machine, you get a slightly better utilization. 2) Rails isn't thread safe (as lots of people have said) so requests come in to mongrel, get threaded up, and then queued behind Rails. Once Rails is done then threads send the response out. If you have a small site and most of your rails actions are quick then don't worry about it. If you need more concurrency then you have to use more processes. 3) Because ruby uses select() it has a OS kernel level limit for the number of files it can handle. Usually that's 1024 but it can be 256 on some systems. If you have one mongrel process then you'll really quickly run out of these when serving files, doing fragment/action caching, etc. You have to add more processes to be able to handle more open files concurrently. That's the why. If you're site is small then just do one. It's easier. Then slowly expand it out and get more hardcore as you need it. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Multiple mongrels for one app
On Tue, 2006-08-22 at 15:11 -0400, hax wrote: Rails is not threadsafe. http://rubyforge.org/pipermail/mongrel-users/2006-August/000930.html //jarkko So what happens when the mongrel gets more than one request at a time? When I run httpref against it, the mongrel process can serve up large number of rails pages before it gets bogged down. And when it does break, it says mongrel timed out this thread: too many open files. So it must be doing some threading, even if rails isn't threadsafe. If it was 1 request per mongrel process, 1 process shouldn't be able to serve up very many per second. httpref tells me its prettty fast though. You actually can find out for yourself right now. I have the latest pre-release up with a new debugging feature just for folks like you. First, install the pre-release like this: sudo gem install mongrel --source=http://mongrel.rubyforge.org/releases/ Then run your application in production mode like normal. Right after it starts up, do this: killall -USR1 mongrel_rails Go look at the log/mongrel.log and you'll see that it says it set $mongrel_debug_client to true. The USR1 signal toggles this client debugging true/false. When it's true Mongrel will print out log messages that help you debug client interactions: * BAD CLIENT is only logged now when USR1 is on. * Full request data is printed out when there's an HTTP parsing error so you can see what the problem is. * It will print the number of threads waiting for rails on 10 second intervals. * It prints stack traces for exceptions found (I'm expanding this more). It's also very light, so you could enable it all the time if you want. Now, what you want to do is hit your app with httperf like this: httperf --server 127.0.0.1 --port 3000 --uri /some/rails/uri --num-conns 1000 --rate 100 And then watch your mongrel.log print out the request counts as they pile up behind rails and then move out. Play with num-conns and rate until you can choke mongrel, find the sweet spot, etc. Have fun and let me know if you find bugs. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Mongrel crashes - bad doggie
On Tue, 2006-08-22 at 22:00 +0200, Kaspar Schiess wrote: Hello List, I have a production machine with very low ram (Xen Virtual Server) that runs mysql and lighty-pound-mongrel (2 instances). About once a day, one of the mongrels just locks up, leaving that in its log files: *** glibc detected *** /usr/bin/ruby: free(): invalid next size (fast): 0x0a744780 *** That's super deep in the ruby interpreter. My thinking is that you've just run out of ram and this is what happens to ruby when a malloc fails. Try out the latest pre-release to see if you still have the lock-ups: sudo gem install mongrel --source=http://mongrel.rubyforge.org/releases/ But the ram problem is only solved by adding more ram or swap. http://www.netadmintools.com/art1.html -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
[Mongrel] Recommendation: Install The Real MySQL Driver
Hi Folks, I've been receiving lots of reports of the following problems: 1) Huge pauses. 2) Slow response on heavy queries. 3) Lost MySQL connections. 4) General database failures. What I've been able to confirm is that this is almost always caused by people using the default mysql.rb that comes with rails in their production setups. ** Please, if you run your Rails application in production then install the compiled mysql gem for your platform: gem install mysql And make sure you always have the latest. A lot of these problems turn out because of differences in the protocol of more recent mysql and the mysql.rb implementation in Rails. Mongrel is just making them worse because it's able to stress the driver more (at least I think so). Thanks a bunch. And please stop by the Lingr room if you have potential bug reports (link in my signature). -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] IIS and response timeout
I think the timeouts were occurring becasue of a combination of IsapiRewriter being configured incorrectly and security settings on the files prompting users for login/passwords. Once I got both of those things straighted out, everything is working great! Thanks for the advice!-- -- Jay TurpinOne test is worth a thousand expert opinions. - Bill Nye (The Science Guy) On 8/22/06, Ed C. [EMAIL PROTECTED] wrote: Are you using SQL Server? I think the connection times-out after 30s,so you're probably seeing a SQL Timeout error.On 8/19/06, Jay Turpin [EMAIL PROTECTED] wrote: I apologize if this has been covered before, I checked the archives and didn't see anything over the last few months. I am running IIS6 with IsapiRewriter and a single mongrel service for each Rails application. One of my apps does a long query to the database to produce a CSV file. After the request has gone beyond 30 seconds, the user get a 500 Internal Server Error. I've looked around for ways to adjust the response timeout, either in IIS or mongrel, but haven't found anything yet. Any ideas? -- -- Jay Turpin One test is worth a thousand expert opinions. - Bill Nye (The Science Guy) ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
