Re: [Mongrel] Determining ideal number of Mongrels for an app?
On Nov 28, 2006, at 15:49 , Zed A. Shaw wrote: Many folks pointed you at the semi-official tuning documentation. 20 is totally over kill and will hurt you more than you know. I find it's about 2-4/CPU depending on the amount of ram you have and the speed of your CPU. BUT YOU SHOULD TUNE EVERY APP EVERY TIME NO MATTER WHAT YOU CHANGED. Don't believe anything as gospel, always test it yourself. Zed, Do dual-core chips count as two CPU's for this calculation? Also what is the recommended amount of ram per mongrel process or per CPU based on your experience? Thanks, ~Wayne ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
On Thu, 30 Nov 2006 00:22:03 -0500 Zed A. Shaw [EMAIL PROTECTED] wrote: Do believe anything I said above. Correction: DON'T believe anything I said above... -- Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu http://www.zedshaw.com/ http://www.awprofessional.com/title/0321483502 -- The Mongrel Book http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
On 11/28/06, Joe Ruby [EMAIL PROTECTED] wrote: What's a rule of thumb for guesstimating how many Mongrels to use in a cluster for an app? I have an app that gets about 5000 unique visitors per day. I figured I'd give it plenty of Mongrels -- twenty to be specific. After running out of memory and hitting the swap periodically, I scaled it back to five and it still seems to serve up visitors fine. So, is there some super-secret math fomula I can use to determine the ideal number of Mongrels I should start in a cluster for an app? See this: http://mongrel.rubyforge.org/docs/how_many_mongrels.html ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
Hey Joe, here's the link I have bookmarked for this: http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html As far as I know, that's still the way to go. Cheers, greg - Joe Ruby [EMAIL PROTECTED] wrote: What's a rule of thumb for guesstimating how many Mongrels to use in a cluster for an app? I have an app that gets about 5000 unique visitors per day. I figured I'd give it plenty of Mongrels -- twenty to be specific. After running out of memory and hitting the swap periodically, I scaled it back to five and it still seems to serve up visitors fine. So, is there some super-secret math fomula I can use to determine the ideal number of Mongrels I should start in a cluster for an app? Joe Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
On Tue, Nov 28, 2006 at 11:03:06AM -0600, Philip Hallstrom wrote: What's a rule of thumb for guesstimating how many Mongrels to use in a cluster for an app? I have an app that gets about 5000 unique visitors per day. I figured I'd give it plenty of Mongrels -- twenty to be specific. After running out of memory and hitting the swap periodically, I scaled it back to five and it still seems to serve up visitors fine. So, is there some super-secret math fomula I can use to determine the ideal number of Mongrels I should start in a cluster for an app? http://mongrel.rubyforge.org/docs/how_many_mongrels.html Just for reference, we get about 500,000 pages a day and run 4 mongrel processes... we've got about 20 servers[1], but 4 gives us the best performance. I'm curious - do you have 4 * 20 Mongrels or just 4? How many requests/sec does Rails report in your logs? -- Thanks, - Jacob Atzen ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
On Tue, 28 Nov 2006 08:27:38 -0800 (PST) Joe Ruby [EMAIL PROTECTED] wrote: What's a rule of thumb for guesstimating how many Mongrels to use in a cluster for an app? I have an app that gets about 5000 unique visitors per day. Pathetic. 5000/day? Sad. Oh wait, is that 5000 visitors doing 100k request/second during the whole day, or 5000 visitors doing .005 request/second at noon? (I'm jokingly saying you should find out what their usage pattern is and translate that to req/sec processed and kb/second transfered first.) I figured I'd give it plenty of Mongrels -- twenty to be specific. After running out of memory and hitting the swap periodically, I scaled it back to five and it still seems to serve up visitors fine. So, is there some super-secret math fomula I can use to determine the ideal number of Mongrels I should start in a cluster for an app? Many folks pointed you at the semi-official tuning documentation. 20 is totally over kill and will hurt you more than you know. I find it's about 2-4/CPU depending on the amount of ram you have and the speed of your CPU. BUT YOU SHOULD TUNE EVERY APP EVERY TIME NO MATTER WHAT YOU CHANGED. Don't believe anything as gospel, always test it yourself. Now, lately I've been wanting to add a couple of qualifiers on the scaling issue related to your resource usage. In some situations people are using up tons of ram or tons of files. If you have 2-4 mongrels you quickly outstrip the ram and files. For these folks I tell them to do a different test that tries to do a lot of concurrent but slow connections to eat up as much ram and file handles as they can. Then, they expand the mongrels out to handle the number of connections they need. Many times, people in these situations have to buy more machines. The main culprit of this is RMagick. If you have RMagick you need to simulate a ton of people concurrently slamming you with files to make sure you have the resources. This is also a reasonably good description of file limit defaults and HTTP performance: http://www.xenoclast.org/doc/benchmark/HTTP-benchmarking-HOWTO/node7.html Play with these settings as well for each, maybe even just increasing the defaults. -- Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu http://www.zedshaw.com/ http://www.awprofessional.com/title/0321483502 -- The Mongrel Book http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
On Tue, Nov 28, 2006 at 11:03:06AM -0600, Philip Hallstrom wrote: What's a rule of thumb for guesstimating how many Mongrels to use in a cluster for an app? I have an app that gets about 5000 unique visitors per day. I figured I'd give it plenty of Mongrels -- twenty to be specific. After running out of memory and hitting the swap periodically, I scaled it back to five and it still seems to serve up visitors fine. So, is there some super-secret math fomula I can use to determine the ideal number of Mongrels I should start in a cluster for an app? http://mongrel.rubyforge.org/docs/how_many_mongrels.html Just for reference, we get about 500,000 pages a day and run 4 mongrel processes... we've got about 20 servers[1], but 4 gives us the best performance. I'm curious - do you have 4 * 20 Mongrels or just 4? How many requests/sec does Rails report in your logs? We have 4 mongrels per server. Just did a quick grep on one of the servers log files and for about 6000 requests we are averaging 38 reqs/sec with a standard deviation of 113. If I throw out about 2700 requests that I know are slow ( 5 req/s mostly cause we can't cache them) the average goes to 65 reqs/sec. Like I said, we only need those for a little bit every year... during the world series of poker. People really like there poker :) http://www.alexaholic.com/cardplayer.com+penny-arcade.com+basecamphq.com -philip ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
On Tue, 28 Nov 2006 15:23:14 -0600 (CST) Philip Hallstrom [EMAIL PROTECTED] wrote: On Tue, Nov 28, 2006 at 11:03:06AM -0600, Philip Hallstrom wrote: Just for reference, we get about 500,000 pages a day and run 4 mongrel processes... we've got about 20 servers[1], but 4 gives us the best performance. I'm curious - do you have 4 * 20 Mongrels or just 4? How many requests/sec does Rails report in your logs? Just did a quick grep on one of the servers log files and for about 6000 requests we are averaging 38 reqs/sec with a standard deviation of 113. Standard Deviation! YES! -- Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu http://www.zedshaw.com/ http://www.awprofessional.com/title/0321483502 -- The Mongrel Book http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
On Tue, 28 Nov 2006 08:27:38 -0800 (PST) Joe Ruby joeat303 at yahoo.com wrote: What's a rule of thumb for guesstimating how many Mongrels to use in a cluster for an app? I have an app that gets about 5000 unique visitors per day. Pathetic. 5000/day? Sad. Oh wait, is that 5000 visitors doing 100k request/second during the whole day, or 5000 visitors doing .005 request/second at noon? It's both! Yeah, it's pathetic and sad, but you think I trust my enterprisey apps to this script-kiddy RoR stuff?!? :P (I'm jokingly saying you should find out what their usage pattern is and translate that to req/sec processed and kb/second transfered first.) I figured I'd give it plenty of Mongrels -- twenty to be specific. After running out of memory and hitting the swap periodically, I scaled it back to five and it still seems to serve up visitors fine. So, is there some super-secret math fomula I can use to determine the ideal number of Mongrels I should start in a cluster for an app? Many folks pointed you at the semi-official tuning documentation. 20 is totally over kill and will hurt you more than you know. I find it's about 2-4/CPU depending on the amount of ram you have and the speed of your CPU. BUT YOU SHOULD TUNE EVERY APP EVERY TIME NO MATTER WHAT YOU CHANGED. Don't believe anything as gospel, always test it yourself. Now, lately I've been wanting to add a couple of qualifiers on the scaling issue related to your resource usage. In some situations people are using up tons of ram or tons of files. If you have 2-4 mongrels you quickly outstrip the ram and files. For these folks I tell them to do a different test that tries to do a lot of concurrent but slow connections to eat up as much ram and file handles as they can. Then, they expand the mongrels out to handle the number of connections they need. Many times, people in these situations have to buy more machines. I know you like doing long lines (scroll, scroll, scroll), but check it, yo: http://rubyforge.org/pipermail/mongrel-users/2006-November/002258.html A joke page about Mongrel MUDCRAP certification that hasn't changed but is now accesible from the site menu: http://mongrel.rubyforge.org/certified.html A JOKE?!? W. T. F. ?!? The main culprit of this is RMagick. If you have RMagick you need to simulate a ton of people concurrently slamming you with files to make sure you have the resources. This is also a reasonably good description of file limit defaults and HTTP performance: http://www.xenoclast.org/doc/benchmark/HTTP-benchmarking-HOWTO/node7.html Play with these settings as well for each, maybe even just increasing the defaults. But seriously, thanks for the tips. :) Joe -- Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu http://www.zedshaw.com/ http://www.awprofessional.com/title/0321483502 -- The Mongrel Book http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help. Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Determining ideal number of Mongrels for an app?
A joke page about Mongrel MUDCRAP certification that hasn't changed but is now accesible from the site menu: http://mongrel.rubyforge.org/certified.html A JOKE?!? W. T. F. ?!? No kidding, I'm still waiting for Mongrel Business Enterprise Edition for the Marketplace... Deluxe. :^) Bryan Thompson Loy Inc (Sorry if this double-posts... wonky email settings here.) ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
