Re: Small Redundant web/mail setup
- Original Message - From: Damian Wiest [EMAIL PROTECTED] To: freebsd-questions@freebsd.org Sent: Monday, October 23, 2006 1:00 PM Subject: Re: Small Redundant web/mail setup On Wed, Oct 18, 2006 at 11:57:04PM -0700, Ted Mittelstaedt wrote: - Original Message - From: Ian Lord [EMAIL PROTECTED] To: freebsd-questions@freebsd.org Sent: Wednesday, October 18, 2006 5:34 AM Subject: Small Redundant web/mail setup Hi, I need to setup a high-availability setup for mail/web setup I was thinking about the following setup: 4 servers total: overkill, just asking for trouble. Data Servers: 1 Server holding all the websites data and mail messages. It would serve these files via nfs to the application servers. It would also run mysql A second server Also sharing it's content via nfs, replicating it's data though rsync each ?? minutes. The mysql would run as a slave of theprimary Application Servers: Both servers would be running apache, php, sendmail and posfix and would serve content from the share nfs drive. 1- Is this a viable solution, I mean by that, Is it Like this big ISP are set up ? no The really big ISP's use proprietary commercial clustering solutions that make multiple systems appear as one single system. We are talking hundreds of thousands to millions of users. We are not talking 5000 users or fewer. You can easily serve 5K users on a single server. You just need to get good hardware. In other words, costs start at $5000 and go up. A lot of people are under the misconception that they can get several cheap $900 servers and assemble them into a redundant setup that is highly reliable. The real secret is in getting expensive name-brand hardware that doesen't go down. If you can afford that, your fine. If you can't, then you need to find a different table to play at. Ted Isn't part of the point in running a redundent configuration that you can buy cheap(er) hardware? No. The point of a redundant setup is to attain 100% uptime. All hardware eventually dies it is just a question of how good the chances are. Cheaper hardware has a much higher chance of dying unexpectedly or having incompatabilities or problems. More expensive hardware has a lower chance. A $600 machine that does not have a good 6 months of burn in time on it in my experience has about a 30% chance of unexpectedly failing. If you put two of them together the chances of both dying at the same time are much lower of course - but it is still higher than the chances of a $5,000 machine dying after 24 hours of burn in time. And once the machine does die, it costs tech time to put things back together. Ultimately, the pursuit of clustering as a cost-effective way of increasing reliability is doomed. Clustering works great if what your initending to do with it is increase power of the cluster beyond what is attainable by a single machine. It also works great in life and health situations where you cannot afford anything less than 99.9% uptime. A $600 machine should be powerful enough to handle that many users. Just make sure you are using RAID 1+0 filesystems, keep replacement parts on hand and are performing regular backups. Baloney. The real question to ask is what is the provider's SLA and how much does an hour of downtime cost the provider. In my experience, the only things to die on servers have been fans, disks (really the motors), and the occasional power supply. The only things a more expensive system may give you are additional power supplies, hot-swap drive bays and multiple CPUs. Other than the system board and possibly the processors, the server's components come from the same sources as your commodity hardware. It's irrelevant. It may come as a surprise to you but a Seagate ST11950N purchased from someplace like Walmart or Costco is different than a Seagate ST11950N that is shipped from Dell in a server, this is true of most other expensive computer components. The component manufacturers make the components from cheaper materials and sloppier tolerances for the retail/desktop market than for the server market. For example a builder like Dell may spec a 20,000 MTBF sleeve bearing case fan from Panasonic for the desktop, and spec a 70,000 MTBF Panasonic Panaflo hydro wave fan for the servers. You really need to read up on hardware, there's tons of info on the Internet. It is possible to spec your own system and build a clone that is as reliable as a name-brand server, I've done it. But it won't cost $600. I think the setup described above is viable, though I would consider running the database (with master-slave replication) and application services on the same server assuming it can handle the load. Also, you can probably get away with using something like rsync to push changes to your
Re: Small Redundant web/mail setup
On 10/18/06, Martin Hepworth [EMAIL PROTECTED] wrote: Have a look at how Cambridge University (UK) have setup their email. Any URLs? I did not find any in the Engineering dept wesite of Cambridge University raj ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Small Redundant web/mail setup
On Wed, Oct 18, 2006 at 11:57:04PM -0700, Ted Mittelstaedt wrote: - Original Message - From: Ian Lord [EMAIL PROTECTED] To: freebsd-questions@freebsd.org Sent: Wednesday, October 18, 2006 5:34 AM Subject: Small Redundant web/mail setup Hi, I need to setup a high-availability setup for mail/web setup I was thinking about the following setup: 4 servers total: overkill, just asking for trouble. Data Servers: 1 Server holding all the websites data and mail messages. It would serve these files via nfs to the application servers. It would also run mysql A second server Also sharing it's content via nfs, replicating it's data though rsync each ?? minutes. The mysql would run as a slave of theprimary Application Servers: Both servers would be running apache, php, sendmail and posfix and would serve content from the share nfs drive. 1- Is this a viable solution, I mean by that, Is it Like this big ISP are set up ? no The really big ISP's use proprietary commercial clustering solutions that make multiple systems appear as one single system. We are talking hundreds of thousands to millions of users. We are not talking 5000 users or fewer. You can easily serve 5K users on a single server. You just need to get good hardware. In other words, costs start at $5000 and go up. A lot of people are under the misconception that they can get several cheap $900 servers and assemble them into a redundant setup that is highly reliable. The real secret is in getting expensive name-brand hardware that doesen't go down. If you can afford that, your fine. If you can't, then you need to find a different table to play at. Ted Isn't part of the point in running a redundent configuration that you can buy cheap(er) hardware? A $600 machine should be powerful enough to handle that many users. Just make sure you are using RAID 1+0 filesystems, keep replacement parts on hand and are performing regular backups. The real question to ask is what is the provider's SLA and how much does an hour of downtime cost the provider. In my experience, the only things to die on servers have been fans, disks (really the motors), and the occasional power supply. The only things a more expensive system may give you are additional power supplies, hot-swap drive bays and multiple CPUs. Other than the system board and possibly the processors, the server's components come from the same sources as your commodity hardware. I think the setup described above is viable, though I would consider running the database (with master-slave replication) and application services on the same server assuming it can handle the load. Also, you can probably get away with using something like rsync to push changes to your WWW servers. I'm not sure about email, but you could NFS export your mail directories from a central server to the two application servers. Just be aware of NFS' failure modes. So, I'd go with two, user-facing systems and an administrative system that receives email and possibly hosts your code repository. If you can afford it, get systems with redundent power supplies and hot-swap drive bays. Depending on your userbase, you may want to consider a robotic tape library so you don't have to manually change tapes. I've heard some talk of people using raw disks for backups, but I don't have any experience with that type of setup. -Damian ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Small Redundant web/mail setup
- Original Message - From: Zbigniew Szalbot [EMAIL PROTECTED] To: Ted Mittelstaedt [EMAIL PROTECTED] Cc: freebsd-questions@freebsd.org; Ian Lord [EMAIL PROTECTED] Sent: Thursday, October 19, 2006 12:06 AM Subject: Re: Small Redundant web/mail setup Hello, On Wed, 18 Oct 2006, Ted Mittelstaedt wrote: The really big ISP's use proprietary commercial clustering solutions that make multiple systems appear as one single system. We are talking hundreds of thousands to millions of users. We are not talking 5000 users or fewer. You can easily serve 5K users on a single server. You just need to get good hardware. In other words, costs start at $5000 and go up. Ian - not sure if it is appropriate to ask but because one day I will need to think about a server with solid hardware, what would you advise me to look at? I mean look company-wise? Or simply select from a list of a server-type machines that costs more than 5K? Well, you probbaly want to start with the name brands who actually know that FreeBSD exists! Start here: http://www.testdrive.hp.com Click on the Sign Up for an Account link right under the FreeBSD logo. Ted ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Small Redundant web/mail setup
Hello, On Thu, 19 Oct 2006, Ted Mittelstaedt wrote: Well, you probbaly want to start with the name brands who actually know that FreeBSD exists! Start here: http://www.testdrive.hp.com That's very helpful - thank you!!! -- Zbigniew Szalbot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Small Redundant web/mail setup
- Original Message - From: Ian Lord [EMAIL PROTECTED] To: freebsd-questions@freebsd.org Sent: Wednesday, October 18, 2006 5:34 AM Subject: Small Redundant web/mail setup Hi, I need to setup a high-availability setup for mail/web setup I was thinking about the following setup: 4 servers total: overkill, just asking for trouble. Data Servers: 1 Server holding all the websites data and mail messages. It would serve these files via nfs to the application servers. It would also run mysql A second server Also sharing it's content via nfs, replicating it's data though rsync each ?? minutes. The mysql would run as a slave of theprimary Application Servers: Both servers would be running apache, php, sendmail and posfix and would serve content from the share nfs drive. 1- Is this a viable solution, I mean by that, Is it Like this big ISP are set up ? no The really big ISP's use proprietary commercial clustering solutions that make multiple systems appear as one single system. We are talking hundreds of thousands to millions of users. We are not talking 5000 users or fewer. You can easily serve 5K users on a single server. You just need to get good hardware. In other words, costs start at $5000 and go up. A lot of people are under the misconception that they can get several cheap $900 servers and assemble them into a redundant setup that is highly reliable. The real secret is in getting expensive name-brand hardware that doesen't go down. If you can afford that, your fine. If you can't, then you need to find a different table to play at. Ted ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Small Redundant web/mail setup
Hello, On Wed, 18 Oct 2006, Ted Mittelstaedt wrote: The really big ISP's use proprietary commercial clustering solutions that make multiple systems appear as one single system. We are talking hundreds of thousands to millions of users. We are not talking 5000 users or fewer. You can easily serve 5K users on a single server. You just need to get good hardware. In other words, costs start at $5000 and go up. Ian - not sure if it is appropriate to ask but because one day I will need to think about a server with solid hardware, what would you advise me to look at? I mean look company-wise? Or simply select from a list of a server-type machines that costs more than 5K? Thanks! -- Zbigniew Szalbot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Small Redundant web/mail setup
Hi, I need to setup a high-availability setup for mail/web setup I was thinking about the following setup: 4 servers total: Data Servers: 1 Server holding all the websites data and mail messages. It would serve these files via nfs to the application servers. It would also run mysql A second server Also sharing it's content via nfs, replicating it's data though rsync each ?? minutes. The mysql would run as a slave of theprimary Application Servers: Both servers would be running apache, php, sendmail and posfix and would serve content from the share nfs drive. 1- Is this a viable solution, I mean by that, Is it Like this big ISP are set up ? 2- Is there a better way to replicate data than RSYNC (without going to san of expensive hardware) ? If not, is there a hotsync feature (I mean by that as soon as server A modify something, server B knows and replicate)? I would appreciate if you could give me feedbacks, suggestions, or if you see any problem that might happen with this kind of setup. Thanks a lot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Small Redundant web/mail setup
Ian Lord wrote: 2- Is there a better way to replicate data than RSYNC (without going to san of expensive hardware) ? If not, is there a hotsync feature (I mean by that as soon as server A modify something, server B knows and replicate)? I've never tried the following setup myself, but you should look into the possibility of using geom ggated/ggatec and gmirror in combination. Basically ggated/ggatec will export the raw block device over the net, so that another computer may use it in a geom stack. You could have the second computer export it's disk device, and let the first one use it in a mirror (raid) setup. Since ggated on the the second computer would claim the device, I think you could only mount it read-only, but it would be synced live, I think. Note that I don't know these tools from experience, only from what I've picked up here and there. You would have to read up the specifics yourself. Svein Halvor signature.asc Description: OpenPGP digital signature
Re: Small Redundant web/mail setup
Ian Lord wrote: Hi, I need to setup a high-availability setup for mail/web setup ... 1 Server holding all the websites data and mail messages. It would serve these files via nfs to the application servers. It would also run mysql A second server Also sharing it's content via nfs, replicating it's data though rsync each ?? minutes. The mysql would run as a slave of theprimary Application Servers: Both servers would be running apache, php, sendmail and posfix and would serve content from the share nfs drive. 1- Is this a viable solution, I mean by that, Is it Like this big ISP are set up ? I don't know any of the answers for sure, but I'd bet they are both 'no'. 2- Is there a better way to replicate data than RSYNC (without going to san of expensive hardware) ? If not, is there a hotsync feature (I mean by that as soon as server A modify something, server B knows and replicate)? I guess so. First of all, I don't really understand the need to have four server, unless there is some point which you didn't tell us. Apart from that, I guess it would be a lot better to try and sync at the application level. MySQL should support this and I bet you can find something alike on the IMAP side (cyrus has that support, but I don't know how stable that is). That leaves you with file system replication only for web sites, but that should be ok as long as it's mostly read-only data. bye av. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Small Redundant web/mail setup
Hi, 1- Is this a viable solution, I mean by that, Is it Like this big ISP are set up ? Not quite likely, but it's possible ofcourse. 2- Is there a better way to replicate data than RSYNC (without going to san of expensive hardware) ? If not, is there a hotsync feature (I mean by that as soon as server A modify something, server B knows and replicate)? IMO there's no better solution than rsync, besides somewhat more hardware / not that I'm aware of no. You could also build one fileserver able to serve nfs, store mail in maildir format and put 2 boxes online running mail/http daemons. Return one server get loadbalancer instead and put that online. That would be more my idea of having high-availability. Ofcourse you could also put a layer 3 switch in front of it instead of a loadballancer. Make sure the hardware of the fileserver is at least able to do raid 1 and has a dual powersupply. That will help the keep the thing up. Hth, Robert ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Small Redundant web/mail setup
Have a look at how Cambridge University (UK) have setup their email. Does alot of this sort of stuff and they've got lots of docs online as to how they did it.. -- Martin On 10/18/06, Ian Lord [EMAIL PROTECTED] wrote: Hi, I need to setup a high-availability setup for mail/web setup I was thinking about the following setup: 4 servers total: Data Servers: 1 Server holding all the websites data and mail messages. It would serve these files via nfs to the application servers. It would also run mysql A second server Also sharing it's content via nfs, replicating it's data though rsync each ?? minutes. The mysql would run as a slave of theprimary Application Servers: Both servers would be running apache, php, sendmail and posfix and would serve content from the share nfs drive. 1- Is this a viable solution, I mean by that, Is it Like this big ISP are set up ? 2- Is there a better way to replicate data than RSYNC (without going to san of expensive hardware) ? If not, is there a hotsync feature (I mean by that as soon as server A modify something, server B knows and replicate)? I would appreciate if you could give me feedbacks, suggestions, or if you see any problem that might happen with this kind of setup. Thanks a lot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
