Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-23 Thread Tobias Weingartner
In article [EMAIL PROTECTED], Nick Holland wrote: Dumping the data from one disk to another is fine and dandy when you are talking about your 40G disk on your home or desktop computer, the fact that you are down for a few hours is no big deal. But what about a server? I don't care how

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-21 Thread Craig Skinner
On Fri, Apr 20, 2007 at 11:04:34PM -0700, Clint Pachl wrote: What do you consider a sane number of front ends, 10, less, more? Well, I think that depends on too many variables. I have a movie server (OBSD) that exports NFS to two home theatre computers (FBSD). The movie server is a dual

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Henning Brauer
* Joachim Schipper [EMAIL PROTECTED] [2007-04-20 00:36]: On Thu, Apr 19, 2007 at 10:51:56PM +0100, Stuart Henderson wrote: I don't think NFS/AFS is that good an idea; you'll need very beefy fileservers and a fast network. NFS may actually be useful; if you really need the files in one

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Joachim Schipper
On Fri, Apr 20, 2007 at 12:36:29PM +0200, Henning Brauer wrote: * Joachim Schipper [EMAIL PROTECTED] [2007-04-20 00:36]: On Thu, Apr 19, 2007 at 10:51:56PM +0100, Stuart Henderson wrote: I don't think NFS/AFS is that good an idea; you'll need very beefy fileservers and a fast network.

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Will Maier
On Fri, Apr 20, 2007 at 09:03:54AM -0500, Jacob Yocom-Piatt wrote: from my observations redundancy is the biggest problem with NFS and that its ability to efficiently serve up data is more than ample. Redundancy is certainly a problem, but lots of US HPC and distributed computing sites have

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Jacob Yocom-Piatt
Joachim Schipper wrote: there is nothing wrong with serving directly from NFS. Really? You have a lot more experience in this area, so I will defer to you if you are sure, but it seems to me that in the sort of system I explicitly assumed (something like a web farm), serving everything

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Tony Abernethy
Jason Beaudoin wrote: snip Use all the tricks you can for YOUR solution, including: * lots of small partitions What are the reasonings behind this? Thanks for the awesome post! I think it runs something like this If there is a problem somewhere on the disk, if it's all one big

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Bob Beck
Bullshit. just use NFS :) -Bob * Steven Harms [EMAIL PROTECTED] [2007-04-19 17:01]: This isn't an OpenBSD specific solution, but you should be able to use an EMC san to accomplish this (we use a fiber channel setup) On 4/19/07, Stuart Henderson [EMAIL PROTECTED] wrote:

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Henning Brauer
* Joachim Schipper [EMAIL PROTECTED] [2007-04-20 14:49]: On Fri, Apr 20, 2007 at 12:36:29PM +0200, Henning Brauer wrote: * Joachim Schipper [EMAIL PROTECTED] [2007-04-20 00:36]: On Thu, Apr 19, 2007 at 10:51:56PM +0100, Stuart Henderson wrote: I don't think NFS/AFS is that good an idea;

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Joachim Schipper
On Fri, Apr 20, 2007 at 07:56:16PM +0200, Henning Brauer wrote: * Joachim Schipper [EMAIL PROTECTED] [2007-04-20 14:49]: On Fri, Apr 20, 2007 at 12:36:29PM +0200, Henning Brauer wrote: * Joachim Schipper [EMAIL PROTECTED] [2007-04-20 00:36]: On Thu, Apr 19, 2007 at 10:51:56PM +0100,

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Daniel Ouellet
Henning Brauer wrote: * Joachim Schipper [EMAIL PROTECTED] [2007-04-20 14:49]: On Fri, Apr 20, 2007 at 12:36:29PM +0200, Henning Brauer wrote: * Joachim Schipper [EMAIL PROTECTED] [2007-04-20 00:36]: On Thu, Apr 19, 2007 at 10:51:56PM +0100, Stuart Henderson wrote: I don't think NFS/AFS is

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread J.C. Roberts
On Friday 20 April 2007 08:32, Tony Abernethy wrote: Jason Beaudoin wrote: snip Use all the tricks you can for YOUR solution, including: * lots of small partitions What are the reasonings behind this? Thanks for the awesome post! I think it runs something like this If there

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-20 Thread Nick Holland
J.C. Roberts wrote: On Friday 20 April 2007 08:32, Tony Abernethy wrote: Jason Beaudoin wrote: snip Use all the tricks you can for YOUR solution, including: * lots of small partitions What are the reasonings behind this? Thanks for the awesome post! I think it runs

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-19 Thread Joachim Schipper
On Wed, Apr 18, 2007 at 03:22:07PM +0530, Siju George wrote: Hi, How Do you handle when you have to Serve terrabytes of Data through http/https/ftp etc? Put it on Differrent machines and use some knid of loadbalancer/intelligent program that directs to the right mahine? use some kind of

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-19 Thread Stuart Henderson
I don't think NFS/AFS is that good an idea; you'll need very beefy fileservers and a fast network. NFS may actually be useful; if you really need the files in one directory space for management/updates that's a way to do it (i.e. mount all the various storage servers by NFS on a management

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-19 Thread Daniel Ouellet
Stuart Henderson wrote: I don't think NFS/AFS is that good an idea; you'll need very beefy fileservers and a fast network. NFS may actually be useful; if you really need the files in one directory space for management/updates that's a way to do it (i.e. mount all the various storage servers by

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-19 Thread Joachim Schipper
On Thu, Apr 19, 2007 at 10:51:56PM +0100, Stuart Henderson wrote: I don't think NFS/AFS is that good an idea; you'll need very beefy fileservers and a fast network. NFS may actually be useful; if you really need the files in one directory space for management/updates that's a way to do it

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-19 Thread Steven Harms
This isn't an OpenBSD specific solution, but you should be able to use an EMC san to accomplish this (we use a fiber channel setup) On 4/19/07, Stuart Henderson [EMAIL PROTECTED] wrote: On 2007/04/19 18:08, Daniel Ouellet wrote: Stuart Henderson wrote: I don't think NFS/AFS is that good an

Re: Webservers with Terrabytes of Data in - recomended setups

2007-04-19 Thread Nick Holland
Siju George wrote: Hi, How Do you handle when you have to Serve terrabytes of Data through http/https/ftp etc? Put it on Differrent machines and use some knid of loadbalancer/intelligent program that directs to the right mahine? use some kind of clustering Software? Waht hardware do

Webservers with Terrabytes of Data in - recomended setups

2007-04-18 Thread Siju George
Hi, How Do you handle when you have to Serve terrabytes of Data through http/https/ftp etc? Put it on Differrent machines and use some knid of loadbalancer/intelligent program that directs to the right mahine? use some kind of clustering Software? Waht hardware do you use to make your System