Hi David,
I designed and implemented a setup like this for our company. We run two datastorage servers (primary and backup) which replicate to each other using drbd. We then run 4 servers in front of those which are diskless and boot via PXE and mount there root filesystems via NFSv3 to the backend datastorage servers. On top of these front end servers we run vservers which then allow us to seperate all of our services and move them between front end servers to deal with hardware failures and load.
You have to be careful in what applications you run on the front end as these need to be nfs nice. But most things work. But we did have problems with mail queues in postfix, which initially caused lots of corruption, but this was fixed by running the mail queue inside a ram disk (initially), and then switched to using a loopback device.
I did have issues with NFS stale handlers when I tried to mount each vserver as a NFS mount. But we just worked around this by just placing everything inside the rootfs of the front server.
We did have a number of issues with performance of IO. Since we actually have a firewall between the NFS servers and the front end servers, we had performance problems with all of the udp traffic creating states on the firewall. We have changed to using NFS over TCP. We also use NFSv3 rather than NFSv2 as it is a lot faster when running under the sync option (which you have to run).
Here are the options we use for our NFS mounts:
nfsvers=3,hard,nointr,tcp,timeo=600,retrans=2,rsize=32768,wsize=32768
Another thing to note is we don't run any of our major databases across NFS. We run them inside vservers on the datastorage servers.
We run everything across gigabit ethernet.
For statistics, we are currently running 17 vservers on 3 servers (1 spare) all mounted across NFS. Our NFS server is running 5 vservers contain different databases. Our bandwidth to our nfs server sits on pretty much 2.5Mbit most of the time (peaks to 25Mbit at times). Most of our vservers are not under heavy use, but we have peaked our out going internet traffic from this (http) hosting to about 20Mbit without any major performance issues.
If your hosting is high io intensive. Then you will probably have issues, if its just static html files you should be ok as the file should only be loaded over nfs once and cached locally. For our environment all of the high io stuff is inside the database which doesn't run over nfs.
Good luck.
Tristan.
David MacKinnon wrote:
Hey. I know I vaguely recall someone talking about running vservers over nfs (ie the vserver directory is hosted on an nfs server, run on a front-end). I was just wondering if anyone who has set up something like this would care to comment on general performance, and any tuning they did?
We're looking at seting up something like this for our hosting facility, to provide greater flexibility, but I didn't realise quite how slow nfs
could be :( We are/were hoping to run over a dozen vservers over this setup.
Any comments/suggestions?
-David
_______________________________________________ Vserver mailing list [EMAIL PROTECTED] http://list.linux-vserver.org/mailman/listinfo/vserver
_______________________________________________ Vserver mailing list [EMAIL PROTECTED] http://list.linux-vserver.org/mailman/listinfo/vserver
