We built our own setup for similar VM storage workloads, based on a dozen Thecus rackmount SATA arrays using a custom load of Fedora 12. 1.5Tb Samsung or Seagate SATA disks.
8 disk software raid 10, dual bonded Gig dedicated to DRBD and heartbeat, serving NFS and iSCSI over Gig. The hardware/disk/drbd/network combo is just fine - not operating anywhere near it's limits. In short, don't use linux NFS for VM storage. It does a really poor job of efficiently transmitting load to the disks. I've spent more than a few weeks trying to tweak NFS clients and server to move the bottleneck on, but throughput and virtual disk latency over NFS is about 10x worse than what it is over iSCSI. Perhaps this is also to do with the extra FS layers a NFS hosted VM disk has to contend with. We built this as an alternative to NetApp FAS-series arrays, it's way slower at NFS, but broadly comparable to our Equallogic iSCSI arrays. Unfortunately, using NFS to manage VMs VMware style is just far more convenient than using 1 iSCSI LUN per VM. regards James M On 14/02/11 19:59, Dennis Jacobfeuerborn wrote: > I'm trying to wrap my head around storage setups that might work for > virtualization and I wonder if > people here have experience with creating a drbd setup for this purpose. > > What I am currently planning to implement is this: > 2 Storage 8-bay nodes with 8gb RAM and a dual-core Xeon processor. > Each system with gets equipped with 8 1TB SATA drives in a raid-5 > configuration. > Networking will either be two dual-port cards or two quad-core cards which I > plan to setup as bonded > interfaces (balance-xor). > > I'm particularly worried about the networking side being a bottleneck for the > setup. I was looking > into 10gbit and infiniband equipment but they drive the cost up quite a bit > and I'm not sure if they > are necessary if I can bond several 1gbit interfaces. > > Any thoughts? > > Regards, > Dennis > _______________________________________________ > drbd-user mailing list > [email protected] > http://lists.linbit.com/mailman/listinfo/drbd-user The information in this e-mail and any attachment is confidential and is intended only for the named recipient(s). The e-mail may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not a named recipient please notify the sender immediately and delete any copies of this message. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any view or opinions presented are solely those of the author and do not necessarily represent those of the company. _______________________________________________ drbd-user mailing list [email protected] http://lists.linbit.com/mailman/listinfo/drbd-user
