On Tue, Jul 05, 2011 at 05:03:40PM +0200, Torbjørn Thorsen spake thusly: > I'm setting up a AoE-based SAN, and I'm not quite sure I've reached a > good performance level. > > I can read and write the raw AoE device (/dev/etherd/*) at more or > less line-speed > on my 1gig Ethernet adapters.
> This means I'm seeing I/O rates of 100 to 120 MB/s when using dd or > something similar. This is in line with what I get also. Sounds like your performance level is as expected (very good). > However, when I put a filesystem on there, I'm seeing rates of 55 to 70 MB/s. > I've tested mostly by using rsync, cp or dd, but I tried bonnie and > saw much the same results. Yep. You are most likely running into physical limitations of the disk. > Since I'm seeing line-speed when using the device directly, I guess this means > that the configuration is more or less okay. Yep. > What kind of performance are you guys seeing on your filesystems when > using 1gig Ethernet adapters ? The speed of the network is not nearly as important as the speed of the disk hardware. I get performance similar to yours when doing streaming reads/writes to at least two disks. A single 7200rpm drive can typically do 70MB/s so you usually need to gang up at least two of these in a mirror or stripe. Many more smaller disks are necessary for higher IOPS. Fortunately, this is a problem completely independent of AoE so lots of people know how to solve it. These days I deploy SuperMicro 24 bay 2.5" servers stuffed full of 10k RPM disks. This seems to get me the most reasonable bang/buck while providing the kind of IOPS I need to run databases, mail servers, etc. The giant/cheap 2T disks you can buy these days are great for archival and backup storage but for actual data processing the advice has been the same for many years: Throw lots of spindles at the problem. -- Tracy Reed Digital signature attached for your safety. Copilotco Professionally Managed PCI Compliant Secure Hosting 866-MY-COPILOT x101 http://copilotco.com
pgpy50WeWKhxL.pgp
Description: PGP signature
------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
