Great, thanks a bunch for some very effective advice. It's very encouraging to see such good advice provided at this speed, I'm very grateful for your insights on this issue.
I have provided some more answers inline, mostly for the benefit of people reading this archived. As I applied all your advice at the same time, I'm not quite sure what made the biggest impact The result was that my AoE benchmarks now outperform the iSCSI alternative. 2011/5/9 Tracy Reed <tr...@ultraviolet.org>: > On Sun, May 08, 2011 at 04:59:36PM +0200, Torbjørn Thorsen spake thusly: >> I really want to stay away from the iSCSI setup, >> but I'm at the end of my wits with regards to getting the required >> performance from AoE. >> >> Any ideas as to what I'm doing wrong ? > > There are a few things you can do to help: > > 1. Make sure you are using a good switch. Cisco, HP, I hear Juniper is good > too. Anything else is likely to be crap. Netgear, Linksys, Dell are all right > out. I'm using HP ProCurve 2510s. > > 1. Make sure you use hardware flow control on the switch. Flow control in the switch was not enabled, and I can't remember seeing that one mentioned too many places either. > > 2. Double check that you really have mtu 9000 on both ends. Wouldn't you know, I had not included the respective ports in the jumbo-enabled VLAN. After enabling the ports on the switch, I verified I had a 9000 MTU by using ping with the "prohibit fragmentation" option enabled and sending frames bigger than 1500 bytes. > > 3. I always run vblade with the -b 16384 option as without it performance was > significantly less. > > 4. Adding these to /etc/sysctl.conf helps a bit too: > > # Optimizations to increase speed for AoE > net.core.rmem_default = 278528 > net.core.rmem_max = 278528 > net.core.wmem_default = 278528 > net.core.wmem_max = 278528 > net.core.netdev_max_backlog=10000 > > # Don't let dirty write cache hang around so long, commit to disk sooner for > # safety. AoE targets can cache a lot of stuff. > vm.dirty_ratio = 3 > vm.dirty_background_ratio = 3 > vm.min_free_kbytes = 65536 > > The first stanza is for performance, the second for safety. Run /sbin/sysctl > -p > to make the system re-read sysctl.conf so the changes can take effect. I enabled all these options, they looked to be right up my storage-enabled alley. As for data safety, should I consider running vblade with the O_SYNC option enabled ? I will be exporting block devices to be used as the filesystem for Xen instances, and I wouldn't want to lie to MySQL living in Xen about data being on disk. > > And make sure you don't have any write alignment issues on the target disk. I > used to try to partition my disk but that really complicates things > alignment-wise so now I just initialize the raw disk (/dev/sdc for example) as > an lvm physical volume and export an lvm from that. Using the raw device here as well, I don't see the need for partitioning at this level. > > -- > Tracy Reed > -- Vennlig hilsen Torbjørn Thorsen Utvikler / driftstekniker Trollweb Solutions AS - Professional Magento Partner www.trollweb.no Telefon dagtid: +47 51215300 Telefon kveld/helg: For kunder med Serviceavtale Besøksadresse: Luramyrveien 40, 4313 Sandnes Postadresse: Maurholen 57, 4316 Sandnes Husk at alle våre standard-vilkår alltid er gjeldende ------------------------------------------------------------------------------ WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd _______________________________________________ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
