Hi, I'm in the process of building a 10 Gbps Ethernet SAN and would really like to use AoE, but I'm experiencing poor performance (320 MB/s sequential reads and writes). The access pattern on this SAN will be almost exclusively sequential so I am interested in maximizing throughput, not IOPS.
The status quo: * RAID 60 (1 MB stripe size) on target capable of 900 MB/s write and 1400 MB/s read on ext4 (sequential). * Jumbo frames (9000) configured and working on target, initiator and Dell PowerConnect 8132 switch (with hardware flow control enabled). * iperf benchmarks TCP performance at 9.87 Gb/s. * aoetools-36, vblade-21 and kernel 3.10.7 with AoE v81 I'm using the following tuning parameters as suggested on this list: net.core.rmem_default = 278528 net.core.rmem_max = 278528 net.core.wmem_default = 278528 net.core.wmem_max = 278528 net.core.netdev_max_backlog = 10000 I exported the RAID 60 as e0.0. Both read and write performance is disappointing, though: # dd if=/dev/etherd/e0.0 of=/dev/null bs=1M count=4K 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 15.9536 s, 269 MB/s # dd if=/dev/zero of=/dev/etherd/e0.0 bs=1M count=4K conv=fdatasync 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 13.6141 s, 315 MB/s Read speed is even worse than write, which is odd. To rule out any doubts about RAID performance I exported a 4 GB ramdisk as e0.1, but the results are just as bad: # dd if=/dev/etherd/e0.1 of=/dev/null bs=1M count=4K 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 15.1406 s, 284 MB/s # dd if=/dev/zero of=/dev/etherd/e0.1 bs=1M count=4K conv=fdatasync 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 13.3615 s, 321 MB/s dd, vblade and the aoe kernel threads are well below 100% CPU usage during these tests. What am I missing here? Thanks in advance, Derick ------------------------------------------------------------------------------ Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk _______________________________________________ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
