On Mon, 23 Mar 2009, Gennadiy Nerubayev wrote:
Hi all,
I'm using 2.0.0rc4 on a pair of infiniband connected servers. Both servers
are running 2.6.28.7, and are using ext4 on a local raid set as the storage
for Gluster. The speeds that I get writing to the aforementioned ext4 mount,
depending on benchmark and method, vary around 400MB/s (with dd) to 550MB/s+
(with iozone, bonnie, and dd directio). Next, I used the Gluster server and
client on both servers, with ib-verbs and replicate (see attached
configuration files). The best speeds that I get when writing to the Gluster
mount with this setup is about 150MB/s, going up to about 200MB/s when doing
simultaneous writes from both nodes. I tried a couple of configuration
tweaks such as writeback, but they don't seem to be having much effect.
That is about what I am getting:
Raw disk:
8589934592 bytes (8.6 GB) copied, 21.0523 seconds, 408 MB/s
Gluster:
8589934592 bytes (8.6 GB) copied, 47.4356 seconds, 181 MB/s
Gluster --disable-direct-io-mode
8589934592 bytes (8.6 GB) copied, 336.514 seconds, 25.5 MB/
The part that is most concerning to me is the --disable-direct-io-mode
stuff since I am trying to use this with xen tap:aio.
Is there something I can do to bring the performance closer to the native
disk speed, or are there some inherent limitations somewhere that I should
be aware of?
You can try some caching, but overall it did not help me out a lot.
<>
Nathan Stratton CTO, BlinkMind, Inc.
nathan at robotics.net nathan at blinkmind.com
http://www.robotics.net http://www.blinkmind.com
_______________________________________________
Gluster-users mailing list
[email protected]
http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
