On 01/15/13 07:06, glusterzhxue wrote:
About the following issue, anyone could help us? Why using VM mounted by
NFS V3(which is also provided by gluster) could approach full
bandwidth(100MB/s),while VM mounted by gluster only reach half of
bandwidth (50MB/s).
The following is the mount command:
1)gluster native client:
mount -t glusterfs server1:/test-volume /var/lib/nova/instance/
2) NFS V3:
mount -t nfs -o vers=3 server1:/test-volume /var/lib/nova/instance/
/var/lib/nova/instance/ is the file directory storing VMs
Thanks.
------------------------------------------------------------------------
zhxue
What is your volume layout?
The output of "gluster volume info test-volume"
What type of network hardware do you have?
What type of ethernet ports on the host running the VMs?
What type of ethernet ports on the hosts containing the storage (the
gluster servers)?
How are they configured?
My configuration has 4 1Gb ports LACP bonded on the VM host and single
10Gb ports on the the storage nodes (gluster servers.)
Our volume is: (4 bricks per gluster server)
Volume Name: vol01
Type: Distributed-Replicate
Volume ID: 5ebcaed8-dd9b-4c48-be75-46457fe2e978
Status: Started
Number of Bricks: 24 x 2 = 48
Transport-type: tcp
With the iozone testing on a FUSE mount, I am currently doing, I get:
~100MB/s writes if the replicated bricks chosen hash out in the LACP
bonding to the different 1Gb ethernet ports.
~50MB/s writes if the replicated bricks chosen hash out in the LACP
bonding to the same 1Gb ethernet ports.
~20MB/s writes for the test using the O_SYNC option for iozone, no
matter what 1Gb ethernet ports end up being used.
Reads are always ~100MB/s as it comes from one brick.
If you have a replica volume, then mount using FUSE, the writes are
duplicated on the client. If you mount via NFS, the writes are
duplicated on the server.
Also of note, with the iozone going full speed, the FUSE client takes
50% of a CPU core.
--
Mr. Flibble
King of the Potato People
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
