Hi Anand,
I use 2 gluster server , this is my volume info:
Volume Name: soul
Type: Distribute
Volume ID: 58f049d0-a38a-4ebe-94c0-086d492bdfa6
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: 192.168.101.133:/dcsdata/d0
Brick2: 192.168.101.134:/dcsdata/d0
each brick use a raid 5 logic disk with 8*2TSATA hdd.
smb.conf:
[gvol]
comment = For samba export of volume test
vfs objects = glusterfs
glusterfs:volfile_server = localhost
glusterfs:volume = soul
path = /
read only = no
guest ok = yes
this my testparm result:
[global]
workgroup = MYGROUP
server string = DCS Samba Server
log file = /var/log/samba/log.vfs
max log size = 500000
max xmit = 262144
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=262144
SO_SNDBUF=262144
stat cache = No
kernel oplocks = No
idmap config * : backend = tdb
aio read size = 262144
aio write size = 262144
aio write behind = true
cups options = raw
in client mount the smb share with cifs to dir /mnt/vfs,
then use iozone executed in the cifs mount dir "/mnt/vfs":
$ ./iozone -s 10G -r 128k -i0 -i1 -t 4
File size set to 10485760 KB
Record Size 128 KB
Command line used: ./iozone -s 10G -r 128k -i0 -i1 -t 4
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 4 processes
Each process writes a 10485760 Kbyte file in 128 Kbyte records
Children see throughput for 4 initial writers = 534315.84 KB/sec
Parent sees throughput for 4 initial writers = 519428.83 KB/sec
Min throughput per process = 133154.69 KB/sec
Max throughput per process = 134341.05 KB/sec
Avg throughput per process = 133578.96 KB/sec
Min xfer = 10391296.00 KB
Children see throughput for 4 rewriters = 536634.88 KB/sec
Parent sees throughput for 4 rewriters = 522618.54 KB/sec
Min throughput per process = 133408.80 KB/sec
Max throughput per process = 134721.36 KB/sec
Avg throughput per process = 134158.72 KB/sec
Min xfer = 10384384.00 KB
Children see throughput for 4 readers = 77403.54 KB/sec
Parent sees throughput for 4 readers = 77402.86 KB/sec
Min throughput per process = 19349.42 KB/sec
Max throughput per process = 19353.42 KB/sec
Avg throughput per process = 19350.88 KB/sec
Min xfer = 10483712.00 KB
Children see throughput for 4 re-readers = 77424.40 KB/sec
Parent sees throughput for 4 re-readers = 77423.89 KB/sec
Min throughput per process = 19354.75 KB/sec
Max throughput per process = 19358.50 KB/sec
Avg throughput per process = 19356.10 KB/sec
Min xfer = 10483840.00 KB
then the use the same command test in the dir mounted with glister fuse:
File size set to 10485760 KB
Record Size 128 KB
Command line used: ./iozone -s 10G -r 128k -i0 -i1 -t 4
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 4 processes
Each process writes a 10485760 Kbyte file in 128 Kbyte records
Children see throughput for 4 initial writers = 887534.72 KB/sec
Parent sees throughput for 4 initial writers = 848830.39 KB/sec
Min throughput per process = 220140.91 KB/sec
Max throughput per process = 223690.45 KB/sec
Avg throughput per process = 221883.68 KB/sec
Min xfer = 10319360.00 KB
Children see throughput for 4 rewriters = 892774.92 KB/sec
Parent sees throughput for 4 rewriters = 871186.83 KB/sec
Min throughput per process = 222326.44 KB/sec
Max throughput per process = 223970.17 KB/sec
Avg throughput per process = 223193.73 KB/sec
Min xfer = 10431360.00 KB
Children see throughput for 4 readers = 605889.12 KB/sec
Parent sees throughput for 4 readers = 601767.96 KB/sec
Min throughput per process = 143133.14 KB/sec
Max throughput per process = 159550.88 KB/sec
Avg throughput per process = 151472.28 KB/sec
Min xfer = 9406848.00 KB
it shows much higher perf.
any places i did wrong?
thank you
-Kane
在 2013-9-18，下午1:19，Anand Avati <[email protected]> 写道：
> How are you testing this? What tool are you using?
>
> Avati
>
>
> On Tue, Sep 17, 2013 at 9:02 PM, kane <[email protected]> wrote:
> Hi Vijay
>
> I used the code in https://github.com/gluster/glusterfs.git with the
> lasted commit:
> commit de2a8d303311bd600cb93a775bc79a0edea1ee1a
> Author: Anand Avati <[email protected]>
> Date: Tue Sep 17 16:45:03 2013 -0700
>
> Revert "cluster/distribute: Rebalance should also verify free inodes"
>
> This reverts commit 215fea41a96479312a5ab8783c13b30ab9fe00fa
>
> Realized soon after merging, ….
>
> which include the patch you mentioned last time improve read perf, written by
> Anand.
>
> but the read perf was still slow:
> write: 500MB/s
> read: 77MB/s
>
> while via fuse :
> write 800MB/s
> read 600MB/s
>
> any advises?
>
>
> Thank you.
> -Kane
>
> 在 2013-9-13，下午10:37，kane <[email protected]> 写道：
>
> > Hi Vijay，
> >
> > thank you for post this message, i will try it soon
> >
> > -kane
> >
> >
> >
> > 在 2013-9-13，下午9:21，Vijay Bellur <[email protected]> 写道：
> >
> >> On 09/13/2013 06:10 PM, kane wrote:
> >>> Hi
> >>>
> >>> We use gluster samba vfs test io,but the read performance via vfs is
> >>> half of write perfomance,
> >>> but via fuse the read and write performance is almost the same.
> >>>
> >>> this is our smb.conf:
> >>> [global]
> >>> workgroup = MYGROUP
> >>> server string = DCS Samba Server
> >>> log file = /var/log/samba/log.vfs
> >>> max log size = 500000
> >>> # use sendfile = true
> >>> aio read size = 262144
> >>> aio write size = 262144
> >>> aio write behind = true
> >>> min receivefile size = 262144
> >>> write cache size = 268435456
> >>> security = user
> >>> passdb backend = tdbsam
> >>> load printers = yes
> >>> cups options = raw
> >>> read raw = yes
> >>> write raw = yes
> >>> max xmit = 262144
> >>> socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=262144
> >>> SO_SNDBUF=262144
> >>> kernel oplocks = no
> >>> stat cache = no
> >>>
> >>> any advises helpful?
> >>>
> >>
> >> This patch has shown improvement in read performance with libgfapi:
> >>
> >> http://review.gluster.org/#/c/5897/
> >>
> >> Would it be possible for you to try this patch and check if it improves
> >> performance in your case?
> >>
> >> -Vijay
> >>
> >
>
>
> _______________________________________________
> Gluster-users mailing list
> [email protected]
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
