Dear gluster experts, We have encountered a severe performance issue related to quota feature of gluster.
My underlying fs is lvm with xfs format. The problem is if quota is enabled the io performance is about 26MB/s but with quota disabled the io performance is 216MB/s. Any one known what's the problem? BTW I have reproduce it several times and it is related to quota indeed. Here's the test: __________________quota enabled________________________________________ -sh-4.1$ sudo gluster volume info test_replica2_dis4 Volume Name: test_replica2_dis4 Type: Distributed-Replicate Volume ID: 9004b9d8-88f9-4312-9c22-1f1a62fd9784 Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: bjzw.miaoyan.cluster1.node1.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick2: bjzw.miaoyan.cluster1.node2.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick3: bjzw.miaoyan.cluster1.node3.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick4: bjzw.miaoyan.cluster1.node4.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick5: bjzw.miaoyan.cluster1.node5.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick6: bjzw.miaoyan.cluster1.node6.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick7: bjzw.miaoyan.cluster1.node7.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick8: bjzw.miaoyan.cluster1.node8.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Options Reconfigured: features.limit-usage: /:5TB features.quota: enable -sh-4.1$ sudo dd if=/dev/zero of=iotest/testfff2 bs=1M count=1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB) copied, 40.3801 s, 26.0 MB/s ______________________________________________________________________ __________________quota disabled________________________________________ -sh-4.1$ sudo gluster volume info test_replica2_dis4 Volume Name: test_replica2_dis4 Type: Distributed-Replicate Volume ID: 9004b9d8-88f9-4312-9c22-1f1a62fd9784 Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: bjzw.miaoyan.cluster1.node1.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick2: bjzw.miaoyan.cluster1.node2.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick3: bjzw.miaoyan.cluster1.node3.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick4: bjzw.miaoyan.cluster1.node4.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick5: bjzw.miaoyan.cluster1.node5.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick6: bjzw.miaoyan.cluster1.node6.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick7: bjzw.miaoyan.cluster1.node7.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Brick8: bjzw.miaoyan.cluster1.node8.qiyi.domain:/mnt/xfsd/test_replica2_dis4 Options Reconfigured: features.limit-usage: /:5TB features.quota: disable -sh-4.1$ sudo dd if=/dev/zero of=iotest/testfff2 bs=1M count=1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB) copied, 4.86384 s, 216 MB/s ______________________________________________________________________ Thank you! -- 符永涛
_______________________________________________ Gluster-users mailing list [email protected] http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
