On Tue, 21 Jul 2020, 21:30 Qing Wang <[email protected]> wrote: > Hi, > > I am trying to test Gluster linear scale-out performance by adding more > storage server/bricks, and measure the storage I/O performance. To vary the > storage server number, I create several "stripe" volumes that contain 2 > brick servers, 3 brick servers, 4 brick servers, and so on. On gluster > client side, I used "dd if=/dev/zero of=/mnt/glusterfs/dns_test_data_26g > bs=1M count=26000" to create 26G data (or larger size), and those data will > be distributed to the corresponding gluster servers (each has gluster brick > on it) and "dd" returns the final I/O throughput. The Internet is 40G > infiniband, although I didn't do any specific configurations to use > advanced features. >
Your dd command is inaccurate, as it'll hit the client cache. It is also single threaded. I suggest switching to fio. Y. > What confuses me is that the storage I/O seems not to relate to the number > of storage nodes, but Gluster documents said it should be linear scaling. > For example, when "write-behind" is on, and when Infiniband "jumbo frame" > (connected mode) is on, I can get ~800 MB/sec reported by "dd", no matter I > have 2 brick servers or 8 brick servers -- for 2 server case, each server > can have ~400 MB/sec; for 4 server case, each server can have ~200MB/sec. > That said, each server I/O does aggregate to the final storage I/O (800 > MB/sec), but this is not "linear scale-out". > > Can somebody help me to understand why this is the case? I certainly can > have some misunderstanding/misconfiguration here. Please correct me if I > do, thanks! > > Best, > Qing > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > [email protected] > https://lists.gluster.org/mailman/listinfo/gluster-users >
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-users mailing list [email protected] https://lists.gluster.org/mailman/listinfo/gluster-users
