On Tue, 21 Jul 2020, 21:43 Qing Wang <[email protected]> wrote: > Hi Yaniv, > > Thanks for the quick response. I forget to mention I am testing the > writing performance, not reading. In this case, would the client cache hit > rate still be a big issue? >
It's not hitting the storage directly. Since it's also single threaded, it may also not saturate it. I highly recommend testing properly. Y. > I'll use fio to run my test once again, thanks for the suggestion. > > Thanks, > Qing > > On Tue, Jul 21, 2020 at 2:38 PM Yaniv Kaul <[email protected]> wrote: > >> >> >> On Tue, 21 Jul 2020, 21:30 Qing Wang <[email protected]> wrote: >> >>> Hi, >>> >>> I am trying to test Gluster linear scale-out performance by adding more >>> storage server/bricks, and measure the storage I/O performance. To vary the >>> storage server number, I create several "stripe" volumes that contain 2 >>> brick servers, 3 brick servers, 4 brick servers, and so on. On gluster >>> client side, I used "dd if=/dev/zero of=/mnt/glusterfs/dns_test_data_26g >>> bs=1M count=26000" to create 26G data (or larger size), and those data will >>> be distributed to the corresponding gluster servers (each has gluster brick >>> on it) and "dd" returns the final I/O throughput. The Internet is 40G >>> infiniband, although I didn't do any specific configurations to use >>> advanced features. >>> >> >> Your dd command is inaccurate, as it'll hit the client cache. It is also >> single threaded. I suggest switching to fio. >> Y. >> >> >>> What confuses me is that the storage I/O seems not to relate to the >>> number of storage nodes, but Gluster documents said it should be linear >>> scaling. For example, when "write-behind" is on, and when Infiniband "jumbo >>> frame" (connected mode) is on, I can get ~800 MB/sec reported by "dd", no >>> matter I have 2 brick servers or 8 brick servers -- for 2 server case, each >>> server can have ~400 MB/sec; for 4 server case, each server can have >>> ~200MB/sec. That said, each server I/O does aggregate to the final storage >>> I/O (800 MB/sec), but this is not "linear scale-out". >>> >>> Can somebody help me to understand why this is the case? I certainly can >>> have some misunderstanding/misconfiguration here. Please correct me if I >>> do, thanks! >>> >>> Best, >>> Qing >>> ________ >>> >>> >>> >>> Community Meeting Calendar: >>> >>> Schedule - >>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC >>> Bridge: https://bluejeans.com/441850968 >>> >>> Gluster-users mailing list >>> [email protected] >>> https://lists.gluster.org/mailman/listinfo/gluster-users >>> >>
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-users mailing list [email protected] https://lists.gluster.org/mailman/listinfo/gluster-users
