On March 24, 2020 12:08:08 AM GMT+02:00, Jayme <[email protected]> wrote: >I too struggle with speed issues in hci. Latency is a big problem with >writes for me especially when dealing with small file workloads. How >are >you testing exactly? > >Look into enabling libgfapi and try some comparisons with that. People >have >been saying it’s much faster, but it’s not a default option and has a >few >bugs. Redhat devs do not appear to be giving its implementation any >priority unfortunately. > >I’ve been considering switching to nfs storage because I’m seeing much >better performance in testing with it. I have some nvme drives on the >way >and am curious how they would perform in hci but I’m thinking the issue >is >not a disk bottleneck (that appears very obvious in your case as well) > > > >On Mon, Mar 23, 2020 at 6:44 PM Christian Reiss ><[email protected]> >wrote: > >> Hey folks, >> >> gluster related question. Having SSD in a RAID that can do 2 GB >writes >> and Reads (actually above, but meh) in a 3-way HCI cluster connected >> with 10gbit connection things are pretty slow inside gluster. >> >> I have these settings: >> >> Options Reconfigured: >> cluster.server-quorum-type: server >> cluster.quorum-type: auto >> cluster.shd-max-threads: 8 >> features.shard: on >> features.shard-block-size: 64MB >> server.event-threads: 8 >> user.cifs: off >> cluster.shd-wait-qlength: 10000 >> cluster.locking-scheme: granular >> cluster.eager-lock: enable >> performance.low-prio-threads: 32 >> network.ping-timeout: 30 >> cluster.granular-entry-heal: enable >> storage.owner-gid: 36 >> storage.owner-uid: 36 >> cluster.choose-local: true >> client.event-threads: 16 >> performance.strict-o-direct: on >> network.remote-dio: enable >> performance.client-io-threads: on >> nfs.disable: on >> storage.fips-mode-rchecksum: on >> transport.address-family: inet >> cluster.readdir-optimize: on >> cluster.metadata-self-heal: on >> cluster.data-self-heal: on >> cluster.entry-self-heal: on >> cluster.data-self-heal-algorithm: full >> features.uss: enable >> features.show-snapshot-directory: on >> features.barrier: disable >> auto-delete: enable >> snap-activate-on-create: enable >> >> Writing inside the /gluster_bricks yields those 2GB/sec writes, >Reading >> the same. >> >> Reading inside the /rhev/data-center/mnt/glusterSD/ dir reads go down >to >> 366mb/sec while writes plummet to to 200mb/sec. >> >> Summed up: Writing into the SSD Raid in the lvm/xfs gluster brick >> directory is fast, writing into the mounted gluster dir is horribly >slow. >> >> The above can be seen and repeated on all 3 servers. The network can >do >> full 10gbit (tested with, among others: rsync, iperf3). >> >> Anyone with some idea on whats missing/ going on here? >> >> Thanks folks, >> as always stay safe and healthy! >> >> -- >> with kind regards, >> mit freundlichen Gruessen, >> >> Christian Reiss >> _______________________________________________ >> Users mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> Privacy Statement: https://www.ovirt.org/privacy-policy.html >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> >https://lists.ovirt.org/archives/list/[email protected]/message/OMAAERV4IUISYEWD4QP5OAM4DK4JTTLF/ >>
Hey Chris,, You got some options. 1. To speedup the reads in HCI - you can use the option : cluster.choose-local: on 2. You can adjust the server and client event-threads 3. You can use NFS Ganesha (which connects to all servers via libgfapi) as a NFS Server. In such case you have to use some clustering like ctdb or pacemaker. Note:disable cluster.choose-local if you use this one 4 You can try the built-in NFS , although it's deprecated (NFS Ganesha is fully supported) 5. Create a gluster profile during the tests. I have seen numerous improperly selected tests -> so test with real-world workload. Synthetic tests are not good. Best Regards, Strahil Nikolov _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/44WU5WNVGQMX24FTCFOMLFWEOCIDNZQT/
