Hi, Thanks for answering!
Yes possible, I’m not too much into NFS and AFM so I might have used the wrong term.. I looked at what you suggested (very interesting reading) and setup multiple cache gateways to our home nfs server with the new afmParallelMount feature. It was as I suspected, for each gateway that does a write it gets 50-60MB/s bandwidth so although this utilizes more when adding it up (4 x gateways = 4 x 50-60MB/s) I’m still confused to why one server with one link cannot utilize more than the 50-60MB/s on 10Gb links ? Even 200-240MB/s is much slower than a regular 10Gbit interface. Best Regards Andi Christiansen Sendt fra min iPhone > Den 21. feb. 2020 kl. 18.25 skrev Tomer Perry <[email protected]>: > > Hi, > > I believe the right term is not multithreaded, but rather multistream. NFS > will submit multiple requests in parallel, but without using large enough > window you won't be able to get much of each stream. > So, the first place to look is here: > https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.4/com.ibm.spectrum.scale.v5r04.doc/bl1adm_tuningbothnfsclientnfsserver.htm > - and while its talking about "Kernel NFS" the same apply to any TCP socket > based communication ( including Ganesha). I tend to test the performance > using iperf/nsdperf ( just make sure to use single stream) in order to see > what is the expected maximum performance. > After that, you can start looking into "how can I get multiple streams?" - > for that there are two options: > https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.4/com.ibm.spectrum.scale.v5r04.doc/bl1ins_paralleldatatransfersafm.htm > and > https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.4/com.ibm.spectrum.scale.v5r04.doc/b1lins_afmparalleldatatransferwithremotemounts.htm > > The former enhance large file transfer, while the latter ( new in 5.0.4) will > help with multiple small files as well. > > > > Regards, > > Tomer Perry > Scalable I/O Development (Spectrum Scale) > email: [email protected] > 1 Azrieli Center, Tel Aviv 67021, Israel > Global Tel: +1 720 3422758 > Israel Tel: +972 3 9188625 > Mobile: +972 52 2554625 > > > > > From: Andi Christiansen <[email protected]> > To: "[email protected]" > <[email protected]> > Date: 21/02/2020 15:25 > Subject: [EXTERNAL] [gpfsug-discuss] Spectrum Scale Ganesha NFS multi > threaded AFM? > Sent by: [email protected] > > > > Hi all, > > i have searched the internet for a good time now with no answer to this.. So > i hope someone can tell me if this is possible or not. > > We use NFS from our Cluster1 to a AFM enabled fileset on Cluster2. That is > working as intended. But when AFM transfers files from one site to another it > caps out at about 5-700Mbit/s which isnt impressive.. The sites are connected > on 10Gbit links but the distance/round-trip is too far/high to use the NSD > protocol with AFM. > > On the cluster where the fileset is exported we can only see 1 session > against the client cluster, is there a way to either tune Ganesha or AFM to > use more threads/sessions? > > We have about 7.7Gbit bandwidth between the sites from the 10Gbit links and > with multiple NFS sessions we can reach the maximum bandwidth(each using > about 50-60MBits per session). > > Best Regards > Andi Christiansen _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
