I’d be curious to hear if all these arguments against iSCSI shouldn’t also apply to NSD protocol over TCP/IP?
-jf man. 17. des. 2018 kl. 01:22 skrev Jonathan Buzzard < jonathan.buzz...@strath.ac.uk>: > On 13/12/2018 20:54, Buterbaugh, Kevin L wrote: > > [SNIP] > > > > > Two things that I am already aware of are: 1) use jumbo frames, and 2) > > run iSCSI over it’s own private network. Other things I should be aware > > of?!? > > > > Yes, don't do it. Really do not do it unless you have datacenter > Ethernet switches and adapters. Those are the ones required for FCoE. > Basically unless you have per channel pause on your Ethernet fabric then > performance will at some point all go to shit. > > So what happens is your NSD makes a whole bunch of requests to read > blocks off the storage array. Requests are small, response is not. The > response can overwhelms the Ethernet channel at which point performance > falls through the floor. Now you might be lucky not to see this, > especially if you have say have 10Gbps links from the storage and 40Gbps > links to the NSD servers, but you are taking a gamble. Also the more > storage arrays you have the more likely you are to see the problem. > > To fix this you have two options. The first is datacenter Ethernet with > per channel pause. This option is expensive, probably in the same ball > park as fibre channel. At least it was last time I looked, though this > was some time ago now. > > The second option is dedicated links between the storage array and the > NSD server. That is the cable goes directly between the storage array > and the NSD server with no switches involved. This option is a > maintenance nightmare. > > At he site where I did this, we had to go option two because I need to > make it work, We ended up ripping it all out are replacing with FC. > > Personally I would see what price you can get DSS storage for, or use > SAS arrays. > > Note iSCSI can in theory work, it's just the issue with GPFS scattering > stuff to the winds over multiple storage arrays so your ethernet channel > gets swamped and standard ethernet pauses all the upstream traffic. The > vast majority of iSCSI use cases don't see this effect. > > There is a reason that to run FC over ethernet they had to turn ethernet > lossless. > > > JAB. > > -- > Jonathan A. Buzzard Tel: +44141-5483420 > HPC System Administrator, ARCHIE-WeSt. > University of Strathclyde, John Anderson Building, Glasgow. G4 0NG > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss