I also don't know much about this, but the ESS quick deployment guide is quite clear on the we should use connected mode for IPoIB:
-------------- Note: If using bonded IP over IB, do the following: Ensure that the CONNECTED_MODE=yes statement exists in the corresponding slave-bond interface scripts located in /etc/sysconfig/network-scripts directory of the management server and I/O server nodes. These scripts are created as part of the IP over IB bond creation. An example of the slave-bond interface with the modification is shown below. --------------- -jf fre. 12. mai 2017 kl. 16.48 skrev Aaron Knister <[email protected]>: > For what it's worth we've seen *significantly* better performance of > streaming benchmarks of IPoIB with connected mode vs datagram mode on IB. > > -Aaron > > On 5/12/17 10:43 AM, Jonathon A Anderson wrote: > > This won’t tell you which to use; but datagram mode and connected mode > in IB is roughly analogous to UDB vs TCP in IP. One is “unreliable” in that > there’s no checking/retry built into the protocol; the other is “reliable” > and detects whether data is received completely and in the correct order. > > > > The last advice I heard for traditional IB was that the overhead of > connected mode isn’t worth it, particularly if you’re using IPoIB (where > you’re likely to be using TCP anyway). That said, on our OPA network we’re > seeing the opposite advice; so I, to, am often unsure what the most correct > configuration would be for any given fabric. > > > > ~jonathon > > > > > > On 5/12/17, 4:42 AM, "[email protected] on > behalf of Damir Krstic" <[email protected] on > behalf of [email protected]> wrote: > > > > I never fully understood the difference between connected v. > datagram mode beside the obvious packet size difference. Our NSD servers > (ESS GL6 nodes) are installed with RedHat 7 and are in connected mode. Our > 700+ clients are running RH6 and > > are in datagram mode. > > > > > > In a month we are upgrading our cluster to RedHat 7 and are debating > whether to leave the compute nodes in datagram mode or whether to switch > them to connected mode. > > What is is the right thing to do? > > > > > > Thanks in advance. > > Damir > > > > > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at spectrumscale.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > -- > Aaron Knister > NASA Center for Climate Simulation (Code 606.2) > Goddard Space Flight Center > (301) 286-2776 > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
