hi all, what is the expected behaviour of a mixed verbsRdmaSend setup: some nodes enabled, most disabled.
we have some nodes that have a very high iops workload, but most of the cluster of 500+ nodes do not have such usecase. we enabled verbsRdmaSend on the managers/quorum nodes (<10) and on the few (<10) clients with this workload, but not on the others (500+). it seems to work out fine, but is this acceptable as config? (the docs mention that enabling verbsrdamSend on a> 100 nodes might lead to errors). the nodes use ipoib as ip network, and running with verbsRdmaSend disabled on all nodes leads to unstable cluster (TX errors (<1 error in 1M packets) on some clients leading to gpfs expel nodes etc). (we still need to open a case wil mellanox to investigate further) many thanks, stijn _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
