On Thursday, 26 November 2020 3:14:05 AM PST Jörg Saßmannshausen wrote: > Now, traditionally I would say that we are going for InfiniBand. However, > for reasons I don't want to go into right now, our existing file storage > (Lustre) will be in a different location. Thus, we decided to go for RoCE > for the file storage and InfiniBand for the HPC applications.
I think John hinted at this, but is there a reason for not going for IB for the cluster and then using Lnet routers to connect out to the Lustre storage via ethernet (with RoCE) ? https://wiki.lustre.org/LNet_Router_Config_Guide We use Lnet routers on our Cray system to bridge between the Aries interconnect inside the XC to the IB fabric our Lustre storage sits on. All the best, Chris -- Chris Samuel : http://www.csamuel.org/ : Berkeley, CA, USA _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf