On Jun 25, 2008 20:04 -0400, Scott Atchley wrote: > When I test Lustre over myri10ge, I do not use "myri10ge" as the > network interface name. I use the actual ethX that myri10ge is > providing: > > options lnet networks="tcp0(eth2),tcp1(eth0)"
If you are using tcpX(ethX), then you are only using TCP for the Lustre Network transport instead of the RDMA MX transport and it isn't as fast or efficient as it could be. > On Jun 20, 2008, at 10:17 AM, Bastian Tweddell wrote: > > > Hello, > > > > on our cluster all computenodes are equiped with 2 interfaces. eth0 > > is > > 1GBit and myri10ge is 10GBit. We have three maschines for MDS, OSS1 > > and OSS2. Their lnet configuration looks like: > > > > options lnet networks="tcp0(myri10ge),tcp1(eth0)" > > > > Client nodes are configured with: > > > > options lnet networks="tcp0(myri10ge)" > > > > When node001 mounts the lustre fs, it uses the 10gbit ethernet as > > configured. Thats verified by network monitoring. But the peer_list > > command executed on the MDS lists a peer to node001 over the 1GB > > ethernet: > > > > lctl > network tcp0 > > lctl > peer_list > > [EMAIL PROTECTED] [0]0.0.0.0->0.0.0.0:0 #0 <- > > thats node001 > > [EMAIL PROTECTED] [1]node009-10g->node015-10g:988 #3 <- > > thats oss1 > > [EMAIL PROTECTED] [1]node009-10g->node016-10g:988 #3 <- > > thats oss2 > > > > where 192.168.10.0/24 is the 10gbit network and 192.168.42.0/24 the > > 1gbit. > > > > What am I doing wrong? Is there a configuration issue? > > And what does [0]0.0.0.0->0.0.0.0:0 #0 in line 1 mean? > > > > Thanks for any hints. > > > > Bastian > > > > > > > > -- > > Bastian Tweddell > > Juelich Supercomputing Centre > > Institute for Advanced Simulation > > Forschungszentrum Juelich GmbH > > 52425 Juelich, Germany > > > > Phone: +49-2461-61-6586 > > Fax: +49-2461-61-6656 > > WWW: http://www.fz-juelich.de/jsc/ > > > > JSC is the coordinator of the > > John von Neumann Institute for Computing > > and member of the > > Gauss Centre for Supercomputing > > > > > > ------------------------------------------------------------------- > > ------------------------------------------------------------------- > > Forschungszentrum Juelich GmbH > > 52425 Juelich > > > > Sitz der Gesellschaft: Juelich > > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > > Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe > > Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), > > Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr. Harald Bolt, > > Dr. Sebastian M. Schmidt > > ------------------------------------------------------------------- > > ------------------------------------------------------------------- > > _______________________________________________ > > Lustre-discuss mailing list > > [email protected] > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
