Why do you need both active? If one is a backup to the other, then bond them as a primary/backup pair, meaning only one will be active at at a time, ie, your designated primary (unless it goes down).
bob On 10/21/2010 9:51 AM, Brock Palen wrote: > On Oct 21, 2010, at 9:48 AM, Joe Landman wrote: > >> On 10/21/2010 09:37 AM, Brock Palen wrote: >>> We recently added a new oss, it has 1 1Gb interface and 1 10Gb >>> interface, >>> >>> The 10Gb interface is eth4 10.164.0.166 The 1Gb interface is eth0 >>> 10.164.0.10 >> They look like they are on the same subnet if you are using /24 ... > You are correct > > Both interfaces are on the same subnet: > > [r...@oss4-gb ~]# route > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use Iface > 10.164.0.0 * 255.255.248.0 U 0 0 0 eth0 > 10.164.0.0 * 255.255.248.0 U 0 0 0 eth4 > 169.254.0.0 * 255.255.0.0 U 0 0 0 eth4 > default 10.164.0.1 0.0.0.0 UG 0 0 0 eth0 > > There is no way to mask the lustre service away from the 1Gb interface? > >>> In modprobe.conf I have: >>> >>> options lnet networks=tcp0(eth4) >>> >>> lctl list_nids 10.164.0....@tcp >>> >>>> From a host I run: >>> lctl which_nid oss4 10.164.0....@tcp >>> >>> But yet I still see traffic over eth0 the 1Gb management network, >>> might higher than I would expect (upto 100MB/s) The management >>> interface is oss4-gb So If I do from a client: >>> >>> lctl which_nid oss4-gb 10.164.0...@tcp >>> >>> Why If I have netwroks=tcp0(eth4) and that list_nids showa only the >>> 10Gb interface, do I have so much traffic over the 1Gb interface? >>> There is some traffic on the 10Gb interface, but I would like to tell >>> lustre 'don't use the 1Gb interface'. >> If they are on the same subnet, its possible that the 1GbE sees the arp >> response first. And then its pretty much guaranteed to have the traffic >> go out that port. >> >> If your subnets are different, this shouldn't be the issue. >> >>> Thanks! >>> >>> Brock Palen www.umich.edu/~brockp Center for Advanced Computing >>> bro...@umich.edu (734)936-1985 >>> >>> >>> >>> _______________________________________________ Lustre-discuss >>> mailing list Lustre-discuss@lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> -- >> Joseph Landman, Ph.D >> Founder and CEO >> Scalable Informatics Inc. >> email: land...@scalableinformatics.com >> web : http://scalableinformatics.com >> http://scalableinformatics.com/jackrabbit >> phone: +1 734 786 8423 x121 >> fax : +1 866 888 3112 >> cell : +1 734 612 4615 >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss@lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss