That may be because your gateway doesn't have an interface on tcp (aka tcp0). I suspect you want to keep your ethernet clients in tcp0, your IB clients in o2ib0, and your router in both. Personally, I find it easiest to just give different module options on each system (rather than try ip2nets stuff).
On the ether clients, I'd try: options lnet networks=tcp0(eth0) routes="o2ib0 LNET-router-eth_IP@tcp0" dead_router_check_interval=300 On IB clients: options lnet networks=o2ib0(ib0) routes="tcp0 LNET-router-IB_IP@ib0" dead_router_check_interval=300 then on the router: options lnet networks=tcp0(eth0),o2ib0(ib0) forwarding=enabled accept_timeout=15 Obviously, your file servers will need to have lnet options similar to the clients: options lnet networks=tcp0(eth0) routes="o2ib0 LNET-router-eth_IP@tcp0" dead_router_check_interval=300 options lnet networks=o2ib0(ib0) routes="tcp0 LNET-router-IB_IP@o2ib0" dead_router_check_interval=300 That's just a guess, your mileage may vary, etc., but I think it's close to what you want. Note that you really want the dead_router_check_interval if you're using lnet routers. Without that parameter, the lustre client will automatically mark a router as failed when it's unavailable but will not check to see if it ever comes back. With this param, it checks every 300 seconds (and re-enables it if found). Hope this helps. -- Mike Shuey On Tue, Jun 14, 2011 at 1:26 PM, Thomas Roth <t.r...@gsi.de> wrote: > Hm, the ethernet FS is in tcp0 - MGS says its nids are MGS-IP@tcp. > So not surprising it refuses that connection. > On the other hand, >> options lnet networks=tcp1(eth0),tcp(eth0:0) routes="o2ib >> LNET-Router-IP@tcp1; tcp Default-Gateway-IP@tcp" > > results in >> Can't create route to tcp via Gateway-IP@tcp > > Cheers, > Thomas > > > On 06/14/2011 07:00 PM, Michael Shuey wrote: >> >> Is your ethernet FS in tcp1, or tcp0? Your config bits indicate the >> client is in tcp1 - do the servers agree? >> >> -- >> Mike Shuey >> >> >> >> On Tue, Jun 14, 2011 at 12:23 PM, Thomas Roth <t.r...@gsi.de> wrote: >> > Hi all, >> > >> > I'd like to mount two Lustre filesystems on one client. Issues with >> more than one MGS set aside, >> > the point here is that one of them is an Infiniband-cluster, the other >> is ethernet-based. >> > And my client is on the ethernet. >> > I have managed to mount the o2ib-fs by setting up an LNET router, but >> now this client's LNET doesn;t >> > known how to reach the ethernet-fs. >> > >> > So the basic modprobe.conf reads >> > > options lnet networks=tcp1(eth0) routes="o2ib LNET-Router-IP@tcp1" >> > This mounts the MGS on the o2ib network. >> > >> > What do I have to add to get to the MGS on the tpc network? >> > >> > Meanwhile I have studied more posts here and came up with >> > > options lnet networks=tcp1(eth0),tcp2(eth0:0) routes="o2ib >> LNET-Router-IP@tcp1; tcp >> > Default-Gateway-IP@tcp2" >> > >> > Doesn't work either, but I see in the log of the (tcp-)MGS: >> > > LustreError: 120-3: Refusing connection from Client-IP for >> MGS-IP@tcp2: No matching NI >> > >> > Somethings getting through ... >> > >> > Any ideas? >> > >> > Regards, >> > Thomas >> > _______________________________________________ >> > Lustre-discuss mailing list >> > Lustre-discuss@lists.lustre.org >> > http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > >> > > > _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss