[Lustre-discuss] Mount 2 clusters, different networks - LNET tcp1-tcp2-o2ib

2011-06-14 Thread Thomas Roth
Hi all, I'd like to mount two Lustre filesystems on one client. Issues with more than one MGS set aside, the point here is that one of them is an Infiniband-cluster, the other is ethernet-based. And my client is on the ethernet. I have managed to mount the o2ib-fs by setting up an LNET router,

Re: [Lustre-discuss] Mount 2 clusters, different networks - LNET tcp1-tcp2-o2ib

2011-06-14 Thread Michael Shuey
Is your ethernet FS in tcp1, or tcp0? Your config bits indicate the client is in tcp1 - do the servers agree? -- Mike Shuey On Tue, Jun 14, 2011 at 12:23 PM, Thomas Roth t.r...@gsi.de wrote: Hi all, I'd like to mount two Lustre filesystems on one client. Issues with more than one MGS set

Re: [Lustre-discuss] Mount 2 clusters, different networks - LNET tcp1-tcp2-o2ib

2011-06-14 Thread Thomas Roth
Hm, the ethernet FS is in tcp0 - MGS says its nids are MGS-IP@tcp. So not surprising it refuses that connection. On the other hand, options lnet networks=tcp1(eth0),tcp(eth0:0) routes=o2ib LNET-Router-IP@tcp1; tcp Default-Gateway-IP@tcp results in Can't create route to tcp via

Re: [Lustre-discuss] Mount 2 clusters, different networks - LNET tcp1-tcp2-o2ib

2011-06-14 Thread Michael Shuey
That may be because your gateway doesn't have an interface on tcp (aka tcp0). I suspect you want to keep your ethernet clients in tcp0, your IB clients in o2ib0, and your router in both. Personally, I find it easiest to just give different module options on each system (rather than try ip2nets

Re: [Lustre-discuss] Mount 2 clusters, different networks - LNET tcp1-tcp2-o2ib

2011-06-14 Thread Thomas Roth
Thanks, Michael. I'll certainly put in the check_interval, that will be needed. However, what I tried was to have an ethernet client that mounts one FS via the LNET router (Infiniband-FS behind it) and simultaneously mounts the other FS, which is on tcp0 - via its default route. So actually

Re: [Lustre-discuss] Mount 2 clusters, different networks - LNET tcp1-tcp2-o2ib - solved?

2011-06-14 Thread Thomas Roth
Hi all, this seems to work with the correct IPs and correct network names ;-[] I now have the following modprobe on my ethernet client: options lnet networks=tcp1(eth0),tcp0(eth0:0) routes=o2ib LNET-Router@tcp1; tcp Default-Route@tcp1 With these options, loading the modules gives me Jun