On Jun 03, 2009 17:45 -0400, Erik Froese wrote: > I'm trying to configure a lustre router so I can mount a test lustre FS over > our standard network here (NYU federated ethernet). > We have a small rocks cluster with one MDS/MGS and 3 OSSs on a private > switch. Its a pretty standard rocks configuration. > The cluster network is 10.1.255.0/24. > > == OSS / Router == > One of the OSS (oss-0-2) is configured as follows: > eth0 - 10.1.255.247 > eth1 - 128.122.x.y > > In its /etc/modprobe.conf I have the following > options lnet forwarding="enabled" > options lnet accept=all > options lnet networks="tcp0(eth0),tcp1(eth1)" > > [r...@oss-0-2 ~]# lctl list_nids > 10.1.255....@tcp > 128.122....@tcp1
I'm not a routing expert, but I think I can see what is wrong here. To reiterate - your private network is tcp0 (10.x), and your external network is tcp1 (128.x). > == Routed Client == > Then I have another client on the 128.122.x.* network. Let's call it > 128.122.x.z It just has eth0 configured as 128.122.x.z > > and in its modprobe.conf > options lnet networks=tcp0(eth0) routes="tcp1 128.122....@tcp0" Here you are configuring your external client to use tcp0 as 128.x, which does NOT match what you have configured on your router. You need to have (I think): options lnet networks=tcp1(eth0) routes="tcp0 128.122....@tcp1" > Now should I be able to mount the lustre fs as such? > mount.lustre 10.1.255....@tcp0:/scratch /scratch > mount.lustre: mount 10.1.255....@tcp:/scratch at /scratch failed: Cannot > send after transport endpoint shutdown This should be fine, once your routing is working. You could try "lctl ping 128.122....@tcp1" to verify you can communicate with your router/OSS and "lctl ping 10.1.255....@tcp0" for the MDS behind it. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
