OK here's where I am now. The public client can ping the routers public address but not the private address.
[r...@routed-client lnet]$ cat /etc/modprobe.conf alias eth0 e1000 alias eth1 e1000 alias scsi_hostadapter megaraid_mbox alias scsi_hostadapter1 ata_piix # eth0 is part of tcp1 (NYU-NET) # In order to get to tcp (Cluster private), use the network on # 128.122.X.Y at tcp1 options lnet accept=all options lnet networks=tcp1(eth0) routes="tcp 128.122....@tcp1" [r...@routed-client lnet]$ lctl network up LNET configured [r...@routed-client lnet]$ cat /proc/sys/lnet/routes Routing disabled net hops state router tcp 1 up 128.122.109...@tcp1 [r...@routed-client lnet]$ cat /proc/sys/lnet/routers ref rtr_ref alive_cnt state last_ping router 3 1 0 up 0 128.122.109...@tcp1 [r...@routed-client lnet]$ lctl ping 128.122.109...@tcp1 1234...@lo 12345-10.1.255....@tcp 12345-128.122.109...@tcp1 [r...@routed-client lnet]$ lctl ping 10.1.255....@tcp failed to ping 10.1.255....@tcp: Input/output error I can see traffic between the routed-client and the router as well as between the router and the MGS/MDS (10.1.255....@tcp) The mgs has the following config. [r...@mgs-0-0 lnet]# cat /etc/modprobe.conf alias scsi_hostadapter mptbase alias scsi_hostadapter1 mptsas alias scsi_hostadapter2 usb-storage alias eth0 e1000 alias eth1 e1000 alias eth2 e1000 alias eth3 e1000 options lnet forwarding="enabled" options lnet accept=all options lnet networks=tcp(eth0) routes="tcp1 10.1.255....@tcp" [r...@mgs-0-0 lnet]# lctl network up LNET configured But it doesn't see any routes or routers. [r...@mgs-0-0 lnet]# cat /proc/sys/lnet/routes Routing disabled net hops state router [r...@mgs-0-0 lnet]# cat /proc/sys/lnet/routers ref rtr_ref alive_cnt state last_ping router And this is what /var/log/messages and dmesg contain with or without enabling neterror logging Jun 11 22:41:07 mgs-0-0 kernel: LustreError: 10869:0:(lib-move.c:1250:lnet_send()) No route to 12345-128.122....@tcp1 Jun 11 22:41:07 mgs-0-0 kernel: LustreError: 10869:0:(lib-move.c:1723:lnet_parse_get()) 10.1.255....@tcp: Unable to send REPLY for GET from 12345-128.122....@tcp1: -113 On Fri, Jun 5, 2009 at 12:48 PM, Isaac Huang <[email protected]> wrote: > On Thu, Jun 04, 2009 at 01:59:48PM -0400, Erik Froese wrote: > > Thanks Andreas and Natalie, > > > > I've made the changes you suggested (setting tcp1 as the external > > network) and I'm able to lctl ping the 128.122.x.y address but I still > > cannot ping the private address for the MDS. > > Please show us the commands you've run and their outputs, together > with error messages in dmesg. It'd help to "echo +neterror > > /proc/sys/lnet/printk" before running the commands. > > > Could the problem be that the lustre fs on the private network is > > actually called tcp and not tcp0? Are those synonymous? > > No, 'tcp' is just a shorthand for 'tcp0' - they are 100% equivalent > to each other. > > Isaac >
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
