Are you running IPoIB? Can you do “lsmod |grep lnet”? Also, ensure you have the right network settings in your /etc/modprobe.d/lnet.conf file (or wherever you may have defined the networks)
Brian Andrus ITACS/Research Computing Naval Postgraduate School Monterey, California voice: 831-656-6238 From: lustre-discuss [mailto:[email protected]] On Behalf Of sohamm Sent: Thursday, July 28, 2016 6:55 PM To: Mohr Jr, Richard Frank (Rick Mohr) Cc: [email protected] Subject: Re: [lustre-discuss] luster client mount issues Hi Rick Client is configured for IB interface. in my understanding i can specific the network of choice in the mount command. tried both tcp and ib. I am still checking on the configurations as suggested in the forumn. will get back with my findings. Thanks On Thursday, July 28, 2016, Mohr Jr, Richard Frank (Rick Mohr) <[email protected]<mailto:[email protected]>> wrote: Is the client supposed to have an IB interface configured, or is it just supposed to mount over ethernet? -- Rick Mohr Senior HPC System Administrator National Institute for Computational Sciences http://www.nics.tennessee.edu > On Jul 20, 2016, at 2:09 PM, sohamm <[email protected]<javascript:;>> wrote: > > Hi > > Any guidance/help on this is greatly appreciated. > > Thanks > > On Mon, Jul 18, 2016 at 7:25 PM, sohamm <[email protected]<javascript:;>> > wrote: > Hi Ben > Both the networks have netmasks of value 255.255.255.0 > > Thanks > > On Mon, Jul 18, 2016 at 10:08 AM, Ben Evans <[email protected]<javascript:;>> > wrote: > What do your netmasks look like on each network? > > From: lustre-discuss <[email protected]<javascript:;>> > on behalf of sohamm <[email protected]<javascript:;>> > Date: Monday, July 18, 2016 at 1:56 AM > To: "[email protected]<javascript:;>" > <[email protected]<javascript:;>> > Subject: Re: [lustre-discuss] lustre-discuss Digest, Vol 124, Issue 17 > > Hi Thomas > Below are the results of the commands you suggested. > > From Client > [root@dev1 ~]# lctl ping 192.168.200.52@o2ib<mailto:192.168.200.52@o2ib> > failed to ping 192.168.200.52@o2ib<mailto:192.168.200.52@o2ib>: Input/output > error > [root@dev1 ~]# lctl ping 192.168.111.52@tcp<mailto:192.168.111.52@tcp> > 12345-0@lo > 12345-192.168.200.52@o2ib<mailto:12345-192.168.200.52@o2ib> > 12345-192.168.111.52@tcp<mailto:12345-192.168.111.52@tcp> > [root@dev1 ~]# mount -t lustre > 192.168.111.52@tcp:/mylustre<mailto:192.168.111.52@tcp:/mylustre> /lustre > mount.lustre: mount > 192.168.111.52@tcp:/mylustre<mailto:192.168.111.52@tcp:/mylustre> at /lustre > failed: Input/output error > Is the MGS running? > mount: mounting > 192.168.111.52@tcp:/mylustre<mailto:192.168.111.52@tcp:/mylustre> on /lustre > failed: Invalid argument > > cat /var/log/messages | tail > Jul 18 01:37:04 dev1 user.warn kernel: [2250504.401397] ib1: multicast join > failed for ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22 > Jul 18 01:37:26 dev1 user.warn kernel: [2250526.257309] LNet: No route to > 12345-192.168.200.52@o2ib<mailto:12345-192.168.200.52@o2ib> via <?> (all > routers down) > Jul 18 01:37:36 dev1 user.warn kernel: [2250536.481862] ib1: multicast join > failed for ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22 > Jul 18 01:41:53 dev1 user.warn kernel: [2250792.947299] LNet: No route to > 12345-192.168.200.52@o2ib<mailto:12345-192.168.200.52@o2ib> via <?> (all > routers down) > > > From MGS > [root@lustre_mgs01_vm03 ~]# lctl ping > 192.168.111.102@tcp<mailto:192.168.111.102@tcp> > 12345-0@lo > 12345-192.168.111.102@tcp<mailto:12345-192.168.111.102@tcp> > > Please let me know what else i can try. Looks like i am missing something > with the ib config? Do i need router setup as part of lnet ? > if i am able to ping mgs from client on the tcp network, it should still work > ? > > Thanks >
_______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
