Hi, Thomas:
We noticed similar issue in our lab too. The best workaround is set `
DEFROUTE=no` ,
so the FIRST setup is best solution. did u see anything issue for this
setup?
for the second setup, 172_16_0_0-255_255_248_0 is not define in the
network table.
Thanks,
Casandra Qiu
...................................................................
Casandra Hong Qiu
Phone: (845) 433-9291, t/l 293-9291
Office: Building 8, 3-B-04
cxh...@us.ibm.com
From: Thomas HUMMEL <thomas.hum...@pasteur.fr>
To: xcat-user@lists.sourceforge.net
Date: 03/10/2020 07:18 AM
Subject: [EXTERNAL] [xcat-user] IPoIB, confignetwork and incorrect
default routes
Hello,
I'm using xCAT-2.15 initial release to provision stateless nodes which
all have one Ethernet nic and one Infiniband nic.
I want to use the IPoIB ULP (Mellanox OFED packages/yum install method),
thus to configure an ip address on both nics.
Note : I don't want to route the ipoib network.
When using nics table + confignetwork (-s) to do so, results vary
depending on the following setups in the nics table
1/
#node,nicips,nichostnamesuffixes,nichostnameprefixes,nictypes,niccustomscripts,nicnetworks,nicaliases,nicextraparams,nicdevices,nicsadapter,comments,disable
"compute","ib0!|maestro-(\d+)$|172.16.($1/250-$1/250/2-2).($1%250
+2)|","ib0!-ib0",,"ib0!Infiniband",,"ib0!maestro_ipoib",,"ib0!
https://github.com/xcat2/xcat-core/pull/6630";,,,,
-> ip on ib0 ok, routes ok
2/ same setup but network defined inside nics table itself
#node,nicips,nichostnamesuffixes,nichostnameprefixes,nictypes,niccustomscripts,nicnetworks,nicaliases,nicextraparams,nicdevices,nicsadapter,comments,disable
"compute","ib0!|maestro-(\d+)$|172.16.($1/250-$1/250/2-2).($1%250
+2)|","ib0!-ib0",,"ib0!Infiniband",,"ib0!172_16_0_0-255_255_248_0",,"ib0!DEFROUTE=no",,,,
-> no ip gets configured on ib0 : why (network seems identical to the
one defined in networks table (see 3/) to me ?
3/ same setup but without forcing no default route :
-> ip on ib0 ok, BOGUS routes table (2 default routes, same gateway,
different nics) :
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt
Iface
0.0.0.0 192.168.144.1 0.0.0.0 UG 0 0 0
enp33s0f0
0.0.0.0 192.168.144.1 0.0.0.0 UG 0 0 0
ib0
172.16.0.0 0.0.0.0 255.255.248.0 U 0 0 0
ib0
192.168.144.0 0.0.0.0 255.255.240.0 U 0 0 0
enp33s0f0
192.168.144.1 0.0.0.0 255.255.255.255 UH 0 0 0
ib0
-> where does this come from ?
Here's, for every of the above tests my related setup :
- networks
#netname,net,mask,mgtifname,gateway,dhcpserver,tftpserver,nameservers,ntpservers,logservers,dynamicrange,staticrange,staticrangeincrement,nodehostname,ddnsdomain,vlanid,domain,mtu,comments,disable
"maestro_net","192.168.144.0","255.255.240.0","eth0","192.168.144.1",,"<xcatmaster>",,,,"192.168.144.2-192.168.147.254",,,,,,"maestro.pasteur.fr","1500",,
"maestro_ipoib","172.16.0.0","255.255.248.0",,,,,,,,,,,,,,,"1500",,
- site
"domain","maestro.pasteur.fr",,
A side, more general and different, question about nics is : what's the
use of nics.networks attribute relative to domain, as in my experience,
domain is always found by matching the ipaddress against networks table
or site.domain is used ?
Thanks for your help
--
Thomas HUMMEL
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_xcat-2Duser&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=n1LR_Py9TQX0dVqfGTbLHUMGx25-C8VtBDS0nCzyNXY&m=Af_6BSVZIUtEt4I4oYIIL-vpdfWzVHTW3h9YMy1YtC0&s=vLUHkQiup_1Wij66TMJ0IzzYCIjfbU7u6uSqagTD-9w&e=
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
