Thanks for the information.
Having said that, we seem to have solved the problem by
1) switching from using /etc/sysconfig/network-scripts/ifcfg-eth0 to /
etc/infiniband/ifcfg-ib0.
2) a minor change to the dhcpd.conf file
3) a change to the firewall so that it does not load at boot time. I
have since just fixed it.
4) a reboot of both the server and client nodes.
I am not exactly sure which piece(s) specifically did the trick.
While we do have the system working, dhclient gives the following
error-filled output whenever we make a request:
/sbin/dhclient-script: configuration for ib0 not found. Continuing
with defaults.
/etc/sysconfig/network-scripts/network-functions: line 78: ib0: No
such file or directory
Listening on Socket/ib0
Sending on Socket/ib0
Sending on Socket/fallback
DHCPREQUEST on ib0 to 255.255.255.255 port 67
DHCPACK from 172.10.10.2
/sbin/dhclient-script: configuration for ib0 not found. Continuing
with defaults.
/etc/sysconfig/network-scripts/network-functions: line 78: ib0: No
such file or directory
bound to 172.10.10.200 -- renewal in 36634 seconds.
Linking the ifcfg-ib0 file to the network-scripts directory seems to
correct this issue. However the machine will attempt to boot that
interface file before the ib_ipoib module is loaded. Although I do
not know what the defaults are per se, they seem to work.
Informationally, we are using CentOS 5.1 x86_64 with a
2.6.18-53.1.21.el5 kernel.
On the client we have the following configuration files...
/etc/infiniband/ifcfg-ib0:
DEVICE=ib0
ONBOOT=yes
BOOTPROTO=dhcp
/etc/dhclient.conf:
interface "ib0" {
send dhcp-client-identifier 20:00:55:00:01:FE:
80:00:00:00:00:00:00:00:11:75:00:00:ff:94:fd;
}
/etc/modprobe.conf
...
alias ib0 ib_ipoib
...
/etc/sysconfig/iptables
...
-A RH-Firewall-1-INPUT -p udp -m udp --sport 67:68 --dport 67:68 -j
ACCEPT
...
On the server we have the following configuration files...
/etc/dhcpd.conf
ddns-update-style interim;
ignore client-updates;
subnet 172.10.10.0 netmask 255.255.255.0 {
default-lease-time 86400;
max-lease-time 86400;
min-lease-time 3600;
option routers 172.10.10.2;
option subnet-mask 255.255.248.0;
option domain-name "univa.com";
option domain-name-servers 172.10.10.2;
range dynamic-bootp 172.10.10.100 172.10.10.200;
}
/etc/infiniband/ifcfg-eth0
DEVICE=ib0
ONBOOT=yes
BOOTPROTO=static
IPADDR=172.10.10.2
NETMASK=255.255.255.0
/etc/modprobe.conf
...
alias ib0 ib_ipoib
...
I have not bothered to configure the firewall on that machine yet.
Please feel free to contact me directly if you would like more
detailed information. Thanks again for the info...
Cheers,
Rod
On Jul 21, 2008, at 8:16 PM, Dave Olson wrote:
On Mon, 21 Jul 2008, Roderick Flores wrote:
| I created the client identifier from the following information:
|
| 20:<4 byte QP Number><8-byte subnet prefix><8 byte GUID>.
|
| This seems to be in line with the patch that we applied to the
DHCP software
| to make it work with infiniband.
I've not been able to get dhcp to work with ipoib, but I've not tried
that patch. Has anybody gotten it to work with recent OFED on any IB
HCAs?
| Lastly, an excerpt from /etc/modules.conf on ip1-test:
|
| alias ib0 ib_ipoib
|
| I tried to use ipath_ether but qlogic's website only has the
RHEL4 release
| (despite having a RHEL5 label).
It's the same set of rpms for RHEL4 and RHEL5 (identical tarball).
The marketing setup
insists on a per-distro download page, but we don't do independent
builds and testing, just the one RHEL build (which also covers CentOS
and Scientific Linux).
Dave Olson
[EMAIL PROTECTED]
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
