FYI, another potentially interesting strategy:
[root@s1 ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
mode DEFAULT group default qlen 1000
link/ether 08:94:ef:40:8a:96 brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT group default qlen 1000
link/ether 08:94:ef:40:8a:97 brd ff:ff:ff:ff:ff:ff
4: enp0s20f0u1u6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
mode DEFAULT group default qlen 1000
link/ether 0a:94:ef:40:87:23 brd ff:ff:ff:ff:ff:ff
[root@s1 ~]# ip netns add mgt
[root@s1 ~]# ip link set dev enp0s20f0u1u6 netns mgt
[root@s1 ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
mode DEFAULT group default qlen 1000
link/ether 08:94:ef:40:8a:96 brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT group default qlen 1000
link/ether 08:94:ef:40:8a:97 brd ff:ff:ff:ff:ff:ff
[root@s1 ~]#
[root@s1 ~]# ping -c 1 169.254.95.118
PING 169.254.95.118 (169.254.95.118) 56(84) bytes of data.
From 172.30.83.1 icmp_seq=1 Destination Host Unreachable
--- 169.254.95.118 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time
0ms
[root@s1 ~]# ip netns exec mgt ping -c 1 169.254.95.118
PING 169.254.95.118 (169.254.95.118) 56(84) bytes of data.
64 bytes from 169.254.95.118: icmp_seq=1 ttl=64 time=2.69 ms
--- 169.254.95.118 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.693/2.693/2.693/0.000 ms
I actually plan to codify that into our tool wrappers to recognize the
situation and take advantage so the nic can exist, but not normally be
usable or interfere.
This discussion is of interest as we are currently talking about the
'BMC-to-OS' behavior for future standardization.
On preferring KCS not to be there, is this because it is extraneous or
because it leaves root-level user access to reconfigure networking and
credentials for the BMC? Would that sort of thing be less worrisome
when configured to only allow read access once in an OS (enough for
'sensors', 'fru', and 'sel' sorts of commands to work, but not for 'lan
set' and such.
Going beyond the general concept of privileged channel, per this thread
the USB ethernet device obviously causes a problem by doing DHCP OFFER
for a point to point IPv4 address. Clearly we have to be flexible, but
if the general behavior is that no DHCPv4, DHCPv6, or RA sorts of
packets come out of the BMC, would the stray USB ethernet device still
be such a problem? If the software behavior was such that it would
prefer to use the fe80::/64 addresses when available, in such a
scenario never establishing a network identity that could be
misconstrued as widely valid by software?
To sum up my current take on what the DMTF guys are pushing for and the
compromise I would propose:
-BMC must *never* offer DHCP or similar by default over an internal NIC
-Any such 'BMC-to-OS' facility must be able to be disabled when OS
boots (it may still be active in firmware 'setup' menu'). For a
platform without a video port, then an alternative hardware mechanism
to reset to defaults must be provided.
-To the extent a mechanism exist to allow privileged users in OS to
configure a BMC without knowledge of any valid user/password (i.e. the
way KCS works), that mechanism must be able to be either disabled or
limited to read-only commands, even if the facility is more generally
up for authenticated use.
-----Original Message-----
From: Stuart Barkley <stua...@4gh.net>
Reply-To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
To: xcat-user@lists.sourceforge.net
Subject: [External] [xcat-user] IBM IPMI USB interface
Date: Thu, 31 May 2018 00:03:07 -0400
We used to use IMM.LanOverUsb=Disabled but I changed it a while back
to be more standard with what IBM seemed to prefer. I will probably
switch back to that mode of operation shortly.
I would prefer the IPMI USB interface not exist at all.
I would also (mostly) prefer that the KCS interface not be there
either, except once or twice that interface was useful to recover an
IMM which had lost it's network configuration. The KCS interface was
probably also used once on each system many years ago when performing
an xCAT discovery operaton.
Other than needing to unpack a couple levels of archives for the IBM
distributed firmware files, flashing the firmware over the Ethernet
IMM interface using iflash has worked very well for us.
On Wed, 30 May 2018 at 14:07 -0000, Jarrod Johnson wrote:
> Note that in such a case, would have to do firmware updates for UEFI
> and IMM remotely (not the worst thing in the world). Another
> alternative is to disable it in firmware:
> # nodeconfig s1 IMM.LanOverUsb=disable
>
> Or
>
> # pasu s1 set IMM.LanOverUsb Disabled
>
> This is a point of interest as there is a push to migrate more
> in-band instrumentation toward using that USB nic, to the point of
> some folks asking to remove IPMI KCS (which I oppose personally,
> never been a fan of IP as an ‘within-the-box’ protocol, precisely
> because software has understandably assumed ‘network’ meant talking
> to other systems not internal point to point).
>
> At the very least I’m pushing to no longer DHCPOFFER anything on
> that interface, and rework the software to use fe80:: addressing
> instead to avoid any network that OpenMPI or other similarly
> optimistic software getting confused by non-viable networks.
>
> As a point of historical interest, originally it did not DHCPOFFER,
> but some OS boot would block waiting for DHCP to complete, so the
> DHCPOFFER was done to speed up boot process of those platforms. I
> don’t think any OS would be foiled by a link but no DHCP offer on a
> stray network anymore.
>
> None of this is likely to happen in currently or past shipped
> product, but would be effective as of the release of future product.
>
>
> From: Kevin Keane <kke...@sandiego.edu>
> Sent: Wednesday, May 30, 2018 12:20 PM
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
> Subject: [External] Re: [xcat-user] CentOS 7, diskless compute and
> NetworkManager
>
> I had a similar problem with the USB-based IPMI network interfaces.
> I blacklisted the driver. In the image for your node, add the
> following file:
> /etc/modprobe.d/usbeth_blacklist.conf
> With the content (verify that cdc_ether is the correct driver
> first!)
> blacklist cdc_ether
> You can probably also edit /etc/sysconfig/network-scripts/ifcfg-usb0
> and set ONBOOT=no
---------------------------------------------------------------------
---------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
