So, for now, I'll put aside all the BMC discovery and just consider the
xCAT-style PXE based discovery, to have a common reference point. Same
capabilities, same limitations.
When PXE goes, the dynamic range is not needed because the following packet is
emitted by a node doing PXE boot:
Ethernet II, Src: Broadcom_e8:0e:b8 (b0:26:28:e8:0e:b8), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Source Port: 68
Destination Port: 67
....
Dynamic Host Configuration Protocol (Discover)
Message type: Boot Request (1)
Hardware type: Ethernet (0x01)
Hardware address length: 6
Hops: 0
Transaction ID: 0x21a78fc6
Seconds elapsed: 0
Bootp flags: 0x8000, Broadcast flag (Broadcast)
Client IP address: 0.0.0.0
Your (client) IP address: 0.0.0.0
Next server IP address: 0.0.0.0
Relay agent IP address: 0.0.0.0
Client MAC address: Broadcom_e8:0e:b8 (b0:26:28:e8:0e:b8)
Client hardware address padding: 00000000000000000000
Server host name not given
Boot file name not given
Magic cookie: DHCP
Option: (53) DHCP Message Type (Discover)
Option: (57) Maximum DHCP Message Size
Option: (55) Parameter Request List
Option: (97) UUID/GUID-based Client Identifier
Length: 17
Client Identifier (UUID): 0525c955-1812-11ea-b4d7-a3337fd960ac
Option: (94) Client Network Device Interface
Option: (93) Client System Architecture
Option: (60) Vendor class identifier
Option: (255) End
Note that the packet has no IP address yet. But all we are really wanting to
drive initial initial discovery is MAC and UUID, which are present. So
confluent receives that packet and then starts searching the ethernet switches
for that MAC and comes up with, say 'node3' based on net.*switch and
net.*switchport matches, and then discovers and then evaluates that DHCP
transaction in the context of 'node3', and if node3 is already set to pxe boot,
then it will jump straight to the static address for that node without touching
any intermediate dynamic IP (there's other options involving external DHCP, but
putting that aside for now.). "Discovery" with respect to gathering node
address is complete (hopefully) before the initial PXE attempt is even complete.
So confluent has external DHCP as optional, because it kind of sort of is a
DHCP server (but it doesn't know how to send a NAK and it will refuse to
consider non-PXE for now, in the interest of trying to avoid conflicting with
'real' DHCP servers on a subnet, and will not answer a node that isn't
specifically slated to deploy something). The idea is that OS deployment is
nice, but an option that users may ignore because they want the BMC management
commands and discovery, but want to use something else for OS deployment, so we
aim for max compatibility.
The 'can do it off' relies on the BMC first approach, in which all the
multicast IPv6/IPv4 trickery potentially involving IPv6 link local addresses to
overcome messed up IPv4 comes in, and requires a supported BMC with either
preconfigured user/password or firmware default user/password to be vaguely
alive on a connected network.
Incidentally, one challenge I have is knowing when people want this detail,
versus how many people want it to be 'invisible'. Skimming this email without
knowing would lead me to think a user has to understand this detail and I might
be dismissive that it's pretty complicated to use, versus explaining the
mechanism of the "don't worry about it" tricks that are in use. Particularly
in documentation, how to clearly provide it, yet make it clear that 'TLDR' can
happily ignore it....
________________________________
From: Thomas HUMMEL <thomas.hum...@pasteur.fr>
Sent: Wednesday, October 23, 2024 10:01 AM
To: xcat-user@lists.sourceforge.net <xcat-user@lists.sourceforge.net>
Subject: Re: [xcat-user] [External] Re: xCAT Consortium Update
On 10/23/24 2:55 PM, Jarrod Johnson via xCAT-user wrote:
>> a) So the BMC config is made out of band ? This would imply 2 DHCP
> servers if 1 vlan for data and 1 for BMC ?
>
> One thing to point out, is that we may have arbitrarily many or few networks,
> similar to xCAT.
Thanks again for your explanations but unfortunately you're too far
ahead for me to fully grasp the principles. I only get part of what you say.
I must miss a lot of concepts you are using.
I come from xCAT where I switch-based discover nodes by pxe-ing them
into genesis which I use to run bmcsetup to configure the BMC which
indeed is on a tagged routed vlan
In this setup I don't see how we can avoid a dynamic range (at least if
we want to be able to simultaneously discover several nodes)
What I think I understand :
1. dhcp is optional in the pxe case as pxe emitted packets are enough to
get the mac address. However pxe seems to be required for later statless
boot ?
2. discovery image (as far as BMC is concerned) is optional if BMC is
configured out of band but only if setup is simple. Ex: if BMC, as in my
case is on a routed tagged vlan you have a bootstrap problem and better
configure it inband from a discovery image or take the risk to override
it's config out of band
What I think you say:
You can get node mac adress by reading it from the BMC (which knows it)
What I don't understand:
how do you make the dynamic range not necessary ?
how do you join mac address with switch/port if nodes are off ?
What configures BMC and how ?
So basically I did not understand the steps such a discovery involve
Note: feel free to ignore this question if answer is inside docs (I just
skim throuh it very quickly)
--
Thomas HUMMEL
HPC Group
Institut PASTEUR
Paris, FRANCE
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fxcat-user&data=05%7C02%7Cjjohnson2%40lenovo.com%7Cc21eefc2eb0340d5675a08dcf36b5d17%7C5c7d0b28bdf8410caa934df372b16203%7C0%7C0%7C638652889724966073%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=dlz7bDnzpG0FREeYCtAiwldtP6nIXVD1XEf1qxhR2B0%3D&reserved=0<https://lists.sourceforge.net/lists/listinfo/xcat-user>
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
