I just remembered another option. You can configure multiple NICs for a given system so that when its mac addr requests PXE install, it can do install (but not need a reserved address or DNS entry). No matter what, we can start install regardless of which NIC starts install. We can still specify which NIC to do DHCP on in anaconda/preseed. Since we're already creating all NICs for a given system, we can just leave the IP address blank. I'll write up a quick patch for this.
On Tue, Jan 28, 2014 at 12:24 PM, Andrey Danin <[email protected]> wrote: > Dnsmasq can do it but Cobbler doesn't allow it. > > > On Mon, Jan 27, 2014 at 9:43 PM, Andrew Woodward <[email protected]> wrote: >> >> So why cant we just set the same IP on each interfaces mac in cobbler? we >> don't really care which one boots, just that the 'right' admin interface is >> saved after we reboot. >> >> >> On Sun, Jan 26, 2014 at 11:03 PM, Andrey Danin <[email protected]> >> wrote: >>> >>> My scenario occurs when eth0 and eth1 are connected to PXE network and a >>> PortFast feature is not enabled on a switch. Eth0 can get a timeout from >>> time to time and if the Cobbler will know a eth0 MAC only, it'll boot it to >>> the bootstrap. >>> >>> >>> >>> On Mon, Jan 27, 2014 at 9:52 AM, Matthew Mosesohn >>> <[email protected]> wrote: >>>> >>>> Andrey, >>>> >>>> This is something a user could always fix himself. Typically you do >>>> DHCP on eth0, then eth1,etc.... and the only case I can think of where >>>> your scenario would occur is if we were trying to provision 50+ nodes >>>> with an underpowered master. But what's more likely is eth0 will start >>>> PXE boot and fail to receive the whole image and reboot and try again. >>>> If you have systems PXE booting on eth0 for bootstrap then eth1 for >>>> provision, then it's a really odd machine and I think we need to >>>> discount that case. >>>> >>>> If this scenario is real and reproducable, we can definitely work >>>> around it and create system records in Cobbler (but not reserve an IP >>>> for them) that correspond to all the other MAC addresses of the >>>> system. We could forward them to a special profile that tells cobbler >>>> to update its records for DNS and the system profile, then reboot it >>>> and provision again. Or at minimum a warning state that we can use to >>>> give feedback to the user that PXE is not working correctly on a >>>> particular node. >>>> >>>> If a user wants to deploy 100 nodes with 6 NICs in each system, a /24 >>>> should be enough, but with our non-optimal logic, it requires a /22 >>>> network to deploy: 100 IPs for bootstrap and 600 IPs for provisioning. >>>> >>>> The solution in place doesn't work and doesn't meet expectations of >>>> real use cases. We should pick an approach that will make the most >>>> people happy. >>>> >>>> -Matthew >>>> >>>> On Mon, Jan 27, 2014 at 9:40 AM, Andrey Danin <[email protected]> >>>> wrote: >>>> > The problem is the Cobbler will force the node to boot to bootstrap if >>>> > he >>>> > doesn't recognize it via known MAC. Here are the boot process >>>> > diagrams. >>>> > The right one: >>>> > # node's embedded PXE software starts a boot-up procedure via eht0 >>>> > # Cobbler knows eth0 MAC and puts to the node a PXE config to >>>> > download and >>>> > run an installation image (CentOS) >>>> > # The node goes to the provisioning state. >>>> > >>>> > The bad one: >>>> > # node runs PXE procedure over eth1 >>>> > # Cobbler doesn't know eth1 MAC and sends to the node a PXE config >>>> > with a >>>> > bootstrap image. >>>> > # the node boots into bootstrap and an installation process fails. >>>> > >>>> > >>>> > On Mon, Jan 27, 2014 at 9:25 AM, Matthew Mosesohn >>>> > <[email protected]> >>>> > wrote: >>>> >> >>>> >> Andrey, >>>> >> >>>> >> We can use the same tactic we use in Ubuntu to ensure that CentOS >>>> >> starts installing on the correct MAC by adding the kernel parameter >>>> >> ks.device=$MACADDR. It will try to DHCP on that interface and >>>> >> eliminate the possibility of having 2 or more NICs on the same subnet >>>> >> as the PXE server. >>>> >> >>>> >> On Mon, Jan 27, 2014 at 9:14 AM, Andrey Danin <[email protected]> >>>> >> wrote: >>>> >> > There is a little chance that a node will boot up via another >>>> >> > interface >>>> >> > and >>>> >> > Cobbler will put it to the bootstrap stage instead of the target >>>> >> > stage. >>>> >> > We >>>> >> > have to provide all MAC addresses of a node to Cobbler but it can't >>>> >> > use >>>> >> > one >>>> >> > IP for all MACs. So we have to allocate a lot of IPs. >>>> >> > >>>> >> > We can improve our mechanism by the following: >>>> >> > * Assign IP to the node's admin interface only. Use it as a >>>> >> > default >>>> >> > behavior but add an UI switch to enable a greedy behavior. >>>> >> > * Run a predeployment DHCP check in order to find all interfaces >>>> >> > which >>>> >> > really can communicate with PXE network and assign IPs to these >>>> >> > interfaces >>>> >> > only. >>>> >> > >>>> >> > >>>> >> > >>>> >> > On Thu, Jan 23, 2014 at 12:29 AM, Ryan Moe <[email protected]> >>>> >> > wrote: >>>> >> >> >>>> >> >> What is the reasoning behind assigning an IP from the admin >>>> >> >> network to >>>> >> >> all >>>> >> >> interfaces [1]? It has caused problems in deployments where IPs >>>> >> >> are >>>> >> >> tightly >>>> >> >> managed and/or machines have a large number of interfaces. >>>> >> >> >>>> >> >> Thanks, >>>> >> >> Ryan >>>> >> >> >>>> >> >> [1] >>>> >> >> >>>> >> >> >>>> >> >> https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/task/helpers.py#L435 >>>> >> >> >>>> >> >> -- >>>> >> >> Mailing list: https://launchpad.net/~fuel-dev >>>> >> >> Post to : [email protected] >>>> >> >> Unsubscribe : https://launchpad.net/~fuel-dev >>>> >> >> More help : https://help.launchpad.net/ListHelp >>>> >> >> >>>> >> > >>>> >> > >>>> >> > >>>> >> > -- >>>> >> > Andrey Danin >>>> >> > [email protected] >>>> >> > skype: gcon.monolake >>>> >> > >>>> >> > -- >>>> >> > Mailing list: https://launchpad.net/~fuel-dev >>>> >> > Post to : [email protected] >>>> >> > Unsubscribe : https://launchpad.net/~fuel-dev >>>> >> > More help : https://help.launchpad.net/ListHelp >>>> >> > >>>> > >>>> > >>>> > >>>> > >>>> > -- >>>> > Andrey Danin >>>> > [email protected] >>>> > skype: gcon.monolake >>> >>> >>> >>> >>> -- >>> Andrey Danin >>> [email protected] >>> skype: gcon.monolake >>> >>> -- >>> Mailing list: https://launchpad.net/~fuel-dev >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~fuel-dev >>> More help : https://help.launchpad.net/ListHelp >>> >> >> >> >> -- >> If google has done it, Google did it right! > > > > > -- > Andrey Danin > [email protected] > skype: gcon.monolake -- Mailing list: https://launchpad.net/~fuel-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~fuel-dev More help : https://help.launchpad.net/ListHelp
