Bug#1031643: Bug#1034062: netcfg prompts for hostname even though preseeded, preventing unattended installation

[Cyril Brulebois]

Oliver Freyermuth  (2023-04-09):
> Likely, that's indeed since stuff has already happened at that point.
> It seems that the newly added "if" worked as expected, so it must have
> been "(none)" at the time of checking, and only changed to "?"
> afterwards.

I've just verified that part, and I'm now quite convinced that checking
for “(none)” is fine:

 - right at the beginning, on the language selection screen, one can “go
   back” and get a shell. There, `uname -a` returns:

   Linux (none) 6.1.0-3-amd64 […]

 - once network discovery has happened, including DHCP, and once the
   hostname prompt is presented, `uname -a` returns:

   Linux 6.1.0-3-amd64 […]

So there was definitely some hostname set at the very beginning, on the
kernel side, and that can be used by env2debconf via hostname without
passing any options, which leverages busybox's safe_gethostname() which
taps into UTS information, which is set an contains “(none)”.

While I'm not sure *what* happens, later on it seems the hostname part
is unset, which leads safe_gethostname() to fail and extract the
hostname, and which ends up returning “?” instead.


If there was any doubt regarding stability, include/linux/uts.h
(linux.git) is almost all dating back all the way to the mother git
commit (2005), with UTS_NODENAME being an exception (2011):

bd5dc17be87b3 (Josh Triplett  2011-06-15 15:08:28 -0700 13) #define 
UTS_NODENAME CONFIG_DEFAULT_HOSTNAME /* set by sethostname() */

See 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bd5dc17be87b3a3073d50b23802647db3ae3fa8e

And the kernel packaging started setting this option back in 2015:

commit 8f0f730e419302af8dc8a6c043363ba300e9fac1
Author: Ben Hutchings 
Date:   Wed Jul 29 02:29:09 2015 +

Explicitly set various config symbols to defaults

svn path=/dists/trunk/linux/; revision=22870

Seeing how the issue was brought up to my attention by one of the kernel
maintainer in Debian, I hope this default value won't change or we'll be
told if that happens (hint hint wink wink)!


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1031643: Bug#1034062: netcfg prompts for hostname even though preseeded, preventing unattended installation

[Oliver Freyermuth]

Hello Cyril,

Am 08.04.23 um 23:50 schrieb Cyril Brulebois:

Oliver Freyermuth  (2023-04-08):

Interestingky, without the "hostname=" parameter, running hostname on
a tty (while the question is shown) echoes:

   ~ # hostname
   ?


I found that part slightly strange. From earlier on IRC:

  fun how we get '(none)' by default and '?' instead with -s.
  ('?' comes from safe_gethostname depending on uts.nodename[0])

so I'm not exactly sure why you're getting '?' by default instead of
'(none)'. Maybe that's once you've reached the network step and stuff
has happened? My observations were right after setting a keymap,
switching to a VT.

For context, I was initially wondering which options were supported by
busybox's hostname, hence my looking into this. (Wasn't entirely sure
how safe / future-proof hardcoding “(none)” would be; still unclear at
this point, but I haven't spent much time on this.)


Likely, that's indeed since stuff has already happened at that point. It seems that the 
newly added "if" worked as expected,
so it must have been "(none)" at the time of checking, and only changed to "?" 
afterwards.


However, that "question mark hostname" is also shown when doing this
with Bullseye, so that seems to be expected behaviour.


That part's reassuring.


Indeed :-).

Cheers,
Oliver

Bug#1031643: Bug#1034062: netcfg prompts for hostname even though preseeded, preventing unattended installation

[Cyril Brulebois]

Oliver Freyermuth  (2023-04-08):
> I can confirm that the question is not shown anymore, and installation
> proceeds unattendedly, so the patched versions test fine on my end
> :-).
> 
> During installation, when switching to a tty and running "hostname
> -f", I see the FQDN assigned via the hostname parameter, as expected.

Great, thanks.

> Then of course we also need to test what happens without the
> "hostname=" paraemter on the kernel commandline. I removed it,
> re-tried, and the question was shown again, so that also seems to work
> as expected.
> 
> Interestingky, without the "hostname=" parameter, running hostname on
> a tty (while the question is shown) echoes:
> 
>   ~ # hostname
>   ?

I found that part slightly strange. From earlier on IRC:

 fun how we get '(none)' by default and '?' instead with -s.
 ('?' comes from safe_gethostname depending on uts.nodename[0])

so I'm not exactly sure why you're getting '?' by default instead of
'(none)'. Maybe that's once you've reached the network step and stuff
has happened? My observations were right after setting a keymap,
switching to a VT.

For context, I was initially wondering which options were supported by
busybox's hostname, hence my looking into this. (Wasn't entirely sure
how safe / future-proof hardcoding “(none)” would be; still unclear at
this point, but I haven't spent much time on this.)

> However, that "question mark hostname" is also shown when doing this
> with Bullseye, so that seems to be expected behaviour.

That part's reassuring.

> Seeing the unintrusiveness of the patch, and taking into account that
> adapting existing deployment software can be avoided (there's
> certainly much more affected tooling out there), that feels like a
> good approach to me, too.

Clearly! Initially I was worried about stashing a special case into the
labyrinth (that the parsing code is) that's below, I didn't think of
actually checking the effects of the linux's having consumed that
parameter would have…


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1034062: netcfg prompts for hostname even though preseeded, preventing unattended installation

[Oliver Freyermuth]

Hallo Cyril,

Am 08.04.23 um 14:53 schrieb Cyril Brulebois:

Oliver Freyermuth  (2023-04-08):

I concur that the question is where to go from here — my real-world
use case is using Foreman [0] for machine deployment, which currently
passes "hostname" exclusively in its preseed/PXE templates. If the
decision is to drop support for the hostname alias, deployment
software like this one will of course need to be adapted (considering
the alternatives, that is certainly a viable option).


Thanks for bringing this topic up. I'm not sure whether that triggered
Andi's looking into it, but there's a patch available in #1031643, and
netboot(-gtk) build artifacts with patched preseed binaries available
for a limited time at:
   https://people.debian.org/~kibi/bug-1031643/

That's entirely untested (by me), Salvatore might test that later on.

I can build an amd64 netinst image if that's easier for you to test and
confirm with.


thanks!

The patched preseed binaries are in fact ideal for testing on my end: Foreman 
usually downloads kernel and initrd from the upstream source,
and takes care of all the parts which may be different for the local 
environment (PXE/bootloader/Preseed/Kickstart/Autoyast...) itself.
So I just replaced the linux and initrd.gz which Foreman downloaded with the 
linux and initrd.gz from the patched source,
and re-tried, with no other changes to Foreman, i.e. it was still using the 
hostname alias only on the kernel command line.

I can confirm that the question is not shown anymore, and installation proceeds 
unattendedly, so the patched versions test fine on my end :-).
During installation, when switching to a tty and running "hostname -f", I see 
the FQDN assigned via the hostname parameter, as expected.

Then of course we also need to test what happens without the "hostname=" 
paraemter on the kernel commandline.
I removed it, re-tried, and the question was shown again, so that also seems to 
work as expected.

Interestingky, without the "hostname=" parameter, running hostname on a tty 
(while the question is shown) echoes:

  ~ # hostname
  ?

However, that "question mark hostname" is also shown when doing this with 
Bullseye, so that seems to be expected behaviour.


I'll hold back on upstreaming this change to Foreman until a decision
on how to proceed with #1031643 is made.


Thanks again for mentioning it, that definitely shifted the balance (at
least for me) in the “let's try and restore the feature” direction.


Seeing the unintrusiveness of the patch, and taking into account that adapting 
existing deployment software can be avoided (there's certainly much more 
affected tooling out there),
that feels like a good approach to me, too.

Cheers,
Oliver

Bug#1034062: netcfg prompts for hostname even though preseeded, preventing unattended installation

[Cyril Brulebois]

Hallo wieder,

Oliver Freyermuth  (2023-04-08):
> indeed, that seems to be the case — I can confirm using the full
> netcfg/get_hostname name on the kernel command line prevents the
> question from being shown, and preseeding proceeds unattendedly again.
> :-)

Thanks for confirming.

> That easily explains why I did not find any code changes in netcfg
> which could explain this change in behaviour, since the reason is due
> to a kernel behavipur change.

The behavorial change was fun to track down, yes…

> So this can probably be closed as a duplicate of #1031643.
> 
> I concur that the question is where to go from here — my real-world
> use case is using Foreman [0] for machine deployment, which currently
> passes "hostname" exclusively in its preseed/PXE templates. If the
> decision is to drop support for the hostname alias, deployment
> software like this one will of course need to be adapted (considering
> the alternatives, that is certainly a viable option).

Thanks for bringing this topic up. I'm not sure whether that triggered
Andi's looking into it, but there's a patch available in #1031643, and
netboot(-gtk) build artifacts with patched preseed binaries available
for a limited time at:
  https://people.debian.org/~kibi/bug-1031643/

That's entirely untested (by me), Salvatore might test that later on.

I can build an amd64 netinst image if that's easier for you to test and
confirm with.

> I'll hold back on upstreaming this change to Foreman until a decision
> on how to proceed with #1031643 is made.

Thanks again for mentioning it, that definitely shifted the balance (at
least for me) in the “let's try and restore the feature” direction.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1034062: netcfg prompts for hostname even though preseeded, preventing unattended installation

[Oliver Freyermuth]

Hello Cyril,

Am 07.04.23 um 23:23 schrieb Cyril Brulebois:

Oliver Freyermuth  (2023-04-07):

Potentially relevant kernel commandline parameters:
netcfg/disable_dhcp=false ethdetect/prompt_missing_firmware=false 
hw-detect/load_firmware=false locale=en_US.UTF-8 hostname=laptop.example.com 
domain=example.com


This is likely #1031643, try using the full netcfg/get_hostname name
instead of the hostname alias on the kernel command line?


indeed, that seems to be the case — I can confirm using the full 
netcfg/get_hostname name on the kernel command line prevents the question from 
being shown,
and preseeding proceeds unattendedly again. :-)
That easily explains why I did not find any code changes in netcfg which could 
explain this change in behaviour, since the reason is due to a kernel behavipur 
change.

So this can probably be closed as a duplicate of #1031643.

I concur that the question is where to go from here — my real-world use case is using 
Foreman [0] for machine deployment, which currently passes "hostname" 
exclusively in its preseed/PXE templates.
If the decision is to drop support for the hostname alias, deployment software 
like this one will of course need to be adapted (considering the alternatives, 
that is certainly a viable option).
I'll hold back on upstreaming this change to Foreman until a decision on how to 
proceed with #1031643 is made.

Thanks a lot and cheers,
Oliver

[0] https://theforeman.org/

Bug#1034062: netcfg prompts for hostname even though preseeded, preventing unattended installation

[Cyril Brulebois]

Hallo Oliver,

Oliver Freyermuth  (2023-04-07):
> With both the Bookworm alpha2 and rc1 netinstaller, I am prompted to
> confirm the preseeded hostname (also sent via DHCP), preventing
> unattended installation.  The very same preseed file did not show any
> such prompt neither in Buster nor Bullseye.
> 
> The prompt is pre-filled with the correct hostname (not the FQDN, just
> the hostname part), but of course this prevents unattended preseed
> installation.
> 
> Relevant logs follow, hostname and FQDN modified to examples.
> 
> Potentially relevant kernel commandline parameters:
> netcfg/disable_dhcp=false ethdetect/prompt_missing_firmware=false 
> hw-detect/load_firmware=false locale=en_US.UTF-8 hostname=laptop.example.com 
> domain=example.com

This is likely #1031643, try using the full netcfg/get_hostname name
instead of the hostname alias on the kernel command line?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1034062: netcfg prompts for hostname even though preseeded, preventing unattended installation

[Oliver Freyermuth]

Package: netcfg
Version: 1.184

With both the Bookworm alpha2 and rc1 netinstaller, I am prompted to confirm 
the preseeded hostname (also sent via DHCP), preventing unattended installation.
The very same preseed file did not show any such prompt neither in Buster nor 
Bullseye.

The prompt is pre-filled with the correct hostname (not the FQDN, just the 
hostname part), but of course this prevents unattended preseed installation.

Relevant logs follow, hostname and FQDN modified to examples.

Potentially relevant kernel commandline parameters:
netcfg/disable_dhcp=false ethdetect/prompt_missing_firmware=false 
hw-detect/load_firmware=false locale=en_US.UTF-8 hostname=laptop.example.com 
domain=example.com

Potentially relevant part of preseed file:
d-i ethdetect/prompt_missing_firmware boolean false
d-i netcfg/choose_interface select auto
d-i netcfg/get_hostname string laptop.exmaple.com
d-i netcfg/get_domain string example.com
d-i netcfg/wireless_wep string
d-i hw-detect/load_firmware boolean true


Relevant part of installer syslog:
Apr  6 16:56:31 netcfg[1051]: DEBUG: State is now 1
Apr  6 16:56:31 netcfg[1051]: DEBUG: State is now 2
Apr  6 16:56:31 netcfg[1051]: DEBUG: State is now 5
Apr  6 16:56:31 netcfg[1051]: INFO: DHCP hostname: "laptop.example.com"
Apr  6 16:56:31 netcfg[1051]: DEBUG: laptop.example.com is a valid FQDN
Apr  6 16:56:31 netcfg[1051]: DEBUG: We have a real FQDN
Apr  6 16:56:31 netcfg[1051]: DEBUG: Preseeding domain from global: example.com
< at this point, the prompt for the hostname is shown, pre-filled with "laptop" 
>