Fwd: netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[Sven-Volker Nowarra]

>> ...
>>
>> $ ./tftpd_rewrite
>> $ doas chgrp _tftpd /tmp/tftpd_rewrite.sock ; doas chmod g+w
/tmp/tftpd_rewrite.sock
>> $ doas tftpd -v -r /tmp/tftpd_rewrite.sock /home/vm
>>
>> $ tftp 127.0.0.1
>> tftp> get /etc/boot.conf
>> Received 38 bytes in 0.0 seconds
>>
>> $ syslogc daemon | tail -n1
>> Jan 29 01:51:49 t440s tftpd[626]: 127.0.0.1: read request for
'/etc/boot.conf'
>> $ cat boot.conf
>> set tty com0
>> boot tftp:/bsd.rd.i386
>
> nice proof of concept :-)
> thx! works well with OpenBSD's tftpd.
> Same logic does not apply to in.tftpd or atftpd. I had copied the OpenBSD
"pxeboot" (amd64 and i386, they differ a bit) to my Linux box. As you can see
in the second line, the requested filename "boot.conf" has no additional IP
address or MAC to filter on:
>
>  in.tftpd[2131]: RRQ from 192.168.88.253 filename /i386/bsd60_i386_pxeboot
>> in.tftpd[2132]: remap: input: /etc/boot.conf
>  in.tftpd[2132]: remap: done
>  in.tftpd[2132]: RRQ from 192.168.88.253 filename /etc/boot.conf
>  in.tftpd[2133]: remap: input: /etc/random.seed
>  in.tftpd[2133]: remap: done
>  in.tftpd[2133]: RRQ from 192.168.88.253 filename /etc/random.seed
>  in.tftpd[2133]: sending NAK (1, File not found) to 192.168.88.253
>  in.tftpd[2134]: remap: input: /bsd
>  in.tftpd[2134]: remap: done
>  in.tftpd[2134]: RRQ from 192.168.88.253 filename /bsd
>  in.tftpd[2134]: sending NAK (1, File not found) to 192.168.88.253
>
> This way, when creating a remapping based on the raw filename, it would
apply to i386 and amd64.
> Anyhow, I will update my doc with the solution for OpenBSD. Already a good
step forwards.
> At the same time I will also update with Anton's remark for a dual server.
>

I am wrong! I have played around with in.tftpd. I could get my OpenBSD systems
to properly netboot i386 and amd64 in the same network segment. Here is all
the configs I used, to achieve OpenBSD clients booting from in.tftpd and it's
"filename remapping":

/etc/hosts:
192.168.88.253 mvbcxfw1.svn.localdomain mvbcxfw1
192.168.88.254 mvbcxfw2.svn.localdomain mvbcxfw2

/etc/tftp.remap:
r /etc/boot.conf /etc/boot.conf.\i

This translates into: for all requests [r] that come with a filename
“/etc/boot.conf” replace it with the filename “/etc/boot.conf” and
attach a dot and the IP address of the tftpd requesting client [\i]. (man
in.tftpd)

/etc/dhcpd.conf
...
group {
  filename "/pxeboot";
  next-server 192.168.88.8;
  host mvbcxfw1 {
option host-name "mvbcxfw1";
hardware ethernet 00:47:11:12:01:27;
fixed-address mvbcxfw1;
  }
  host mvbcxfw2 {
option host-name "mvbcxfw2";
hardware ethernet 00:47:11:12:01:28;
fixed-address mvbcxfw2;
  }
}

I created two files in the /srv/tftpboot/etc/ directory:

/srv/tftpboot/etc/boot.conf.192.168.88.253:
echo #
echo # in.tftpd greetings to client 192.168.88.253
echo #
boot /i386/bsd60_i386_bsd.rd

/srv/tftpboot/etc/boot.conf.192.168.88.254:
echo #
echo # in.tftpd greetings to client 192.168.88.254
echo #
boot /amd64/bsd60_amd64_bsd.rd

I then start the xinetd, with the in.tftpd activated. The config file
/etc/xinetd.d/tftp:
service tftp
{
socket_type = dgram
protocol = udp
wait = yes
user = root
group = tftp
server = /usr/sbin/in.tftpd
server_args = -m /etc/tftpd.remap -vvv -s /srv/tftpboot
}

Alternativly, one can also do a manual start:
/usr/sbin/in.tftpd -l -m /etc/tftpd.remap -vvv -s /srv/tftpboot

I boot my first i386 client, and see this in journalctl -f
/usr/sbin/in.tftpd:
  Jan 30 02:04:33 mleap42ix in.tftpd[4125]: remap: input: /pxeboot
  Jan 30 02:04:33 mleap42ix in.tftpd[4125]: remap: done
  Jan 30 02:04:33 mleap42ix in.tftpd[4125]: RRQ from 192.168.88.253 filename
/pxeboot
  Jan 30 02:04:33 mleap42ix in.tftpd[4126]: remap: input: /etc/boot.conf
  Jan 30 02:04:33 mleap42ix in.tftpd[4126]: remap: rule 0: rewrite:
/etc/boot.conf.192.168.88.253
  Jan 30 02:04:33 mleap42ix in.tftpd[4126]: remap: done
  Jan 30 02:04:33 mleap42ix in.tftpd[4126]: RRQ from 192.168.88.253 filename
/etc/boot.conf remapped to /etc/boot.conf.192.168.88.253
...
  Jan 30 02:04:33 mleap42ix in.tftpd[4128]: remap: input:
/i386/bsd60_i386_bsd.rd
  Jan 30 02:04:33 mleap42ix in.tftpd[4128]: remap: done
  Jan 30 02:04:33 mleap42ix in.tftpd[4128]: RRQ from 192.168.88.253 filename
/i386/bsd60_i386_bsd.rd

Line number 5 and line number 7 show the change. So with this one can define a
per IP address config file, and define in the boot.conf . whichever kernel
we want to boot. The equivalent result for the amd64 client would show the
remapping, and boot the amd64 kernel.

rgds,
Volker

Re: netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[Sven-Volker Nowarra]

> Am 29.01.2017 um 14:23 schrieb Jiri B :
>
>>> Isn't better to use rewrite/file remapping instead of hacking pxeboot?
>>> If an i386 machine would request /etc/boot.conf via tftp you could
rewrite
>>> it to (based on fact you know that that machine is i386 - during
provisioning)
>>> /etc/i386/boot.conf. For the client I suppose it would still think it
gets
>>> /etc/boot.conf.
>
> A POC...
>
> j.
>
> ~~~
> #!/usr/bin/perl -w
>
> use IO::Socket::UNIX;
>
> my $socket_path = '/tmp/tftpd_rewrite.sock';
> unlink $socket_path if -e $socket_path;
> my $socket = IO::Socket::UNIX->new(
>Local  => $socket_path,
>Type   => SOCK_STREAM,
>Listen => SOMAXCONN,
> );
> die "Can't create socket: $!" unless $socket;
>
> while (1) {
>next unless my $connection = $socket->accept;
>$connection->autoflush(1);
>while (my $line = <$connection>) {
>chomp($line);
># XXX
># conditionals here
>if ($line =~ /^127.0.0.1 read \/etc\/boot.conf$/) {
>print $connection "/etc/boot.conf.i386\n";
>} elsif ($line =~ /^\S+ read \/etc\/boot.conf$/) {
>print $connection "/etc/boot.conf\n";
>}
>}
> }
> ~~~
>
> $ ./tftpd_rewrite
> $ doas chgrp _tftpd /tmp/tftpd_rewrite.sock ; doas chmod g+w
/tmp/tftpd_rewrite.sock
> $ doas tftpd -v -r /tmp/tftpd_rewrite.sock /home/vm
>
> $ tftp 127.0.0.1
> tftp> get /etc/boot.conf
> Received 38 bytes in 0.0 seconds
>
> $ syslogc daemon | tail -n1
> Jan 29 01:51:49 t440s tftpd[626]: 127.0.0.1: read request for
'/etc/boot.conf'
> $ cat boot.conf
> set tty com0
> boot tftp:/bsd.rd.i386

nice proof of concept :-)
thx! works well with OpenBSD's tftpd.
Same logic does not apply to in.tftpd or atftpd. I had copied the OpenBSD
"pxeboot" (amd64 and i386, they differ a bit) to my Linux box. As you can see
in the second line, the requested filename "boot.conf" has no additional IP
address or MAC to filter on:

  in.tftpd[2131]: RRQ from 192.168.88.253 filename /i386/bsd60_i386_pxeboot
> in.tftpd[2132]: remap: input: /etc/boot.conf
  in.tftpd[2132]: remap: done
  in.tftpd[2132]: RRQ from 192.168.88.253 filename /etc/boot.conf
  in.tftpd[2133]: remap: input: /etc/random.seed
  in.tftpd[2133]: remap: done
  in.tftpd[2133]: RRQ from 192.168.88.253 filename /etc/random.seed
  in.tftpd[2133]: sending NAK (1, File not found) to 192.168.88.253
  in.tftpd[2134]: remap: input: /bsd
  in.tftpd[2134]: remap: done
  in.tftpd[2134]: RRQ from 192.168.88.253 filename /bsd
  in.tftpd[2134]: sending NAK (1, File not found) to 192.168.88.253

This way, when creating a remapping based on the raw filename, it would apply
to i386 and amd64.
Anyhow, I will update my doc with the solution for OpenBSD. Already a good
step forwards.
At the same time I will also update with Anton's remark for a dual server.

thx, rgds,
Volker

Re: netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[Jiri B]

> > Isn't better to use rewrite/file remapping instead of hacking pxeboot?
> > If an i386 machine would request /etc/boot.conf via tftp you could rewrite
> > it to (based on fact you know that that machine is i386 - during 
> > provisioning)
> > /etc/i386/boot.conf. For the client I suppose it would still think it gets
> > /etc/boot.conf.

A POC...

j.

~~~
#!/usr/bin/perl -w

use IO::Socket::UNIX;

my $socket_path = '/tmp/tftpd_rewrite.sock';
unlink $socket_path if -e $socket_path;
my $socket = IO::Socket::UNIX->new(
Local  => $socket_path,
Type   => SOCK_STREAM,
Listen => SOMAXCONN,
);
die "Can't create socket: $!" unless $socket;

while (1) {
next unless my $connection = $socket->accept;
$connection->autoflush(1);
while (my $line = <$connection>) {
chomp($line);
# XXX
# conditionals here
if ($line =~ /^127.0.0.1 read \/etc\/boot.conf$/) {
print $connection "/etc/boot.conf.i386\n";
} elsif ($line =~ /^\S+ read \/etc\/boot.conf$/) {
print $connection "/etc/boot.conf\n";
}
}
}
~~~

$ ./tftpd_rewrite
$ doas chgrp _tftpd /tmp/tftpd_rewrite.sock ; doas chmod g+w 
/tmp/tftpd_rewrite.sock
$ doas tftpd -v -r /tmp/tftpd_rewrite.sock /home/vm

$ tftp 127.0.0.1 
tftp> get /etc/boot.conf
Received 38 bytes in 0.0 seconds

$ syslogc daemon | tail -n1
Jan 29 01:51:49 t440s tftpd[626]: 127.0.0.1: read request for '/etc/boot.conf'
$ cat boot.conf
set tty com0
boot tftp:/bsd.rd.i386

Re: netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[Jiri B]

On Sun, Jan 29, 2017 at 01:17:48AM +0200, li...@wrant.com wrote:
> Sample excerpts from host specific DHCP server config, for i386 and amd64:
> 
>   next-server 10.0.0.32;
>   filename "auto_upgrade";
> 
>   next-server 10.0.0.64;
>   filename "auto_upgrade";
> 
> Quoting autoinstall(8) for netbooting:  http://man.openbsd.org/autoinstall 
> 
>   On architectures where the filename statement is used to provide the
>   name of the file to netboot it is necessary to create symbolic links
>   called auto_install and auto_upgrade that point to the expected boot
>   program and to change the value of the filename statement in the
>   dhcpd.conf(5) file to be auto_install or auto_upgrade.
> 
>   # ln -s /tftpboot/i386/pxeboot  /tftpboot/i386/auto_upgrade
>   # ln -s /tftpboot/amd64/pxeboot /tftpboot/amd64/auto_upgrade
> 
> Needless to say, you need to populate the /tftpboot/{i386,amd64} locations
> with the system installation packages from the local mirror / compilation.
> 
> It is also quite easy to combine both the DHCP server and two instances of
> tftpd(8), started independently listening on 2 IP address aliases, serving
> pxeboot(8) respectively for i386 and amd64 systems stand alone each other.
> 
> See rcctl(8) to run a second copy of a daemon http://man.openbsd.org/rcctl
> 
>   The recommended way to run a second copy of a given daemon for a
>   different purpose is to create a symbolic link to its rc.d(8) control
>   script: 
> 
>   # ln -s /etc/rc.d/tftpd /etc/rc.d/tftpd2
>   # rcctl set tftpd status on
>   # rcctl set tftpd2 status on
>   # rcctl set tftpd flags -4 -l 10.0.0.32 /tftpboot/i386
>   # rcctl set tftpd2 flags -4 -l 10.0.0.64 /tftpboot/amd64
>   # rcctl start tftpd
>   # rcctl start tftpd2

Nice trick to define multiple tftp servers for each x86 architecture :)

Thanks!

j.

Re: netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[lists]

Sat, 28 Jan 2017 00:17:40 +0100 Sven-Volker Nowarra 
> I am netbooting many systems, and last recently stepped on the issue, that I
> had an amd64 and an i386 client in the same network. I wanted to boot them
> into a "full" OpenBSD (not ramdisk kernel). That is not possible with the
> default installation, cause pxeboot can not distinguish between these
> Intel/AMD systems.

Hi Sven-Volker, Jiri,

I am also booting over network many systems in the same network, including
i386 and amd64 systems.  It is in fact quite possible to achieve this with
the default installation, with the help of the DHCP next-server statement.

> DHCP server can distinguish by MAC address, but then when
> pxeboot is loaded, the kernel is per default "bsd". This must clash either
> with i386 or amd64 architecture, whatever was dropped into tftpboot direcotry.

Quoting dhcpd.conf(5), please see here:  http://man.openbsd.org/dhcpd.conf

next-server server-name;

  The next-server statement is used to specify the host address of the
  server from which the initial boot file (specified in the filename
  statement) is to be loaded. server-name should be a numeric IP address
  or a domain name. If no next-server parameter applies to a given
  client, the DHCP server's IP address is used.

Sample excerpts from host specific DHCP server config, for i386 and amd64:

next-server 10.0.0.32;
filename "auto_upgrade";

next-server 10.0.0.64;
filename "auto_upgrade";

Quoting autoinstall(8) for netbooting:  http://man.openbsd.org/autoinstall 

  On architectures where the filename statement is used to provide the
  name of the file to netboot it is necessary to create symbolic links
  called auto_install and auto_upgrade that point to the expected boot
  program and to change the value of the filename statement in the
  dhcpd.conf(5) file to be auto_install or auto_upgrade.

# ln -s /tftpboot/i386/pxeboot  /tftpboot/i386/auto_upgrade
# ln -s /tftpboot/amd64/pxeboot /tftpboot/amd64/auto_upgrade

Needless to say, you need to populate the /tftpboot/{i386,amd64} locations
with the system installation packages from the local mirror / compilation.

It is also quite easy to combine both the DHCP server and two instances of
tftpd(8), started independently listening on 2 IP address aliases, serving
pxeboot(8) respectively for i386 and amd64 systems stand alone each other.

See rcctl(8) to run a second copy of a daemon http://man.openbsd.org/rcctl

  The recommended way to run a second copy of a given daemon for a
  different purpose is to create a symbolic link to its rc.d(8) control
  script: 

# ln -s /etc/rc.d/tftpd /etc/rc.d/tftpd2
# rcctl set tftpd status on
# rcctl set tftpd2 status on
# rcctl set tftpd flags -4 -l 10.0.0.32 /tftpboot/i386
# rcctl set tftpd2 flags -4 -l 10.0.0.64 /tftpboot/amd64
# rcctl start tftpd
# rcctl start tftpd2

For the list of the flags for tftpd(8), see:  http://man.openbsd.org/tftpd

When you need to upgrade a system, link the boot.conf to a file with this:

boot tftp:/bsd.rd

When done, link it back the boot.conf file to a file that has the generic:

boot hd0a:/bsd

This way you always net boot the system, then make it auto_upgrade, or run
the already installed system from the first hard disk where the system is.

Of course you may want to adjust these as per your environment and set up.
I would be glad if you could share an idea how to toggle boot.conf better.

I would be further much more grateful if there is in the near future a way
to provide automatic response file for local install for the serve system.

> So I went through some older mailing list entries, adapted them, and updated
> my meanwhile extensive netboot document. I updated this into a PDF, covering
> many, many details (now ~50 pages). Wanted to give something back to the
> community. The PDF is currently located here:
> http://nowarra.ch/Volker/netboot_OpenBSD/170127_netbooting_OpenBSD60.pdf

Nice read, thank you.  Hopefully you could further extend it, for multiple
concurrent network boot environments for simpler setup w/o custom patches.

Kind regards,
Anton

Re: netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[Sven-Volker Nowarra]

> Am 28.01.2017 um 14:56 schrieb Jiri B :
>
> On Sat, Jan 28, 2017 at 12:17:40AM +0100, Sven-Volker Nowarra wrote:
>> I am netbooting many systems, and last recently stepped on the issue, that
I
>> had an amd64 and an i386 client in the same network. I wanted to boot them
>> into a "full" OpenBSD (not ramdisk kernel). That is not possible with the
>> default installation, cause pxeboot can not distinguish between these
>> Intel/AMD systems. DHCP server can distinguish by MAC address, but then
when
>> pxeboot is loaded, the kernel is per default "bsd". This must clash either
>> with i386 or amd64 architecture, whatever was dropped into tftpboot
direcotry.
>> So I went through some older mailing list entries, adapted them, and
updated
>> my meanwhile extensive netboot document. I updated this into a PDF,
covering
>> many, many details (now ~50 pages). Wanted to give something back to the
>> community. The PDF is currently located here:
>> http://nowarra.ch/Volker/netboot_OpenBSD/170127_netbooting_OpenBSD60.pdf
>>
>
> Thanks, interesting document.
>
> Isn't better to use rewrite/file remapping instead of hacking pxeboot?
> If an i386 machine would request /etc/boot.conf via tftp you could rewrite
> it to (based on fact you know that that machine is i386 - during
provisioning)
> /etc/i386/boot.conf. For the client I suppose it would still think it gets
> /etc/boot.conf.
>
> j.

If this works, I could get rid of recompiling pxeboot everytime a new release
comes out. Well, sometimes pxeboot also supports "older" OpenBSDs, but that is
another topic.

I understand, the tftp server has a "root dir" for the client specified. In
the dhcpd.conf I declare per client a MAC address and its filename (usually
"/pxeboot"). The i386 pxeboot manual says: "pxeboot boot program will look for
an /etc/boot.conf configuration file on the TFTP server." I didn't find a
reference to a different sub structure...

Anyway, I tried a structure like you proposed, but pxeboot didn't find the
boot.conf, and didn't even show the echo lines from this file (so useless to
play with bsd location). This was my setup:

location of boot.conf:
  /tftpboot/etc/i386/boot.conf

$ cat /tftpboot/etc/i386/boot.conf
echo ###
echo ### hello from tftpd@192.168.88.12, with /etc/i386/boot.conf ###
echo ###
boot bsd.rd

$ cat /etc/dhcpd.conf | grep filename
   filename "/pxeboot";

I also tried to play with the dhcpd.conf settings, by using a different subdir
for pxeboot, but I didn't get the system to find "his" boot.conf in the i386
directory.

Re: netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[Jiri B]

On Sat, Jan 28, 2017 at 06:41:34PM +0100, Sven-Volker Nowarra wrote:
> > Isn't better to use rewrite/file remapping instead of hacking pxeboot?
> > If an i386 machine would request /etc/boot.conf via tftp you could rewrite
> > it to (based on fact you know that that machine is i386 - during 
> > provisioning)
> > /etc/i386/boot.conf. For the client I suppose it would still think it gets
> > /etc/boot.conf.

> If this works, I could get rid of recompiling pxeboot everytime a
> new release comes out. Well, sometimes pxeboot also supports "older"
> OpenBSDs, but that is another topic.
> 

> I understand, the tftp server has a "root dir" for the client
> specified. In the dhcpd.conf I declare per client a MAC address and
> its filename (usually "/pxeboot"). The i386 pxeboot manual says:
> "pxeboot boot program will look for an /etc/boot.conf configuration
> file on the TFTP server." I didn't find a reference to a different
> sub structure...
> 
> Anyway, I tried a structure like you proposed, but pxeboot didn't
> find the boot.conf, and didn't even show the echo lines from this
> file (so useless to play with bsd location). This was my setup:
>
> location of boot.conf:
>   /tftpboot/etc/i386/boot.conf
> 
> $ cat /tftpboot/etc/i386/boot.conf
> echo ### 
> echo ### hello from tftpd@192.168.88.12, with /etc/i386/boot.conf ###
> echo ### 
> boot bsd.rd 
> 
> $ cat /etc/dhcpd.conf | grep filename
>filename "/pxeboot";
> 
> I also tried to play with the dhcpd.conf settings, by using a different 
> subdir for pxeboot, but I didn't get the system to find "his" boot.conf in 
> the i386 directory. 

It seems you missed part about tftpd rewrite/file remapping. The client will 
still
request /etc/boot.conf but you fake it via rewrite script.

man tftpd

 -r socket
 Issue filename rewrite requests to the specified UNIX domain
 socket.  tftpd will write lines in the format "IP OP filename",
 terminated by a newline, where IP is the client's IP address, and
 OP is one of "read" or "write".  tftpd expects replies in the
 format "filename" terminated by a newline.  All rewrite requests
 from the daemon must be answered (even if it is with the original
 filename) before the TFTP request will continue.  By default
 tftpd does not use filename rewriting.

j.

Re: netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[Jiri B]

On Sat, Jan 28, 2017 at 12:17:40AM +0100, Sven-Volker Nowarra wrote:
> I am netbooting many systems, and last recently stepped on the issue, that I
> had an amd64 and an i386 client in the same network. I wanted to boot them
> into a "full" OpenBSD (not ramdisk kernel). That is not possible with the
> default installation, cause pxeboot can not distinguish between these
> Intel/AMD systems. DHCP server can distinguish by MAC address, but then when
> pxeboot is loaded, the kernel is per default "bsd". This must clash either
> with i386 or amd64 architecture, whatever was dropped into tftpboot direcotry.
> So I went through some older mailing list entries, adapted them, and updated
> my meanwhile extensive netboot document. I updated this into a PDF, covering
> many, many details (now ~50 pages). Wanted to give something back to the
> community. The PDF is currently located here:
> http://nowarra.ch/Volker/netboot_OpenBSD/170127_netbooting_OpenBSD60.pdf
> 

Thanks, interesting document.

Isn't better to use rewrite/file remapping instead of hacking pxeboot?
If an i386 machine would request /etc/boot.conf via tftp you could rewrite
it to (based on fact you know that that machine is i386 - during provisioning)
/etc/i386/boot.conf. For the client I suppose it would still think it gets
/etc/boot.conf.

j.

netbooting OpenBSD (6.0) i386 and amd64 clients from one server

[Sven-Volker Nowarra]

I am netbooting many systems, and last recently stepped on the issue, that I
had an amd64 and an i386 client in the same network. I wanted to boot them
into a "full" OpenBSD (not ramdisk kernel). That is not possible with the
default installation, cause pxeboot can not distinguish between these
Intel/AMD systems. DHCP server can distinguish by MAC address, but then when
pxeboot is loaded, the kernel is per default "bsd". This must clash either
with i386 or amd64 architecture, whatever was dropped into tftpboot direcotry.
So I went through some older mailing list entries, adapted them, and updated
my meanwhile extensive netboot document. I updated this into a PDF, covering
many, many details (now ~50 pages). Wanted to give something back to the
community. The PDF is currently located here:
http://nowarra.ch/Volker/netboot_OpenBSD/170127_netbooting_OpenBSD60.pdf