Re: Status and open questions for debian-installer with backports support

[Steve McIntyre]

On Mon, Aug 13, 2018 at 01:44:48PM +0200, Julien Cristau wrote:
>On 08/13/2018 01:57 AM, Steve McIntyre wrote:
>> On Thu, Aug 02, 2018 at 10:56:59AM +0200, Cyril Brulebois wrote:
>>> => Question: should we restrict architectures we build images for?
>> 
>> Probably. I'm only thinking x86, arm64 and maybe armhf. Wait for
>> people to ask for others and add on a case-by-case basis.
>> 
>I'd even suggest s/x86/amd64/
>
>New 32-bit x86 hardware shouldn't be a huge use case nowadays.

That's a fair point, yes.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"You can't barbecue lettuce!" -- Ellie Crane

Re: Status and open questions for debian-installer with backports support

[Julien Cristau]

On 08/13/2018 01:57 AM, Steve McIntyre wrote:
> On Thu, Aug 02, 2018 at 10:56:59AM +0200, Cyril Brulebois wrote:
>> => Question: should we restrict architectures we build images for?
> 
> Probably. I'm only thinking x86, arm64 and maybe armhf. Wait for
> people to ask for others and add on a case-by-case basis.
> 
I'd even suggest s/x86/amd64/

New 32-bit x86 hardware shouldn't be a huge use case nowadays.

Cheers,
Julien

Re: Status and open questions for debian-installer with backports support

[Cyril Brulebois]

John Paul Adrian Glaubitz  (2018-08-13):
> Can we also add the patches for choose-mirror and net-retriever to support
> Debian Ports at some point? We have lots of users that are confused that
> debian-installer does not provide a mirror list from where they can choose
> but rather have to type in the mirror information manually.
> 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879145 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879130
> 
> There is also:
> 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879147 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879151

I'm aware of these issues, but it seems to make sense to concentrate
on release architectures, which have a huge user base, and which need
improved hardware support.

There are enough moving parts as it is to avoid adding ports-specific
issues to the mix (as already mentioned in my reply to one of those
bug reports). Please let us figure out how to deal with release archs
and backports first.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Status and open questions for debian-installer with backports support

[John Paul Adrian Glaubitz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 08/02/2018 10:56 AM, Cyril Brulebois wrote:
> It's been a while since my first proof of concept (demonstrated at the 2014 
> mini-DebConf in Paris), but I think I've managed to reach a point where I'm
> rather content and ready to get bits and pieces merged where they belong. 
> That doesn't mean I have everything figured out, that's why I'm reaching out 
> to
> both the installer & images team!

Can we also add the patches for choose-mirror and net-retriever to support
Debian Ports at some point? We have lots of users that are confused that
debian-installer does not provide a mirror list from where they can choose
but rather have to type in the mirror information manually.

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879145 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879130

There is also:

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879147 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879151

Thanks,
Adrian

- -- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEYv+KdYTgKVaVRgAGdCY7N/W1+RMFAltxXXoACgkQdCY7N/W1
+ROgbg//dxJC1qX7Jfa3qzbqnjQ7pUh68/1OwEF6xwQj4NrkFgDchVVTASvzEwul
goKx+tELK3LVfy+ncchRrD2vImT1aY4iBpywCLSi/T9bE+z/15SOEZlb1QZw2bxp
1vw4HW13HzOBrR7TK+lu7QeAcWMKanWMO+T53IT47/NeQXhbtyCeJ1AP3+mPwqjR
GmR4rsnFX/kjYfIXDLEI9ByMbd3NQ/IQNEbibxxIYiHrW3Tat4xBfw0vwMplySdt
PDo11uqrrbbuhBXTVnxEilqC2Td+Yr9uglEZ6hJ4+RPDBLcCInnLgw5GCvfQq80v
6igQIQp+nJkZPohFawffX6grEzmxvuOSUZSfuyjAioKuBWevBggV2W4/LiplALA1
pUTvYW6RvDjtaCfpVrCH1uUcQMCvZorwl35R9UXRxKHwcCq/nWHNi5ym0g7bfdKK
gX8LkdjtnYiT+7pJDK3tw1dCMzYsys9BUmBprkjCtfvHkfQSvyJKVng+xOwCvfX5
KKj9ohCYjTBcpe9cHbu0ZlvR8moW7OlpWcS5HyKK1PR37BNWfPjV+O6KX+ovUnIT
cUjyGZMYoxfeusr/MY9OF02iSLaogtGVFJySlAppJoJ4ERCbeQNDuqZ71zWMD1QM
rCR3qlzbq1umVxjQE7nlTm2eTvOT92R7mR2ijKDz4kVMFreX/rM=
=8M/T
-END PGP SIGNATURE-

Re: Status and open questions for debian-installer with backports support

[Steve McIntyre]

Apologies for taking a while to get back to you here... :-/

Also adding a CC to the -kernel list.

On Thu, Aug 02, 2018 at 10:56:59AM +0200, Cyril Brulebois wrote:
>
>This is a long read… I've highlighted “to do”-like items with an arrow
>(=>).

ACK!

>debian-installer
>
>
>First things first, we needed a debian-installer build that knew about a
>backported linux kernel, so I hacked my way in debian-installer.git; it
>wasn't too hard since we can count on apt to resolve dependencies, and
>after all, we only needed the kernel-image udeb and its modules.
>
>The latest branch is stretch-backports-v2, which combines both a number
>of commits that are aimed at the master branch, and a couple others that
>want to only be used in a stretch-backports branch:
>
>  
> https://salsa.debian.org/installer-team/debian-installer/commits/stretch-backports-v2

All looks sane enough...

>net-retriever
>-
>
>As usual, I've tested the resulting installer with the netboot(-gtk)
>image and of course it failed to load additional udebs. These netboot*
>images are aimed at booting from the network, and they also need to
>fetch additional udebs from a mirror. Since net-retriever only knows
>about a single source of udebs, it needed to be told about multiple
>sources (stretch and stretch-backports) and also how to merge entries.
>I've detailed my choices in this commit in particular:
>
>  
> https://salsa.debian.org/installer-team/net-retriever/commit/09535b0613a85e02d511b92b60683b20405b6d42
>
>For more details, see the whole stretch-backports-v2 branch for
>net-retriever:
>
>  
> https://salsa.debian.org/installer-team/net-retriever/commits/stretch-backports-v2
>
>Now, it doesn't matter too much, since the stretch-backports repository
>will evolve over time, and udebs for the specific kernel used during the
>d-i build will disappear. I think this is too much of a volatile (no
>pun intended) target to support. It's still a very nice way to test d-i
>without having to build an ISO with debian-cd, so I'd like to merge
>these changes in net-retriever anyway.

ACK. Agreed with the "inefficient" comments here, but if it works...

>base-installer
>--
>
>Second, it makes quite some sense to not only run d-i with a newer
>kernel, but to also install it, so that users can boot from their brand
>new hardware that wasn't supported by d-i with stable components only.

Of course, yeah.

>The kernel selection and installation is implemented in base-installer
>(see library.sh), and that happens to run before apt-setup. Since I
>didn't want to butcher sources.list manually to install the kernel
>directly at this stage, I've decided to implement installing the latest
>kernel from backports in a finish-install script (also shipped in the
>base-installer package, for consistency), running at the very end of the
>installation process.

Ah, OK. More on this later.

>This only happens when backports support was enabled (this is detected
>through the presence of /etc/udebs-backports-source).
>
>At first, using the linux-image-$arch metapackage looked easy enough but
>of course that works for amd64, but not for i386 (one would need to pick
>686 or 686-pae). I've still pushed a branch with this implementation as
>it can be useful as is:
>
>  
> https://salsa.debian.org/installer-team/base-installer/commits/stretch-backports-v0
>
>We could probably modify library.sh to reuse it, but I'd like to have
>minimal changes (for reasons explained in the very last section). We
>could probably just iterate over all installed linux-image-'*' packages,
>pick the one from the src:linux package, and upgrade it from the
>backports repository.
>
>=> I'll look into that later on.

OK.

>apt-setup
>-
>
>Also only when /etc/udebs-backports-source is present, the backports
>service gets automatically enabled, which makes the later installation
>of linux-image packages possible.
>
>Single commit for this:
>
>  
> https://salsa.debian.org/installer-team/apt-setup/commit/6133bdcd5a7903105ad967ab087f2f12f9d1c59d

Absolutely, yeah.

>debian-cd
>=
>
>As mentioned above, tricks were needed in net-retriever for the netboot*
>images, to make it possible to load linux kernel modules udebs. Those
>tricks aren't needed when generating an ISO image. As a reminder, on the
>d-i side, a cdrom_isolinux image is generated, and the debian-cd tool
>needs to be configured to use that image instead of the official image
>available on the configured mirror.
>
>I didn't make this dynamic yet, but I've modified my local configuration
>to set BACKPORTS=backports-list, to include a number of packages from
>the stretch-backports suite. This file was generating by listing all
>udebs produced by the linux source package (as done in the d-i build
>system):
>
>USE_UDEBS_BACKPORTS_FROM=stretch-backports
>...
>source=linux
>...
>binaries=$(grep-dctrl -s Package -F Source $source 
> 

Re: Status and open questions for debian-installer with backports support

[Cyril Brulebois]

Hi,

Thanks for your comments. I really should have thought about putting
the kernel team in copy but it's been a long write-up…

Ben Hutchings  (2018-08-03):
> This looks rather inefficient.  Maybe not a big deal for the small
> number of udebs though.

That's correct on both count: inefficient but also not a pratical issue.

> (Time to put a more powerful scripting language in the installer?
> Even awk would be a step up!)

I'd rather discuss that separately (too many topics already). The
net-retriever part is mainly going to be used by people preparing the
upload anyway.

> > base-installer
> > --
> [...]
> > We could probably modify library.sh to reuse it, but I'd like to have
> > minimal changes (for reasons explained in the very last section). We
> > could probably just iterate over all installed linux-image-'*' packages,
> > pick the one from the src:linux package, and upgrade it from the
> > backports repository.
> 
> You mean the src:linux-latest package, right?

Exactly, sorry for the typo.

> This is kind of unfortunate, but should work.  Even when the set of
> flavours for a architecture is changed, src:linux-latest builds
> transitional packages to support upgrades.

Right, that's what I had in mind, having witnessed the transition from
586 to 686.

> > debian-cd
> > =
> [...]
> > => It might be nice to have some kind of consistency check, at least
> >to make sure the ABI is the same for the linux kernel produced by
> >the d-i build, and for the modules available in the backports
> >suite. [Note: this might be true for weekly builds too, see recent
> >complaints/bug reports.] Bonus points if the source version is
> >checked too, for extra caution.
> 
> It is not a nice bonus, but really important that the source version is
> equal.  Using module udebs older than the kernel image will usually
> (but not always) work, but using newer module udebs will often result
> in unresolvable symbols for some modules.

Right now, we build debian-installer and trigger debian-cd builds under
a udeb freeze, so we cannot be out of sync for alpha releases. That's
why I called it a “bonus” for regular uploads.

But yeah, for weekly builds we don't have such a guarantee, and we won't
have that either for backports (even if we can just coordinate with the
kernel team when we're about to build images with backports enabled).

> > The finish-install script ran as expected, but I ended up with a 4.16
> > kernel from backports in the installed system, as the linux-latest
> > binaries still point to it rather than to 4.17 (presumably because
> > linux/stretch-backports is still Needs-Build on mipsel at the moment).
> 
> That was just forgotten, I'm afraid.  Bastian has just updated linux-
> latest.

Yes, I've seen the git notification pop up on IRC earlier today, thanks!

> > Users would have reproducible results over
> >time, instead of a jumpy target. I don't know debian-cd enough to
> >determine how feasible it would be. I'm also not sure what would happen
> >if we had “old” linux-image packages on the ISO, and “newer” ones on
> >mirrors.
> 
> If they *do* enable network sources then they should get the latest
> version.  This is not so different from installing stable and getting a
> security update instead of the version on the installation image.

Right, that seems reasonable.

(I think I was still a bit surprised by having had 4.16 installed with
4.17 running in d-i; which might not work too well… The other way
around, i.e. a higher version installed, shouldn't be an issue though.)

> > Open questions
> > ==
> > 
> > Now that I've explained (in length…) how it works, I think it's high
> > time we define what our target is.
> > 
> > Due to the volatility of a backports suite, I would tend to not even try
> > to support netboot.
> 
> Agreed.
> 
> > Aiming for a least a netinst image would look good
> > to me. If we can manage that, we can probably also produce a CD1 for
> > those who want to have more packages than just what's on a netinst (I've
> > had at least one such request). I'm not going to debate how many
> > desktops we should have CD1 for, that's really not my call. I'll just
> > mention that size constraints might be tricky since we'll have both the
> > linux-image packages for the base suite and those from the backports
> > suite. The whole DVD/BR thing is probably entirely overkill.
> 
> As I understand it, CD#1 is already fairly useless for non-networked
> (or limited network) installations and DVD#1 is a lot more useful.

OK, I'll leave that up to Steve and the images team anyway.

> > => Choice to make: netinst and maybe CD1(s)? Something else?
> > 
> > It's probably a good idea to consider building matching unofficial
> > image(s) with firmwares embedded. I'm not an expert regarding debian-cd
> > so help is much welcome. Tweaks might be needed to get firmwares
> > installed and/or 

Re: Status and open questions for debian-installer with backports support

[Ben Hutchings]

On Thu, 2018-08-02 at 10:56 +0200, Cyril Brulebois wrote:
[...]
> debian-installer
> 
[...]
> net-retriever
> -
[...]
>   
> https://salsa.debian.org/installer-team/net-retriever/commit/09535b0613a85e02d511b92b60683b20405b6d42

This looks rather inefficient.  Maybe not a big deal for the small
number of udebs though.

(Time to put a more powerful scripting language in the installer?  Even
awk would be a step up!)

[...]
> base-installer
> --
[...]
> We could probably modify library.sh to reuse it, but I'd like to have
> minimal changes (for reasons explained in the very last section). We
> could probably just iterate over all installed linux-image-'*' packages,
> pick the one from the src:linux package, and upgrade it from the
> backports repository.

You mean the src:linux-latest package, right?

This is kind of unfortunate, but should work.  Even when the set of
flavours for a architecture is changed, src:linux-latest builds
transitional packages to support upgrades.

[...]
> debian-cd
> =
[...]
> => It might be nice to have some kind of consistency check, at least
>to make sure the ABI is the same for the linux kernel produced by
>the d-i build, and for the modules available in the backports
>suite. [Note: this might be true for weekly builds too, see recent
>complaints/bug reports.] Bonus points if the source version is
>checked too, for extra caution.

It is not a nice bonus, but really important that the source version is
equal.  Using module udebs older than the kernel image will usually
(but not always) work, but using newer module udebs will often result
in unresolvable symbols for some modules.

[...]
> The finish-install script ran as expected, but I ended up with a 4.16
> kernel from backports in the installed system, as the linux-latest
> binaries still point to it rather than to 4.17 (presumably because
> linux/stretch-backports is still Needs-Build on mipsel at the moment).

That was just forgotten, I'm afraid.  Bastian has just updated linux-
latest.

> It was downloaded from the network mirror.
> 
> => It would be nice if we could include the linux-image-* packages from
>backports (and their dependencies) directly on the installation image,
>so as to be indpendent of what's happening in the stretch-backports
>suite on online mirrors.

I agree, it should be possible to install without network sources
enabled.

> Users would have reproducible results over
>time, instead of a jumpy target. I don't know debian-cd enough to
>determine how feasible it would be. I'm also not sure what would happen
>if we had “old” linux-image packages on the ISO, and “newer” ones on
>mirrors.

If they *do* enable network sources then they should get the latest
version.  This is not so different from installing stable and getting a
security update instead of the version on the installation image.

> Open questions
> ==
> 
> Now that I've explained (in length…) how it works, I think it's high
> time we define what our target is.
> 
> Due to the volatility of a backports suite, I would tend to not even try
> to support netboot.

Agreed.

> Aiming for a least a netinst image would look good
> to me. If we can manage that, we can probably also produce a CD1 for
> those who want to have more packages than just what's on a netinst (I've
> had at least one such request). I'm not going to debate how many
> desktops we should have CD1 for, that's really not my call. I'll just
> mention that size constraints might be tricky since we'll have both the
> linux-image packages for the base suite and those from the backports
> suite. The whole DVD/BR thing is probably entirely overkill.

As I understand it, CD#1 is already fairly useless for non-networked
(or limited network) installations and DVD#1 is a lot more useful.

> => Choice to make: netinst and maybe CD1(s)? Something else?
> 
> It's probably a good idea to consider building matching unofficial
> image(s) with firmwares embedded. I'm not an expert regarding debian-cd
> so help is much welcome. Tweaks might be needed to get firmwares
> installed and/or upgraded from the backports repository, be it to embed
> them on the image, or to make them available to the installed system.

Yes, firmware would need to come from backports.

Since there is no dependency relation between the kernel and firmware
package, I don't prune old files until they are no longer referenced by
the kernel version in any supported suite.  So the CD image build would
only need to use firmware packages from backports, not two different
versions.

[...]
> Last question: how often do we produce such an image? I don't think it's
> reasonable to do that every time a new major version of linux is made
> available in the backports repository. Doing that once around the middle
> of the release cycle would look good to me. This would kind of mimic the
> 

Status and open questions for debian-installer with backports support

[Cyril Brulebois]

Hi,

It's been a while since my first proof of concept (demonstrated at the
2014 mini-DebConf in Paris), but I think I've managed to reach a point
where I'm rather content and ready to get bits and pieces merged where
they belong. That doesn't mean I have everything figured out, that's
why I'm reaching out to both the installer & images team!


The primary question we need to discuss is what we are aiming for. I'll
start with a (not so) little flashback to explain what I've worked on,
why, and what is likely (not) going to be our limited target.

This is a long read… I've highlighted “to do”-like items with an arrow
(=>).


debian-installer


First things first, we needed a debian-installer build that knew about a
backported linux kernel, so I hacked my way in debian-installer.git; it
wasn't too hard since we can count on apt to resolve dependencies, and
after all, we only needed the kernel-image udeb and its modules.

The latest branch is stretch-backports-v2, which combines both a number
of commits that are aimed at the master branch, and a couple others that
want to only be used in a stretch-backports branch:

  
https://salsa.debian.org/installer-team/debian-installer/commits/stretch-backports-v2


net-retriever
-

As usual, I've tested the resulting installer with the netboot(-gtk)
image and of course it failed to load additional udebs. These netboot*
images are aimed at booting from the network, and they also need to
fetch additional udebs from a mirror. Since net-retriever only knows
about a single source of udebs, it needed to be told about multiple
sources (stretch and stretch-backports) and also how to merge entries.
I've detailed my choices in this commit in particular:

  
https://salsa.debian.org/installer-team/net-retriever/commit/09535b0613a85e02d511b92b60683b20405b6d42

For more details, see the whole stretch-backports-v2 branch for
net-retriever:

  
https://salsa.debian.org/installer-team/net-retriever/commits/stretch-backports-v2

Now, it doesn't matter too much, since the stretch-backports repository
will evolve over time, and udebs for the specific kernel used during the
d-i build will disappear. I think this is too much of a volatile (no
pun intended) target to support. It's still a very nice way to test d-i
without having to build an ISO with debian-cd, so I'd like to merge
these changes in net-retriever anyway.


base-installer
--

Second, it makes quite some sense to not only run d-i with a newer
kernel, but to also install it, so that users can boot from their brand
new hardware that wasn't supported by d-i with stable components only.

The kernel selection and installation is implemented in base-installer
(see library.sh), and that happens to run before apt-setup. Since I
didn't want to butcher sources.list manually to install the kernel
directly at this stage, I've decided to implement installing the latest
kernel from backports in a finish-install script (also shipped in the
base-installer package, for consistency), running at the very end of the
installation process.

This only happens when backports support was enabled (this is detected
through the presence of /etc/udebs-backports-source).

At first, using the linux-image-$arch metapackage looked easy enough but
of course that works for amd64, but not for i386 (one would need to pick
686 or 686-pae). I've still pushed a branch with this implementation as
it can be useful as is:

  
https://salsa.debian.org/installer-team/base-installer/commits/stretch-backports-v0

We could probably modify library.sh to reuse it, but I'd like to have
minimal changes (for reasons explained in the very last section). We
could probably just iterate over all installed linux-image-'*' packages,
pick the one from the src:linux package, and upgrade it from the
backports repository.

=> I'll look into that later on.



apt-setup
-

Also only when /etc/udebs-backports-source is present, the backports
service gets automatically enabled, which makes the later installation
of linux-image packages possible.

Single commit for this:

  
https://salsa.debian.org/installer-team/apt-setup/commit/6133bdcd5a7903105ad967ab087f2f12f9d1c59d



debian-cd
=

As mentioned above, tricks were needed in net-retriever for the netboot*
images, to make it possible to load linux kernel modules udebs. Those
tricks aren't needed when generating an ISO image. As a reminder, on the
d-i side, a cdrom_isolinux image is generated, and the debian-cd tool
needs to be configured to use that image instead of the official image
available on the configured mirror.

I didn't make this dynamic yet, but I've modified my local configuration
to set BACKPORTS=backports-list, to include a number of packages from
the stretch-backports suite. This file was generating by listing all
udebs produced by the linux source package (as done in the d-i build
system):

USE_UDEBS_BACKPORTS_FROM=stretch-backports
...
source=linux
...