Re: Improving firmware reporting
On Sun, 2016-05-22 at 21:43 +0200, Cyril Brulebois wrote: > Ben Hutchings (2016-05-22): > > > > All the binary packages built from firmware-nonfree get it > > automatically, but I forgot there were so many other firmware packages. > > Maybe your way is better for now. > ACK. Easy enough to toggle between both anyway. Maybe I should even > keep both codepaths active and use them to detect inconsistencies > between file list in Contents and Appstream metadata in Components… > > > > > > > > > amd64-microcode > > > atmel-firmware > > > bluez-firmware > > > dahdi-firmware-nonfree > > > firmware-crystalhd > > > firmware-ipw2x00 > > firmware-ipw2x00 does have it. > Alright. I ran diff plus some pipes, without checking each and every > package, which explains this kind of false positive. > > It seems the following files are under lib/firmware but not listed in > Appstream metadata: > lib/firmware/ipw2x00.LICENSE > lib/firmware/isci/isci_firmware.bin > > Not sure about the former It's a symlink to the licence text, presumably because Bastian read the distributor licence as requiring we put it in the same directory as the firmware. > but I suppose the latter should be listed there? [...] The latter is in firmware-linux-free, which as I said doesn't generate DEP-11 metadata. I'll fix that in the next upload. Ben. -- Ben Hutchings friends: People who know you well, but like you anyway. signature.asc Description: This is a digitally signed message part
Re: Improving firmware reporting
Ben Hutchings (2016-05-22): > All the binary packages built from firmware-nonfree get it > automatically, but I forgot there were so many other firmware packages. > Maybe your way is better for now. ACK. Easy enough to toggle between both anyway. Maybe I should even keep both codepaths active and use them to detect inconsistencies between file list in Contents and Appstream metadata in Components… > > amd64-microcode > > atmel-firmware > > bluez-firmware > > dahdi-firmware-nonfree > > firmware-crystalhd > > firmware-ipw2x00 > > firmware-ipw2x00 does have it. Alright. I ran diff plus some pipes, without checking each and every package, which explains this kind of false positive. It seems the following files are under lib/firmware but not listed in Appstream metadata: lib/firmware/ipw2x00.LICENSE lib/firmware/isci/isci_firmware.bin Not sure about the former but I suppose the latter should be listed there? > > firmware-linux-free > > firmware-zd1211 > > intel-microcode > > ixp4xx-microcode > > prism2-usb-firmware-installer > > > > (firmware-linux-free might be special because in main; not sure.) > > That means it's built from a separate source package which I haven't > yet added DEP-11 generation to. But it's also recommended by linux- > image-* so gets installed by default anyway. Well, as far as I can tell from a debian-installer build, this package isn't used at build-time. That it gets installed by default is one thing; but as I mentioned in my initial reply, we might need to load firmware(s) within the installer context, sometimes before having network set up, or the /target filesystem, etc. (I have no idea whether that is relevant for any of the firmwares included in this particular package.) Also: Thanks for your feedback. KiBi. signature.asc Description: Digital signature
Re: Improving firmware reporting
On Sun, 2016-05-22 at 21:11 +0200, Cyril Brulebois wrote: > Cyril Brulebois (2016-05-22): > > > > Ben Hutchings (2016-05-22): > > > > > > That information already exists in DEP-11 metadata that APT will > > > download for us. > > Right, forgot about that. But what the mapping is built from doesn't > > really change the need for embedding it, see below. > I've modified the script to use dep11 information. Comparing results, it > seems some packages are missing those “AppStream metadata”, and should > probably get a bug report accordingly? All the binary packages built from firmware-nonfree get it automatically, but I forgot there were so many other firmware packages. Maybe your way is better for now. > amd64-microcode > atmel-firmware > bluez-firmware > dahdi-firmware-nonfree > firmware-crystalhd > firmware-ipw2x00 firmware-ipw2x00 does have it. > firmware-linux-free > firmware-zd1211 > intel-microcode > ixp4xx-microcode > prism2-usb-firmware-installer > > (firmware-linux-free might be special because in main; not sure.) That means it's built from a separate source package which I haven't yet added DEP-11 generation to. But it's also recommended by linux- image-* so gets installed by default anyway. Ben. > In the meanwhile I've pushed the commit to the dep11 branch instead of > the master one (still in the hw-detect repository). > > > KiBi. -- Ben Hutchings friends: People who know you well, but like you anyway. signature.asc Description: This is a digitally signed message part
Re: Improving firmware reporting
Cyril Brulebois (2016-05-22): > Ben Hutchings (2016-05-22): > > That information already exists in DEP-11 metadata that APT will > > download for us. > > Right, forgot about that. But what the mapping is built from doesn't > really change the need for embedding it, see below. I've modified the script to use dep11 information. Comparing results, it seems some packages are missing those “AppStream metadata”, and should probably get a bug report accordingly? amd64-microcode atmel-firmware bluez-firmware dahdi-firmware-nonfree firmware-crystalhd firmware-ipw2x00 firmware-linux-free firmware-zd1211 intel-microcode ixp4xx-microcode prism2-usb-firmware-installer (firmware-linux-free might be special because in main; not sure.) In the meanwhile I've pushed the commit to the dep11 branch instead of the master one (still in the hw-detect repository). KiBi. signature.asc Description: Digital signature
Re: Improving firmware reporting
Ben Hutchings (2016-05-22):
> On Sun, 2016-05-22 at 19:26 +0200, Cyril Brulebois wrote:
> > [ Note: I've added d-l-e to the loop since people there might have some
> > insight about the prompt update I'm proposing. ]
> >
> > Hi,
> >
> > An email earlier today reminded me of this old topic: it would be nice
> > if hw-detect would point us to the right firmware package(s) instead of
> > letting D-I only report the list of missing firmware files.
> >
> > I've modified hw-detect to make it possible to build (and update) such
> > a mapping:
> >
> > ${firmware_filename} ${firmware_package} ${section}
>
> That information already exists in DEP-11 metadata that APT will
> download for us.
Right, forgot about that. But what the mapping is built from doesn't
really change the need for embedding it, see below.
> > which is going to be shipped as usr/share/hw-detect/firmware-map in
> > the hw-detect binary.
> >
> > Like choose-mirror, this file is generated outside of the source and
> > binary builds (since it needs network access), and will need updating
> > from time to time (ideally once before every release, but I'll probably
> > add a cron job on d-i.debian.org to get notifications).
> [...]
>
> Please use the existing index which will stay up-to-date.
Well, this needs to be embedded within d-i for a number of installation
images. We can't rely on a network mirror in any cases, and we don't
have dep11 in cdrom images at the moment either. Checking with
debian-stretch-DI-alpha6-i386-netinst.iso:
dists/stretch/contrib/binary-i386/Release
dists/stretch/main/binary-i386/Packages.gz
dists/stretch/main/binary-i386/Release
dists/stretch/main/debian-installer/binary-i386/Packages.gz
dists/stretch/main/debian-installer/binary-i386/Release
dists/stretch/main/i18n/Translation-ca.gz
dists/stretch/main/i18n/Translation-cs.gz
dists/stretch/main/i18n/Translation-da.gz
dists/stretch/main/i18n/Translation-de_DE.gz
dists/stretch/main/i18n/Translation-de.gz
dists/stretch/main/i18n/Translation-en.gz
dists/stretch/main/i18n/Translation-eo.gz
dists/stretch/main/i18n/Translation-es.gz
dists/stretch/main/i18n/Translation-eu.gz
dists/stretch/main/i18n/Translation-fi.gz
dists/stretch/main/i18n/Translation-fr.gz
dists/stretch/main/i18n/Translation-hr.gz
dists/stretch/main/i18n/Translation-hu.gz
dists/stretch/main/i18n/Translation-id.gz
dists/stretch/main/i18n/Translation-it.gz
dists/stretch/main/i18n/Translation-ja.gz
dists/stretch/main/i18n/Translation-km.gz
dists/stretch/main/i18n/Translation-ko.gz
dists/stretch/main/i18n/Translation-nb.gz
dists/stretch/main/i18n/Translation-nl.gz
dists/stretch/main/i18n/Translation-pl.gz
dists/stretch/main/i18n/Translation-pt_BR.gz
dists/stretch/main/i18n/Translation-pt.gz
dists/stretch/main/i18n/Translation-ro.gz
dists/stretch/main/i18n/Translation-ru.gz
dists/stretch/main/i18n/Translation-sk.gz
dists/stretch/main/i18n/Translation-sr.gz
dists/stretch/main/i18n/Translation-sv.gz
dists/stretch/main/i18n/Translation-uk.gz
dists/stretch/main/i18n/Translation-vi.gz
dists/stretch/main/i18n/Translation-zh_CN.gz
dists/stretch/main/i18n/Translation-zh.gz
dists/stretch/main/i18n/Translation-zh_TW.gz
dists/stretch/Release
KiBi.
signature.asc
Description: Digital signature
Re: Improving firmware reporting
On Sun, 2016-05-22 at 19:26 +0200, Cyril Brulebois wrote:
> [ Note: I've added d-l-e to the loop since people there might have some
> insight about the prompt update I'm proposing. ]
>
> Hi,
>
> An email earlier today reminded me of this old topic: it would be nice
> if hw-detect would point us to the right firmware package(s) instead of
> letting D-I only report the list of missing firmware files.
>
> I've modified hw-detect to make it possible to build (and update) such
> a mapping:
>
> ${firmware_filename} ${firmware_package} ${section}
That information already exists in DEP-11 metadata that APT will
download for us.
> which is going to be shipped as usr/share/hw-detect/firmware-map in
> the hw-detect binary.
>
> Like choose-mirror, this file is generated outside of the source and
> binary builds (since it needs network access), and will need updating
> from time to time (ideally once before every release, but I'll probably
> add a cron job on d-i.debian.org to get notifications).
[...]
Please use the existing index which will stay up-to-date.
Ben.
--
Ben Hutchings
friends: People who know you well, but like you anyway.
signature.asc
Description: This is a digitally signed message part
Improving firmware reporting
[ Note: I've added d-l-e to the loop since people there might have some
insight about the prompt update I'm proposing. ]
Hi,
An email earlier today reminded me of this old topic: it would be nice
if hw-detect would point us to the right firmware package(s) instead of
letting D-I only report the list of missing firmware files.
I've modified hw-detect to make it possible to build (and update) such
a mapping:
${firmware_filename} ${firmware_package} ${section}
which is going to be shipped as usr/share/hw-detect/firmware-map in
the hw-detect binary.
Like choose-mirror, this file is generated outside of the source and
binary builds (since it needs network access), and will need updating
from time to time (ideally once before every release, but I'll probably
add a cron job on d-i.debian.org to get notifications).
=> The question is now: how do we present this to users?
The current template is:
| Template: hw-detect/load_firmware
| Type: boolean
| Default: true
| # :sl2:
| _Description: Load missing firmware from removable media?
| Some of your hardware needs non-free firmware files to operate. The
| firmware can be loaded from removable media, such as a USB stick or
| floppy.
| .
| The missing firmware files are: ${FILES}
| .
| If you have such media available now, insert it, and continue.
The FILES variable gets set by check-missing-firmware.sh through:
db_subst hw-detect/load_firmware FILES "$files"
and we could do the same with a PACKAGES variable which would contain
e.g.:
firmware-atheros (non-free), prism2-usb-firmware-installer (contrib)
(by looping over $files and grepping into this new mapping file.)
Quick idea:
| _Description: Load missing firmware from removable media?
| Some of your hardware needs non-free firmware files to operate. The
| firmware can be loaded from removable media, such as a USB stick or
| floppy.
| .
| The missing firmware files are: ${FILES}
| .
| The following packages contain some of these files: ${PACKAGES}
| .
| If you have such media available now, insert it, and continue.
The trick is: we might reach some point where (at least) it's not
possible to resolve a filename to a package. That's why I've used a
rather cautious wording above. Having an extra line with the list of
unresolved filenames would be cumbersome in the general case. Having it
only conditionally would mean having troubles translating it.
Maybe sticking to the simple sentence below would be appropriate for
most cases:
| They can be found in the following packages: ${PACKAGES}
What do you think?
KiBi.
signature.asc
Description: Digital signature
