Re: Improving firmware reporting

[Ben Hutchings]

On Sun, 2016-05-22 at 21:43 +0200, Cyril Brulebois wrote:
> Ben Hutchings  (2016-05-22):
> > 
> > All the binary packages built from firmware-nonfree get it
> > automatically, but I forgot there were so many other firmware packages.
> > Maybe your way is better for now.
> ACK. Easy enough to toggle between both anyway. Maybe I should even
> keep both codepaths active and use them to detect inconsistencies
> between file list in Contents and Appstream metadata in Components…
> 
> > 
> > > 
> > > amd64-microcode
> > > atmel-firmware
> > > bluez-firmware
> > > dahdi-firmware-nonfree
> > > firmware-crystalhd
> > > firmware-ipw2x00
> > firmware-ipw2x00 does have it.
> Alright. I ran diff plus some pipes, without checking each and every
> package, which explains this kind of false positive.
> 
> It seems the following files are under lib/firmware but not listed in
> Appstream metadata:
>   lib/firmware/ipw2x00.LICENSE
>   lib/firmware/isci/isci_firmware.bin
> 
> Not sure about the former

It's a symlink to the licence text, presumably because Bastian read the
distributor licence as requiring we put it in the same directory as the
firmware.

> but I suppose the latter should be listed there?
[...]

The latter is in firmware-linux-free, which as I said doesn't generate
DEP-11 metadata.  I'll fix that in the next upload.

Ben.

-- 
Ben Hutchings
friends: People who know you well, but like you anyway.

signature.asc
Description: This is a digitally signed message part

Re: Improving firmware reporting

[Cyril Brulebois]

Ben Hutchings  (2016-05-22):
> All the binary packages built from firmware-nonfree get it
> automatically, but I forgot there were so many other firmware packages.
> Maybe your way is better for now.

ACK. Easy enough to toggle between both anyway. Maybe I should even
keep both codepaths active and use them to detect inconsistencies
between file list in Contents and Appstream metadata in Components…

> > amd64-microcode
> > atmel-firmware
> > bluez-firmware
> > dahdi-firmware-nonfree
> > firmware-crystalhd
> > firmware-ipw2x00
> 
> firmware-ipw2x00 does have it.

Alright. I ran diff plus some pipes, without checking each and every
package, which explains this kind of false positive.

It seems the following files are under lib/firmware but not listed in
Appstream metadata:
  lib/firmware/ipw2x00.LICENSE
  lib/firmware/isci/isci_firmware.bin

Not sure about the former but I suppose the latter should be listed
there?

> > firmware-linux-free
> > firmware-zd1211
> > intel-microcode
> > ixp4xx-microcode
> > prism2-usb-firmware-installer
> > 
> > (firmware-linux-free might be special because in main; not sure.)
> 
> That means it's built from a separate source package which I haven't
> yet added DEP-11 generation to.  But it's also recommended by linux-
> image-* so gets installed by default anyway.

Well, as far as I can tell from a debian-installer build, this package
isn't used at build-time. That it gets installed by default is one
thing; but as I mentioned in my initial reply, we might need to load
firmware(s) within the installer context, sometimes before having
network set up, or the /target filesystem, etc.

(I have no idea whether that is relevant for any of the firmwares
included in this particular package.)

Also: Thanks for your feedback.


KiBi.


signature.asc
Description: Digital signature

Re: Improving firmware reporting

[Ben Hutchings]

On Sun, 2016-05-22 at 21:11 +0200, Cyril Brulebois wrote:
> Cyril Brulebois  (2016-05-22):
> > 
> > Ben Hutchings  (2016-05-22):
> > > 
> > > That information already exists in DEP-11 metadata that APT will
> > > download for us.
> > Right, forgot about that. But what the mapping is built from doesn't
> > really change the need for embedding it, see below.
> I've modified the script to use dep11 information. Comparing results, it
> seems some packages are missing those “AppStream metadata”, and should
> probably get a bug report accordingly?

All the binary packages built from firmware-nonfree get it
automatically, but I forgot there were so many other firmware packages.
Maybe your way is better for now.

> amd64-microcode
> atmel-firmware
> bluez-firmware
> dahdi-firmware-nonfree
> firmware-crystalhd
> firmware-ipw2x00

firmware-ipw2x00 does have it.

> firmware-linux-free
> firmware-zd1211
> intel-microcode
> ixp4xx-microcode
> prism2-usb-firmware-installer
> 
> (firmware-linux-free might be special because in main; not sure.)

That means it's built from a separate source package which I haven't
yet added DEP-11 generation to.  But it's also recommended by linux-
image-* so gets installed by default anyway.

Ben.

> In the meanwhile I've pushed the commit to the dep11 branch instead of
> the master one (still in the hw-detect repository).
> 
> 
> KiBi.
-- 
Ben Hutchings
friends: People who know you well, but like you anyway.

signature.asc
Description: This is a digitally signed message part

Re: Improving firmware reporting

[Cyril Brulebois]

Cyril Brulebois  (2016-05-22):
> Ben Hutchings  (2016-05-22):
> > That information already exists in DEP-11 metadata that APT will
> > download for us.
> 
> Right, forgot about that. But what the mapping is built from doesn't
> really change the need for embedding it, see below.

I've modified the script to use dep11 information. Comparing results, it
seems some packages are missing those “AppStream metadata”, and should
probably get a bug report accordingly?

amd64-microcode
atmel-firmware
bluez-firmware
dahdi-firmware-nonfree
firmware-crystalhd
firmware-ipw2x00
firmware-linux-free
firmware-zd1211
intel-microcode
ixp4xx-microcode
prism2-usb-firmware-installer

(firmware-linux-free might be special because in main; not sure.)

In the meanwhile I've pushed the commit to the dep11 branch instead of
the master one (still in the hw-detect repository).


KiBi.


signature.asc
Description: Digital signature

Re: Bug#758116: Allow to select Blends selection during installation - just "DE", "Web server", "Mail server" is NOT enough

[Ole Streicher]

Hi Cyril,

Am 21.05.2016 um 23:11 schrieb Cyril Brulebois:
> so it would be nice to support all desc files shipped in tasksel-data
> rather than hardcoding debian-tasks.desc when the --internal-tasks-only
> flag is passed.

If you want to do it in the way it was proposed some days ago (move the
blends and the desktop choice into separate pages):

You could use a "Section" keyword in the tasks header: this is already
there for the structuration of tasks. Then the "main" task would just
display everything without a section, and one option for each section
(currently "Desktop Environment" and "Debian Pure Blends"). Enabling
these options leads to follow-up screens showing their content.

Aside from keeping the initial tasksel screen clean, this would also
naturally remove the confusing checkboxes that are currently on the
sections headers.

Best regards

Ole

Re: Improving firmware reporting

[Cyril Brulebois]

Ben Hutchings  (2016-05-22):
> On Sun, 2016-05-22 at 19:26 +0200, Cyril Brulebois wrote:
> > [ Note: I've added d-l-e to the loop since people there might have some
> >   insight about the prompt update I'm proposing. ]
> > 
> > Hi,
> > 
> > An email earlier today reminded me of this old topic: it would be nice
> > if hw-detect would point us to the right firmware package(s) instead of
> > letting D-I only report the list of missing firmware files.
> > 
> > I've modified hw-detect to make it possible to build (and update) such
> > a mapping:
> > 
> > ${firmware_filename} ${firmware_package} ${section}
> 
> That information already exists in DEP-11 metadata that APT will
> download for us.

Right, forgot about that. But what the mapping is built from doesn't
really change the need for embedding it, see below.

> > which is going to be shipped as usr/share/hw-detect/firmware-map in
> > the hw-detect binary.
> > 
> > Like choose-mirror, this file is generated outside of the source and
> > binary builds (since it needs network access), and will need updating
> > from time to time (ideally once before every release, but I'll probably
> > add a cron job on d-i.debian.org to get notifications).
> [...]
> 
> Please use the existing index which will stay up-to-date.

Well, this needs to be embedded within d-i for a number of installation
images. We can't rely on a network mirror in any cases, and we don't
have dep11 in cdrom images at the moment either. Checking with
debian-stretch-DI-alpha6-i386-netinst.iso:

dists/stretch/contrib/binary-i386/Release
dists/stretch/main/binary-i386/Packages.gz
dists/stretch/main/binary-i386/Release
dists/stretch/main/debian-installer/binary-i386/Packages.gz
dists/stretch/main/debian-installer/binary-i386/Release
dists/stretch/main/i18n/Translation-ca.gz
dists/stretch/main/i18n/Translation-cs.gz
dists/stretch/main/i18n/Translation-da.gz
dists/stretch/main/i18n/Translation-de_DE.gz
dists/stretch/main/i18n/Translation-de.gz
dists/stretch/main/i18n/Translation-en.gz
dists/stretch/main/i18n/Translation-eo.gz
dists/stretch/main/i18n/Translation-es.gz
dists/stretch/main/i18n/Translation-eu.gz
dists/stretch/main/i18n/Translation-fi.gz
dists/stretch/main/i18n/Translation-fr.gz
dists/stretch/main/i18n/Translation-hr.gz
dists/stretch/main/i18n/Translation-hu.gz
dists/stretch/main/i18n/Translation-id.gz
dists/stretch/main/i18n/Translation-it.gz
dists/stretch/main/i18n/Translation-ja.gz
dists/stretch/main/i18n/Translation-km.gz
dists/stretch/main/i18n/Translation-ko.gz
dists/stretch/main/i18n/Translation-nb.gz
dists/stretch/main/i18n/Translation-nl.gz
dists/stretch/main/i18n/Translation-pl.gz
dists/stretch/main/i18n/Translation-pt_BR.gz
dists/stretch/main/i18n/Translation-pt.gz
dists/stretch/main/i18n/Translation-ro.gz
dists/stretch/main/i18n/Translation-ru.gz
dists/stretch/main/i18n/Translation-sk.gz
dists/stretch/main/i18n/Translation-sr.gz
dists/stretch/main/i18n/Translation-sv.gz
dists/stretch/main/i18n/Translation-uk.gz
dists/stretch/main/i18n/Translation-vi.gz
dists/stretch/main/i18n/Translation-zh_CN.gz
dists/stretch/main/i18n/Translation-zh.gz
dists/stretch/main/i18n/Translation-zh_TW.gz
dists/stretch/Release


KiBi.


signature.asc
Description: Digital signature

Re: Improving firmware reporting

[Ben Hutchings]

On Sun, 2016-05-22 at 19:26 +0200, Cyril Brulebois wrote:
> [ Note: I've added d-l-e to the loop since people there might have some
>   insight about the prompt update I'm proposing. ]
> 
> Hi,
> 
> An email earlier today reminded me of this old topic: it would be nice
> if hw-detect would point us to the right firmware package(s) instead of
> letting D-I only report the list of missing firmware files.
> 
> I've modified hw-detect to make it possible to build (and update) such
> a mapping:
> 
> ${firmware_filename} ${firmware_package} ${section}

That information already exists in DEP-11 metadata that APT will
download for us.

> which is going to be shipped as usr/share/hw-detect/firmware-map in
> the hw-detect binary.
> 
> Like choose-mirror, this file is generated outside of the source and
> binary builds (since it needs network access), and will need updating
> from time to time (ideally once before every release, but I'll probably
> add a cron job on d-i.debian.org to get notifications).
[...]

Please use the existing index which will stay up-to-date.

Ben.

-- 
Ben Hutchings
friends: People who know you well, but like you anyway.

signature.asc
Description: This is a digitally signed message part

Re: stretch installer question

[Jude DaShiell]

I don't need ssh-server, I need ssh-client and that's what was missing. 
Beyond that though, why is iwconfig on any of these isos?  The iw 
package was supposed to have replaced iwconfig especially with the more 
modern linux kernels.


On Sun, 22 May 2016, Steve McIntyre wrote:


Date: Sat, 21 May 2016 19:31:38
From: Steve McIntyre 
To: Jude DaShiell 
Cc: debian-cd@lists.debian.org, debian-b...@lists.debian.org
Subject: Re: stretch installer question

On Sat, May 21, 2016 at 12:13:10PM -0400, Jude DaShiell wrote:

Is the package selection section of the stretch installer as broken in all
versions as it is in the x86_64 version of firmware testing iso?  I preserved
logs but once installation was finished and system rebooted, ssh was nowhere
to be found on the new system.  I may be able to copy the installation log or
logs off of the new system onto a flash drive then copy those logs back onto
another hard drive to send them along but can't do it directly from the newly
installed system since neither telnet rlogin or ssh got onto it.


Not everybody wants to run servers (even ssh) by default on new
machine installations, so ssh isn't installed *by default*. However,
there is an option in tasksel to install "SSH server" if that's what
you want.




--

Improving firmware reporting

[Cyril Brulebois]

[ Note: I've added d-l-e to the loop since people there might have some
  insight about the prompt update I'm proposing. ]

Hi,

An email earlier today reminded me of this old topic: it would be nice
if hw-detect would point us to the right firmware package(s) instead of
letting D-I only report the list of missing firmware files.

I've modified hw-detect to make it possible to build (and update) such
a mapping:

${firmware_filename} ${firmware_package} ${section}

which is going to be shipped as usr/share/hw-detect/firmware-map in
the hw-detect binary.

Like choose-mirror, this file is generated outside of the source and
binary builds (since it needs network access), and will need updating
from time to time (ideally once before every release, but I'll probably
add a cron job on d-i.debian.org to get notifications).


=> The question is now: how do we present this to users?

The current template is:
| Template: hw-detect/load_firmware
| Type: boolean
| Default: true
| # :sl2:
| _Description: Load missing firmware from removable media?
|  Some of your hardware needs non-free firmware files to operate. The
|  firmware can be loaded from removable media, such as a USB stick or
|  floppy.
|  .
|  The missing firmware files are: ${FILES}
|  .
|  If you have such media available now, insert it, and continue.

The FILES variable gets set by check-missing-firmware.sh through:

db_subst hw-detect/load_firmware FILES "$files"

and we could do the same with a PACKAGES variable which would contain
e.g.:

firmware-atheros (non-free), prism2-usb-firmware-installer (contrib)

(by looping over $files and grepping into this new mapping file.)


Quick idea:
| _Description: Load missing firmware from removable media?
|  Some of your hardware needs non-free firmware files to operate. The
|  firmware can be loaded from removable media, such as a USB stick or
|  floppy.
|  .
|  The missing firmware files are: ${FILES}
|  .
|  The following packages contain some of these files: ${PACKAGES}
|  .
|  If you have such media available now, insert it, and continue.

The trick is: we might reach some point where (at least) it's not
possible to resolve a filename to a package. That's why I've used a
rather cautious wording above. Having an extra line with the list of
unresolved filenames would be cumbersome in the general case. Having it
only conditionally would mean having troubles translating it.

Maybe sticking to the simple sentence below would be appropriate for
most cases:
| They can be found in the following packages: ${PACKAGES}


What do you think?


KiBi.


signature.asc
Description: Digital signature

Re: Bug#758116: Allow to select Blends selection during installation - just "DE", "Web server", "Mail server" is NOT enough

[Petter Reinholdtsen]

[Cyril Brulebois]
> Please explain how you came to that conclusion.

I'm sorry, but the thread so far do not make me believe you are not
really want to understand what I mean, but instead look for a way to
push your view and any explanation I come up with would be brushed away.

I believe it is best for me to not get involved in d-i.  To me, based on
the current and earlier email and IRC exchanges, d-i development seem
like a toxic environment and I believe my effort is better spent
elsewhere.  Thus I do not see the point of spending the time to try to
explain why and how my view is fundamentally different from yours, as I
am conviced the effort will be wasted.  It make me sad, but I just do
not have the energy to try to do something about it.

I was hoping to work on hw-detect and isenkram integration, but have not
been able to muster the motivation to do it so far.  I will probably
limit myself to adding an udeb for isenkram and leave the d-i part to
others, even if it probably mean automatic firmware setup will not
become part of the official installer.

I suspect the cause is just a question of incompatible personalities
involved, and either that the culture was different back when we ran the
d-i project at the start or that I changed so much the culture is no
longer friendly to me.

-- 
Happy hacking
Petter Reinholdtsen

Re: Bug#758116: Allow to select Blends selection during installation - just "DE", "Web server", "Mail server" is NOT enough

[Cyril Brulebois]

Petter Reinholdtsen  (2016-05-22):
> [Cyril Brulebois]
> > There's no udebs involved in what I summarized for Blends.
> 
> Exactly.

Thanks for confirming that your “Being able to add extra tasks using
udebs is a feature, not a bug.” wasn't really on topic then.

> I suspect using udebs to enable blends is be a better idea than
> making the Blends tasksel tasks priority standard.

Having this kind of move forced on us doesn't seem reasonable to me,
which has been exactly my point over the past few mails.

Let me reiterate: I don't want this to happen ever again.


> > Also: If pkgsel changes the way it calls tasksel, debian-edu udebs can
> > certainly interact with it so that it behaves as desired.
> 
> You misunderstand the role of the udebs.

Please explain how you came to that conclusion.

> The Debian Edu udeb ask for education-tasks to be installed, and
> then the normal d-i take care of the rest to get the correct Debian
> Edu tasks installed using tests and the locale settings.  Sure, we
> can come up with a new way to do it, but my point is that we are
> using this feature of tasksel today, and there is no alternative I
> know of that is equally robust and well integrated into the
> installer.

What? I'm talking about a future evolution. I can't see why something
using a pre-pkgsel.d hook to prepare things for d-i couldn't be
updated to create e.g. an extra file to get pkgsel to behave as
intended. I don't see why such implementation details would be
important in this discussion, and that's why I mentioned “debian-edu
udebs can certainly interact with it so that it behaves as desired”.

Pretty sure I'm not the one “misunderstanding” anything here.


KiBi.


signature.asc
Description: Digital signature

For image

[Sanj-Sanjeev]

Dear Sir,

  Please  provide  me Debian image file link

Thanks & Regards
Sanjeev Kumar
Braviatech pvt.ltd