Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-17 Thread Samuel Thibault
I have reverted the change for now.

But we need to discuss among various porters, I have really seen desire
for enabling the unreleased suite in ports other than the hurd port.

Samuel



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-17 Thread Samuel Thibault
John Paul Adrian Glaubitz, le lun. 17 août 2020 10:36:43 +0200, a ecrit:
> On 8/17/20 10:28 AM, Samuel Thibault wrote:
> > John Paul Adrian Glaubitz, le lun. 17 août 2020 10:17:24 +0200, a ecrit:
> >> "unreleased" is not a regular package archive and all
> >> packages in there are usually only added temporarily.
> > 
> > AFAIK that's not what ports' daily life really is.
> 
> Yes, it is. I actually think I have a good overview over this.

Then opinions differ and we need to discuss about it among porters
because that's really not something I have seen specific to the Hurd, I
have seen other porters wanting this.

> >> Does Debian Hurd require packages from "unreleased" for it's base system?
> > 
> > The Hurd needs netdde from there. It happens that the hurd package
> > itself is still in sid but that's only by luck because there's the
> > arch:all hurd-doc package.
> 
> And that package is required for the base-system and cannot be installed
> later?

netdde contains the network drivers, so yes, it is pretty much part of
the base system. Strictly speaking it could be installed "later" at the
task installation stage, but people are supposed to be able to skip that
step and still having a system that works.

> > But that's far from being specific to the Hurd. Various ports have
> > various requirements such as bootloaders.
> 
> These are always shipped on the installation CD. It also affects m68k
> only as alpha, hppa, powerpc, ppc64, sh4 and sparc64 use either GRUB or have
> their bootloaders in unstable. 
> 
> Does Hurd not use GRUB?

It does. I'm just here mentioning one of the cases that I have seen
mentioned from other ports. I don't know the details there, but there
are probably others like firmware packages, etc.  Yes probably these
can also be shipped by hand on debian-cd, but that looks to me like
duplicate work while we can simply enable unreleased.

> >> Since "unreleased" is not properly maintained due to the fact that all
> >> packages there are built and uploaded manually, I rather prefer them not
> >> to be enabled by default, especially not for the base system.
> > 
> > They are uploaded manually by the porters themselves. They are
> > responsible for what they upload there, so I don't see why it would
> > be harmful? On the contrary, it provides flexibility for fixing
> > installability without having to wait for sid to have fixes uploaded.
> 
> The problem with "unreleased" is that there are no binNMUs and no proper
> maintenance by a package maintainer. I always try to avoid using "unreleased"
> whenever possible.

Sure. But that's very convenient to keep things installable without
having to wait for sid to catch up with whatever issue is encountered by
the port. Otherwise the sid upload latency makes the port uninstallable
for long periods of time, possibly all the time when these periods
overlap.

Samuel



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-17 Thread John Paul Adrian Glaubitz
Hello!

On 8/17/20 10:28 AM, Samuel Thibault wrote:
> John Paul Adrian Glaubitz, le lun. 17 août 2020 10:17:24 +0200, a ecrit:
>> On 8/17/20 1:06 AM, Samuel Thibault wrote:
 I checked the patch and it enables "unreleased" unconditionally. I'm
 not sure whether that's a good idea.
>>>
>>> I don't think we really have a better choice: writing yet another arch
>>> list looks dirty, and trying whether the suite is available is probably
>>> quite complex to implement.
>>
>> ... I don't quite understand why "unreleased" is supposed to be part of the
>> base installation.
> 
> For ports architectures, it often is, because the sid archive does not
> contain various packages that ports need.

Well, that's not correct since the only additional packages needed are some
bootloaders (emile for m68k, for example) and those are shipped on the
installation CD.

In the case of Debian Hurd, the case here is a package temporarily being
sourced from "unreleased" because the version from unstable FTBFS on
unstable.

>> "unreleased" is not a regular package archive and all
>> packages in there are usually only added temporarily.
> 
> AFAIK that's not what ports' daily life really is.

Yes, it is. I actually think I have a good overview over this.

>> Does Debian Hurd require packages from "unreleased" for it's base system?
> 
> The Hurd needs netdde from there. It happens that the hurd package
> itself is still in sid but that's only by luck because there's the
> arch:all hurd-doc package.

And that package is required for the base-system and cannot be installed
later?

> But that's far from being specific to the Hurd. Various ports have
> various requirements such as bootloaders.

These are always shipped on the installation CD. It also affects m68k
only as alpha, hppa, powerpc, ppc64, sh4 and sparc64 use either GRUB or have
their bootloaders in unstable. 

Does Hurd not use GRUB?

>> Since "unreleased" is not properly maintained due to the fact that all
>> packages there are built and uploaded manually, I rather prefer them not
>> to be enabled by default, especially not for the base system.
> 
> They are uploaded manually by the porters themselves. They are
> responsible for what they upload there, so I don't see why it would
> be harmful? On the contrary, it provides flexibility for fixing
> installability without having to wait for sid to have fixes uploaded.

The problem with "unreleased" is that there are no binNMUs and no proper
maintenance by a package maintainer. I always try to avoid using "unreleased"
whenever possible.

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-17 Thread Samuel Thibault
Samuel Thibault, le lun. 17 août 2020 10:28:54 +0200, a ecrit:
> John Paul Adrian Glaubitz, le lun. 17 août 2020 10:17:24 +0200, a ecrit:
> > On 8/17/20 1:06 AM, Samuel Thibault wrote:
> > >> I checked the patch and it enables "unreleased" unconditionally. I'm
> > >> not sure whether that's a good idea.
> > > 
> > > I don't think we really have a better choice: writing yet another arch
> > > list looks dirty, and trying whether the suite is available is probably
> > > quite complex to implement.
> > 
> > ... I don't quite understand why "unreleased" is supposed to be part of the
> > base installation.
> 
> For ports architectures, it often is, because the sid archive does not
> contain various packages that ports need.
> 
> > "unreleased" is not a regular package archive and all
> > packages in there are usually only added temporarily.
> 
> AFAIK that's not what ports' daily life really is.

See also 

9bb019085356 ('Enable the "unreleased" suite for ports architectures.')
in the apt-setup package.

Samuel



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-17 Thread Samuel Thibault
John Paul Adrian Glaubitz, le lun. 17 août 2020 10:17:24 +0200, a ecrit:
> On 8/17/20 1:06 AM, Samuel Thibault wrote:
> >> I checked the patch and it enables "unreleased" unconditionally. I'm
> >> not sure whether that's a good idea.
> > 
> > I don't think we really have a better choice: writing yet another arch
> > list looks dirty, and trying whether the suite is available is probably
> > quite complex to implement.
> 
> ... I don't quite understand why "unreleased" is supposed to be part of the
> base installation.

For ports architectures, it often is, because the sid archive does not
contain various packages that ports need.

> "unreleased" is not a regular package archive and all
> packages in there are usually only added temporarily.

AFAIK that's not what ports' daily life really is.

> Does Debian Hurd require packages from "unreleased" for it's base system?

The Hurd needs netdde from there. It happens that the hurd package
itself is still in sid but that's only by luck because there's the
arch:all hurd-doc package.

But that's far from being specific to the Hurd. Various ports have
various requirements such as bootloaders.

> Since "unreleased" is not properly maintained due to the fact that all
> packages there are built and uploaded manually, I rather prefer them not
> to be enabled by default, especially not for the base system.

They are uploaded manually by the porters themselves. They are
responsible for what they upload there, so I don't see why it would
be harmful? On the contrary, it provides flexibility for fixing
installability without having to wait for sid to have fixes uploaded.

Samuel



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-17 Thread John Paul Adrian Glaubitz
Hello Samuel!

On 8/17/20 1:06 AM, Samuel Thibault wrote:
> John Paul Adrian Glaubitz, le lun. 17 août 2020 00:55:08 +0200, a ecrit:
>>> and then a couple other changes, I'm waiting for the above to be merged
>>> before I can submit them.
>>
>> What happens when a port does not have any packages in "unreleased"?
> 
> My debian-cd changes create empty files so that d-i is happy (exactly
> like is done for -backports).

OK, that's a possible workaround. However, ...

>> I checked the patch and it enables "unreleased" unconditionally. I'm
>> not sure whether that's a good idea.
> 
> I don't think we really have a better choice: writing yet another arch
> list looks dirty, and trying whether the suite is available is probably
> quite complex to implement.

... I don't quite understand why "unreleased" is supposed to be part of the
base installation. "unreleased" is not a regular package archive and all
packages in there are usually only added temporarily.

Does Debian Hurd require packages from "unreleased" for it's base system?

Since "unreleased" is not properly maintained due to the fact that all
packages there are built and uploaded manually, I rather prefer them not
to be enabled by default, especially not for the base system.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-16 Thread Samuel Thibault
John Paul Adrian Glaubitz, le lun. 17 août 2020 00:55:08 +0200, a ecrit:
> > and then a couple other changes, I'm waiting for the above to be merged
> > before I can submit them.
> 
> What happens when a port does not have any packages in "unreleased"?

My debian-cd changes create empty files so that d-i is happy (exactly
like is done for -backports).

> I checked the patch and it enables "unreleased" unconditionally. I'm
> not sure whether that's a good idea.

I don't think we really have a better choice: writing yet another arch
list looks dirty, and trying whether the suite is available is probably
quite complex to implement.

Samuel



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-16 Thread John Paul Adrian Glaubitz
Hi!

On 8/17/20 12:49 AM, Samuel Thibault wrote:
> AIUI it was only tested on the mini iso.

Well, then it should not have been merged :(.

> I noticed the issue, and worked on it, there is:
> 
> https://salsa.debian.org/images-team/debian-cd/-/merge_requests/7
> 
> and then a couple other changes, I'm waiting for the above to be merged
> before I can submit them.

What happens when a port does not have any packages in "unreleased"? I checked
the patch and it enables "unreleased" unconditionally. I'm not sure whether
that's a good idea.

FWIW, I have commit access to debian-cd, so I can merge your changes but I would
like to discuss first whether enabling "unreleased" unconditionally will work
even if a port doesn't have any package in this suite.

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-16 Thread Samuel Thibault
Hello,

John Paul Adrian Glaubitz, le lun. 17 août 2020 00:38:15 +0200, a ecrit:
> This seems to have broken debian-installer on Debian Ports.
> 
> Installing the base system now fails with:
> 
> Aug 16 22:34:35 debootstrap: /usr/sbin/debootstrap --components=main 
> --debian-installer --resolve-deps 
> --include=debian-ports-archive-keyring,debian-ports-archive-keyring 
> --extra-suites=unreleased --no-check-gpg sid /target file:///cdrom/  
> Aug 16 22:34:40 base-installer: error: exiting on error 
> base-installer/debootstrap-failed 
>   
> Aug 16 22:34:41 main-menu[235]: WARNING **: Configuring 'bootstrap-base' 
> failed with error code 1  
>  
> Aug 16 22:34:41 main-menu[235]: WARNING **: Menu item 'bootstrap-base' failed.
> 
> Has this actually been tested to work?

AIUI it was only tested on the mini iso.


I noticed the issue, and worked on it, there is:

https://salsa.debian.org/images-team/debian-cd/-/merge_requests/7

and then a couple other changes, I'm waiting for the above to be merged
before I can submit them.

Samuel



Re: Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)

2020-08-16 Thread John Paul Adrian Glaubitz
Hi!

This seems to have broken debian-installer on Debian Ports.

Installing the base system now fails with:

Aug 16 22:34:35 debootstrap: /usr/sbin/debootstrap --components=main 
--debian-installer --resolve-deps 
--include=debian-ports-archive-keyring,debian-ports-archive-keyring 
--extra-suites=unreleased --no-check-gpg sid /target file:///cdrom/  
Aug 16 22:34:40 base-installer: error: exiting on error 
base-installer/debootstrap-failed   

Aug 16 22:34:41 main-menu[235]: WARNING **: Configuring 'bootstrap-base' failed 
with error code 1   
Aug 16 22:34:41 main-menu[235]: WARNING **: Menu item 'bootstrap-base' failed.

Has this actually been tested to work?

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913