Bug#1050872: release.debian.org: mips64el apparently relies on buster (oldoldstable) LTS?

2023-10-28 Thread Luca Boccassi
On Sun, 3 Sep 2023 13:20:48 +0300 Adrian Bunk  wrote:
> On Wed, Aug 30, 2023 at 04:54:01PM +0100, Simon McVittie wrote:
> >...
> > However, if I understand correctly, Luca has been told that some
official
> > mips64el buildds are running mipsel user-space on mips64el hardware
which
> > only works with the buster kernel,
> >...
> > Is this true?
> 
> Yes:
> https://lists.debian.org/debian-mips/2023/07/msg00052.html
> 
> >...
> > If our official buildds for a release architecture are unable to
run on
> > either the stable or oldstable kernel, I think that raises some
important
> > questions about suitability for inclusion in future releases.
> >...
> 
> This problem affects older (and slower) buildds, the newer buildds 
> don't run with any kernel older than oldstable.
> 
> Turning off the affected buildds is an option.

Hi,

Any news on this topic? Debootstrap has been updated in p-u and o-p-u,
do we need to worry about Buster, or where those mips buildds retired?

-- 
Kind regards,
Luca Boccassi


signature.asc
Description: This is a digitally signed message part


Bug#1050872: release.debian.org: mips64el apparently relies on buster (oldoldstable) LTS?

2023-09-03 Thread Adrian Bunk
On Wed, Aug 30, 2023 at 04:54:01PM +0100, Simon McVittie wrote:
>...
> However, if I understand correctly, Luca has been told that some official
> mips64el buildds are running mipsel user-space on mips64el hardware which
> only works with the buster kernel,
>...
> Is this true?

Yes:
https://lists.debian.org/debian-mips/2023/07/msg00052.html

>...
> If our official buildds for a release architecture are unable to run on
> either the stable or oldstable kernel, I think that raises some important
> questions about suitability for inclusion in future releases.
>...

This problem affects older (and slower) buildds, the newer buildds 
don't run with any kernel older than oldstable.

Turning off the affected buildds is an option.

>...
> >From the point of view of the project having control over its own future,
> I would have hoped that official Debian infrastructure would only be using
> suites that are supported by Debian as a whole,
>...

This problem is not limited to mips:
https://nagios.debian.org/cgi-bin/icinga/status.cgi?hostgroup=buster=overview

E.g. the Python2 removal in bullseye removed planet-venus,
which was never ported to Python3 but is used by planet.debian.org.
A replacement is being written in Perl, but it is still WIP.

> Thanks,
> smcv

cu
Adrian



Bug#1050872: release.debian.org: mips64el apparently relies on buster (oldoldstable) LTS?

2023-08-31 Thread Simon McVittie
On Thu, 31 Aug 2023 at 08:30:00 +0200, Aurelien Jarno wrote:
> This has already been discussed on IRC that you should not worry about
> debootstrap for buster, provide you leave us ~1 month. Not sure yet what
> we'll do, it could be your option, stopping the buildds as discussed on
> IRC, or maybe yet another option.

Thank you! (I was not part of that IRC conversation.)

Some of the forward progress with DEP-17 is blocked by getting an updated
debootstrap into 12.2 and 11.8, which are planned for 2023-10-07 if I
understand correctly, so resolving this within the next month would take
mips64el off the critical path.

Another option that occurred to me would be that the affected buildds
could perhaps be upgraded to bullseye user-space, but with the kernel
metapackage held back to buster's? Obviously that's not ideal (no
kernel security updates) but at least that way they'd have user-space
security updates to components like sshd, together with the same
schroot/sbuild/debootstrap behaviours as most of the other architectures'
buildds.

smcv



Bug#1050872: release.debian.org: mips64el apparently relies on buster (oldoldstable) LTS?

2023-08-31 Thread Aurelien Jarno
Hi,

On 2023-08-31 00:29, Simon McVittie wrote:
> On Wed, 30 Aug 2023 at 21:26:24 +0200, Aurelien Jarno wrote:
> > On 2023-08-30 16:54, Simon McVittie wrote:
> > > Luca Boccassi and I have been preparing stable and oldstable updates for
> > > debootstrap so that the transition described in DEP-17 can continue.
> > > Because DEP-17 involves changes to trixie/sid chroots' bootstrap
> > > procedures, the updated debootstrap needs to be deployable to every
> > > official buildd
> >
> > We have issues running [bookworm?] and bullseye kernels on
> > some arm32 and mips*el buildds. The problem on arm has been solved by
> > decommissioning the hardware or by hosts dying. We still have problems
> > with a big part of the mips*el hosts.
> 
> Would it be possible to make an exception to the usual rule that buildds
> get their debootstrap from (old)stable point releases, and manually
> install a newer debootstrap (the version proposed for bullseye should
> be suitable) onto the affected mips*el machines? I see that they already
> have a newer-than-buster version of sbuild, possibly from the
> buster-backports suite (which was discontinued when buster was handed over
> to the LTS team).
> 
> I would prefer not to spend time preparing and testing a special buster
> version of debootstrap and negotiating with the Debian 10 LTS team to get
> it into buster/updates in the security archive; and it's not clear to
> me that there is actually any apt repository that we could put it into
> that would be accepted by the affected buildds, because buster is read-only
> in the main Debian archive, and debian-security no longer has
> dists/buster/updates/main/binary-mips64el at all?
> 
> (debian-security does have binary-all, and debootstrap is Architecture:
> all, but I'm not sure how much that would help us with buster's apt, since
> separate Packages files for binary-all seem to be a relatively new thing.)

This has already been discussed on IRC that you should not worry about
debootstrap for buster, provide you leave us ~1 month. Not sure yet what
we'll do, it could be your option, stopping the buildds as discussed on
IRC, or maybe yet another option.

> > > From the point of view of the project having control over its own future,
> > > I would have hoped that official Debian infrastructure would only be using
> > > suites that are supported by Debian as a whole, rather than handing over
> > > control and responsibility to the Debian-LTS subproject.
> 
> Sam Hartman pointed out on #debian-devel that this is worse than I thought,
> because Debian-LTS doesn't cover mips*el. So as far as I can see, there is
> no channel that gets security updates onto these buildds at all?

Yes this is correct for mips*el. We still have debian-lts for the arm
hosts though.

Regards
Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://aurel32.net


signature.asc
Description: PGP signature


Bug#1050872: release.debian.org: mips64el apparently relies on buster (oldoldstable) LTS?

2023-08-30 Thread Simon McVittie
On Wed, 30 Aug 2023 at 21:26:24 +0200, Aurelien Jarno wrote:
> On 2023-08-30 16:54, Simon McVittie wrote:
> > Luca Boccassi and I have been preparing stable and oldstable updates for
> > debootstrap so that the transition described in DEP-17 can continue.
> > Because DEP-17 involves changes to trixie/sid chroots' bootstrap
> > procedures, the updated debootstrap needs to be deployable to every
> > official buildd
>
> We have issues running [bookworm?] and bullseye kernels on
> some arm32 and mips*el buildds. The problem on arm has been solved by
> decommissioning the hardware or by hosts dying. We still have problems
> with a big part of the mips*el hosts.

Would it be possible to make an exception to the usual rule that buildds
get their debootstrap from (old)stable point releases, and manually
install a newer debootstrap (the version proposed for bullseye should
be suitable) onto the affected mips*el machines? I see that they already
have a newer-than-buster version of sbuild, possibly from the
buster-backports suite (which was discontinued when buster was handed over
to the LTS team).

I would prefer not to spend time preparing and testing a special buster
version of debootstrap and negotiating with the Debian 10 LTS team to get
it into buster/updates in the security archive; and it's not clear to
me that there is actually any apt repository that we could put it into
that would be accepted by the affected buildds, because buster is read-only
in the main Debian archive, and debian-security no longer has
dists/buster/updates/main/binary-mips64el at all?

(debian-security does have binary-all, and debootstrap is Architecture:
all, but I'm not sure how much that would help us with buster's apt, since
separate Packages files for binary-all seem to be a relatively new thing.)

> > From the point of view of the project having control over its own future,
> > I would have hoped that official Debian infrastructure would only be using
> > suites that are supported by Debian as a whole, rather than handing over
> > control and responsibility to the Debian-LTS subproject.

Sam Hartman pointed out on #debian-devel that this is worse than I thought,
because Debian-LTS doesn't cover mips*el. So as far as I can see, there is
no channel that gets security updates onto these buildds at all?

smcv



Bug#1050872: release.debian.org: mips64el apparently relies on buster (oldoldstable) LTS?

2023-08-30 Thread Aurelien Jarno
Hi,

On 2023-08-30 16:54, Simon McVittie wrote:
> Package: release.debian.org
> Severity: normal
> X-Debbugs-Cc: debian-m...@lists.debian.org, bl...@debian.org, 
> debian-wb-t...@lists.debian.org
> 
> Luca Boccassi and I have been preparing stable and oldstable updates for
> debootstrap so that the transition described in DEP-17 can continue.
> Because DEP-17 involves changes to trixie/sid chroots' bootstrap
> procedures, the updated debootstrap needs to be deployable to every
> official buildd, and we've been told that (old)stable point releases
> are the preferred way to achieve that.
> 
> When Luca asked how many suites we needed this change in, we were hoping
> the answer would be stable only, and maybe oldstable (which is still
> in its 1 year of overlapping support from the security team and DDs
> in general).
> 
> However, if I understand correctly, Luca has been told that some official
> mips64el buildds are running mipsel user-space on mips64el hardware which
> only works with the buster kernel, and therefore those official buildds
> are still stuck on buster, meaning we also need to prepare a buster
> version of debootstrap and get it into Debian 10 LTS via buster-security.
> 
> Is this true?

This is correct. We have issues running buster and bullseye kernels on
some arm32 and mips*el buildds. The problem on arm has been solved by
decommissioning the hardware or by hosts dying. We still have problems
with a big part of the mips*el hosts.

> >From the point of view of the project having control over its own future,
> I would have hoped that official Debian infrastructure would only be using
> suites that are supported by Debian as a whole, rather than handing over
> control and responsibility to the Debian-LTS subproject.

That's also our wishes. Unfortunately real life makes things more
difficult, even on x86. And not speaking about buildds, some services
also do not run on newer debian releases.

> Also, from the point of view of continued development of testing/unstable,
> I would have hoped that packages in testing/unstable could safely
> assume that they will run on at least the kernel from stable (or maybe
> oldstable for a short time after a new stable release), following our
> usual "skipping a release is unsupported" rule. Obviously if the buildds
> are running on an oldoldstable kernel, any mips64el package that might
> be used at compile-time or for build-time tests will be unable to make
> that assumption.

Yes, this is definitely problem, and it has already been reported.

> Please could someone with knowledge of the buildds clarify the situation?
> 
> If our official buildds for a release architecture are unable to run on
> either the stable or oldstable kernel, I think that raises some important
> questions about suitability for inclusion in future releases.

I don't think this assumption should change, instead we should have a
way to upgrade those hosts. Probably we'll just decommission them
instead.

Regards
Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://aurel32.net



Bug#1050872: release.debian.org: mips64el apparently relies on buster (oldoldstable) LTS?

2023-08-30 Thread Simon McVittie
Package: release.debian.org
Severity: normal
X-Debbugs-Cc: debian-m...@lists.debian.org, bl...@debian.org, 
debian-wb-t...@lists.debian.org

Luca Boccassi and I have been preparing stable and oldstable updates for
debootstrap so that the transition described in DEP-17 can continue.
Because DEP-17 involves changes to trixie/sid chroots' bootstrap
procedures, the updated debootstrap needs to be deployable to every
official buildd, and we've been told that (old)stable point releases
are the preferred way to achieve that.

When Luca asked how many suites we needed this change in, we were hoping
the answer would be stable only, and maybe oldstable (which is still
in its 1 year of overlapping support from the security team and DDs
in general).

However, if I understand correctly, Luca has been told that some official
mips64el buildds are running mipsel user-space on mips64el hardware which
only works with the buster kernel, and therefore those official buildds
are still stuck on buster, meaning we also need to prepare a buster
version of debootstrap and get it into Debian 10 LTS via buster-security.

Is this true?

>From the point of view of the project having control over its own future,
I would have hoped that official Debian infrastructure would only be using
suites that are supported by Debian as a whole, rather than handing over
control and responsibility to the Debian-LTS subproject.

Also, from the point of view of continued development of testing/unstable,
I would have hoped that packages in testing/unstable could safely
assume that they will run on at least the kernel from stable (or maybe
oldstable for a short time after a new stable release), following our
usual "skipping a release is unsupported" rule. Obviously if the buildds
are running on an oldoldstable kernel, any mips64el package that might
be used at compile-time or for build-time tests will be unable to make
that assumption.

Please could someone with knowledge of the buildds clarify the situation?

If our official buildds for a release architecture are unable to run on
either the stable or oldstable kernel, I think that raises some important
questions about suitability for inclusion in future releases.

Thanks,
smcv