Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-12-12 Thread Mattia Rizzolo
This thread went OT talking about ports, but oh well…

On Wed, Dec 12, 2018 at 04:03:25AM +0100, Adam Borowski wrote:
> On Tue, Dec 11, 2018 at 09:46:21PM +0100, Gregor Riepl wrote:
> > The build and package delivery infrastructure should offer the same features
> > for both first and second class archs, including installer image building 
> > for
> > all "tiers" (stable, testing, unstable).
> 
> It seems to me that the important bit is the testing suite.  As a (now
> lapsed) x32 porter, I tried to implement that on my own (goal being an
> unofficial, weakly security supported[1] Jessie for x32).  And tracking
> testing on my own proved to be too hard.

Pretty much everything of what Adrian is mentioning (above is part of
it), would be fixed if the ports archive were to move to using a full
dak instance.

I know that James was working on adding the few missing features that
would be ports-specific, but I haven't heard an update on that work
since a while.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature


Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-12-11 Thread Adam Borowski
[Oy vey, crosspost list from hell -- not sure how to trim...]

On Tue, Dec 11, 2018 at 09:46:21PM +0100, Gregor Riepl wrote:
> I do think this just reinforces the point that second-class architectures
> should have better, more robust support from the Debian project.

> For example, arch-specific packages most decidedly have a place in Debian

> The build and package delivery infrastructure should offer the same features
> for both first and second class archs, including installer image building for
> all "tiers" (stable, testing, unstable).

It seems to me that the important bit is the testing suite.  As a (now
lapsed) x32 porter, I tried to implement that on my own (goal being an
unofficial, weakly security supported[1] Jessie for x32).  And tracking
testing on my own proved to be too hard.  What directly defeated me were
binNMUs: with every arch having its own NMU counter and hidden triggers,
this is already a mess.  Add NMUs due to private ported packages, and all
hell breaks loose.

The rest is easy in comparison: a porter team can decide whether to snapshot
testing as unofficial stable; point releases are a matter of running a
buildd job (and fixing failures), same for security.  We'd be able to
concentrate on actual arch-specific issues.

> The main difference should (IMHO) be the amount of support you get: While a
> first-class arch will get faster fixes and a more stable dependency tree,
> other archs will be more "sloppy", for example by not blocking stable releases
> with their own RC bugs etc.

Yeah, a completely one-way relationship: no issue on second-class would
block first-class.

> If this can be fulfilled, I don't think being a second-class arch will be such
> a big deal. Not sure how far Debian is from this goal, but I can understand
> that many DDs and DMs would rather invest their time into projects they have a
> stake in, rather than hardware they don't (or don't want to?) understand.

Yes, x32 suffers from needing obscure and hard to get hardware. :)


Meow!

[1]. The vast majority of security issues are non arch dependent, so blindly
tracking and building first-class security updates gets us nearly all the
way, reducing the work to babysitting buildds and looking into FTBFSes or
yet another whole-new-language-ecosystem getting allowed into stable.
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ Ivan was a worldly man: born in St. Petersburg, raised in
⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned
⠈⠳⣄ to the city of his birth to die.



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-12-11 Thread Gregor Riepl
Hi Adrian

I do think this just reinforces the point that second-class architectures
should have better, more robust support from the Debian project.

For example, arch-specific packages most decidedly have a place in Debian
(although they should not be the norm). There will always be such packages, as
proven by many that are available on first-class archs but not on second-class
ones (protobuf springs to mind:
https://buildd.debian.org/status/package.php?p=protobuf).

The build and package delivery infrastructure should offer the same features
for both first and second class archs, including installer image building for
all "tiers" (stable, testing, unstable).

The main difference should (IMHO) be the amount of support you get: While a
first-class arch will get faster fixes and a more stable dependency tree,
other archs will be more "sloppy", for example by not blocking stable releases
with their own RC bugs etc.

If this can be fulfilled, I don't think being a second-class arch will be such
a big deal. Not sure how far Debian is from this goal, but I can understand
that many DDs and DMs would rather invest their time into projects they have a
stake in, rather than hardware they don't (or don't want to?) understand.

Regards,
Greg



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-12-11 Thread John Paul Adrian Glaubitz
Hello!

On 12/9/18 3:18 PM, Matthias Klose wrote:
> To me it looks sometimes that Debian is used for testing by upstream, and for
> that the mips architectures don't need to be release architectures.

A note on this: If you decide to move MIPS to Debian Ports, you will make the
port unusable to most users as Debian Ports has a rather rudimentary FTP archive
setup which has some annoying side effects.

There is no support for a testing release, there is no support for cruft and the
FTP maintainers will eventually remove any MIPS-only packages from the Debian
archive which don't build on other architectures which usually affects packages
like boot loaders meaning that it will no longer be easily possible to build the
debian-installer package and consequently build installation images. The 32-bit
PowerPC port lost quite a number of users because of this change. Not because 
the
port was not healthy but because people want to be able to install a stable 
release.

Debian unfortunately doesn't have really good support for Tier II 
architectures, it's
either release or something based on unstable that requires extra elbow grease 
from
both users and maintainers.

Please also keep in mind that removing MIPS from the list of release 
architectures
would mean one less open platform on which Debian is supported. Neither anything
based on ARM, x86 or IBM Z provides a true open platform due to the proprietary
nature of these architectures. There are some efforts in this regard on IBM 
POWER,
but the hardware is still rather expensive, unfortunately. I do hope that RISC-V
will catch up in the future though.

I also think that the broad architecture support is one of the selling points 
of Debian
and if we were to limit Debian's architecture support to just ARM, x86, POWER 
and IBM Z,
I fear that Debian would more and more be turned into a mere development 
project for Ubuntu
and other derivatives rather than being an operating system of its own.

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-12-09 Thread Matthias Klose
On 07.07.18 17:24, YunQiang Su wrote:
> Niels Thykier  于2018年6月28日周四 上午4:06写道:
>> List of concerns for architectures
>> ==
>>
>> The following is a summary from the current architecture qualification
>> table.
>>
>>  * Concern for ppc64el and s390x: we are dependent on sponsors for
>>hardware.
>>(Raised by DSA; carried over from stretch)
>>
>>  * Concern for armel and armhf: only secondary upstream support in GCC
>>(Raised by the GCC maintainer; carried over from stretch)

I don't think anybody objected about the status for armhf.  I didn't follow
armel issues too closely.

>>  * Concern for mips, mips64el, mipsel and ppc64el: no upstream support
>>in GCC
>>(Raised by the GCC maintainer; carried over from stretch)
>>
> 
> This is a misunderstanding as MIPS company had some unrest in recent half 
> year.
> Currently we are stable now, and the shape of gcc upstream is also good.

This is an optimistic view.  While currently not having any RC issues, we still
see mips* specific issues popping up more often than on other release 
architectures.

According to https://gcc.gnu.org/gcc-8/criteria.html there is no mips*-linux*
target listed as either primary or secondary platform. As far as I understand
the mips porters argue that this is covered by mipsisa64-elf, a bare metal
target.  I don't agree with this view, because

 - testing is missing on mips*-linux-* targets.  If you look at
   the gcc-testresults ML, you see only test reports submitted for
   the Debian GCC packages, nothing else.

 - A bare metal target is usually only built/used for C and C++. I
   doubt that other frontends are tested.

 - Configurations like libgcjit are not tested/used upstream, and not
   addressed. See #798710.

The Debian bug tracking for the MIPS port could be better, I usually need some
pings to the MIPS porters to get things forwarded or addressed.

To me it looks sometimes that Debian is used for testing by upstream, and for
that the mips architectures don't need to be release architectures.

Matthias



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-07-07 Thread YunQiang Su
Niels Thykier  于2018年6月28日周四 上午4:06写道:
>
> Hi,
>
>
> As part of the interim architecture qualification for buster, we request
> that DSA, the security team and the toolchain maintainers review and
> update their list of known concerns for buster release architectures.
>
> Summary of the current concerns and issues:
>  * DSA have announced a blocking issue for armel and armhf (see below)
>  * Concerns from DSA about ppc64el and s390x have been carried over from
>stretch.
>  * Concerns from the GCC maintainers about armel, armhf, mips, mips64el
>and mipsel have been carried over from stretch.
>
> If the issues and concerns from you or your team are not up to date,
> then please follow up to this email (keeping debian-release@l.d.o and
> debian-ports@l.d.o in CC to ensure both parties are notified).
>
> Whilst porters remain ultimately responsible for ensuring the
> architectures are ready for release, we do expect that you / your team
> are willing to assist with clarifications of the concerns and to apply
> patches/changes in a timely manner to resolve the concerns.
>
>
> List of blocking issues by architecture
> ===
>
> The following is a summary from the current architecture qualification
> table.
>
> armel/armhf:
> 
>
>  * Undesirable to keep the hardware running beyond 2020.  armhf VM
>support uncertain. (DSA)
>- Source: [DSA Sprint report]
>
>
> [DSA Sprint report]:
> https://lists.debian.org/debian-project/2018/02/msg4.html
>
>
> List of concerns for architectures
> ==
>
> The following is a summary from the current architecture qualification
> table.
>
>  * Concern for ppc64el and s390x: we are dependent on sponsors for
>hardware.
>(Raised by DSA; carried over from stretch)
>
>  * Concern for armel and armhf: only secondary upstream support in GCC
>(Raised by the GCC maintainer; carried over from stretch)
>
>  * Concern for mips, mips64el, mipsel and ppc64el: no upstream support
>in GCC
>(Raised by the GCC maintainer; carried over from stretch)
>

This is a misunderstanding as MIPS company had some unrest in recent half year.
Currently we are stable now, and the shape of gcc upstream is also good.

>
> Architecture status
> ===
>
> These are the architectures currently being built for buster:
>
>  * Intel/AMD-based: amd64, i386
>  * ARM-based: arm64, armel, armhf
>  * MIPS-based: mips, mipsel, mips64el

We are plan to drop mips(eb) and keep mipsel/mips64el.

>  * Other: ppc64el, s390x
>
> If the blocking issues cannot be resolved, affected architectures are at
> risk of removal from testing before buster is frozen.
>
> We are currently unaware of any new architectures likely to be ready in
> time for inclusion in buster.
>
> On behalf of the release team,
> Niels Thykier
>


-- 
YunQiang Su



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Florian Weimer
* Riku Voipio:

> On Thu, Jun 28, 2018 at 08:11:14PM +0200, Florian Weimer wrote:
>> * Niels Thykier:
>> 
>> > armel/armhf:
>> > 
>> >
>> >  * Undesirable to keep the hardware running beyond 2020.  armhf VM
>> >support uncertain. (DSA)
>> >- Source: [DSA Sprint report]
>> 
>> Fedora is facing an issue running armhf under virtualization on arm64:
>> 
>>   
>
> I think you mean:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1576593

Yes, that's right, I copy-pasted the wrong bug URL.  It's filed as a
Red Hat Enterprise Linux bug because the Fedora builders run Fedora
VMs on that platform.



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Lennart Sorensen
On Fri, Jun 29, 2018 at 10:20:50AM +0100, Luke Kenneth Casson Leighton wrote:
>  in addition, arm64 is usually speculative OoO (Cavium ThunderX V1
> being a notable exception) which means it's vulnerable to spectre and
> meltdown attacks, whereas 32-bit ARM is exclusively in-order.  if you
> want to GUARANTEE that you've got spectre-immune hardware you need
> either any 32-bit system (where even Cortex A7 has virtualisattion) or
> if 64-bit is absolutely required use Cortex A53.

The Cortex A8, A7 and A5 are in order.  The A9, A15, A17 etc are out of
order execution.  So any 32 bit arm worth using is pretty much always
out of order execution.

For 64 bit, I think the A35 and A53 are in order while the A57, A72 etc
are out of order.

Of course non Cortex designs vary (I think Marvel's JP4 core was out of
order execution for example).

After all, in general in order execution equals awful performance.

-- 
Len Sorensen



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Luke Kenneth Casson Leighton
On Fri, Jun 29, 2018 at 12:50 PM, Julien Cristau  wrote:

> Everyone, please avoid followups to debian-po...@lists.debian.org.
> Unless something is relevant to *all* architectures (hint: discussion of
> riscv or arm issues don't qualify), keep replies to the appropriate
> port-specific mailing list.

 apologies, julien: as an outsider i'm not completely familiar with
the guidelines.  the reduction in memory-usage at the linker phase
"-Wl,--no-keep-memory" however - and the associated inherent
slowly-inexorably-increasing size is i feel definitely something that
affects all ports.

 it is really *really* tricky to get any kind of traction *at all*
with people on this.  it's not gcc's problem to solve, it's not one
package's problem to solve, it's not any one distros problem to solve,
it's not any one port's problem to solve and so on, *and* it's a
slow-burn problem that's taking *literally* a decade to become more
and more of a problem.  consequently getting reports and feedback to
the binutils team is... damn hard.

l.



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Julien Cristau
On 06/27/2018 10:03 PM, Niels Thykier wrote:
> Hi,
> 
> 
> As part of the interim architecture qualification for buster, we request
> that DSA, the security team and the toolchain maintainers review and
> update their list of known concerns for buster release architectures.
> 
Everyone, please avoid followups to debian-po...@lists.debian.org.
Unless something is relevant to *all* architectures (hint: discussion of
riscv or arm issues don't qualify), keep replies to the appropriate
port-specific mailing list.

Thanks,
Julien



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Luke Kenneth Casson Leighton
On Fri, Jun 29, 2018 at 12:23 PM, Adam D. Barratt
 wrote:

>>  i don't know: i'm an outsider who doesn't have the information in
>> short-term memory, which is why i cc'd the debian-riscv team as they
>> have current facts and knowledge foremost in their minds.  which is
>> why i included them.
>
> It would have been wiser to do so *before* stating that nothing was
> happening as if it were a fact.

 true... apologies.

>>  ah.  so what you're saying is, you could really do with some extra
>> help?
>
> I don't think that's ever been in dispute for basically any core team
> in Debian.

 :)

> That doesn't change the fact that the atmosphere around the change in
> question has made me feel very uncomfortable and unenthused about SRM
> work. (I realise that this is somewhat of a self-feeding energy
> monster.)

 i hear ya.

l.



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Adam D. Barratt
On Fri, 2018-06-29 at 11:44 +0100, Luke Kenneth Casson Leighton wrote:
[...]
> On Fri, Jun 29, 2018 at 10:35 AM, Adam D. Barratt
>  wrote:
> 
> > >  what is the reason why that package is not moving forward?
> > 
> > I assume you're referring to the dpkg upload that's in proposed-
> > updates
> > waiting for the point release in two weeks time?
> 
>  i don't know: i'm an outsider who doesn't have the information in
> short-term memory, which is why i cc'd the debian-riscv team as they
> have current facts and knowledge foremost in their minds.  which is
> why i included them.

It would have been wiser to do so *before* stating that nothing was
happening as if it were a fact.

> > I'm also getting very tired of the repeated vilification of SRM
> > over
> > this, and if there were any doubt can assure you that it is not
> > increasing at least my inclination to spend my already limited free
> > time on Debian activity.
> 
>  ah.  so what you're saying is, you could really do with some extra
> help?

I don't think that's ever been in dispute for basically any core team
in Debian.

That doesn't change the fact that the atmosphere around the change in
question has made me feel very uncomfortable and unenthused about SRM
work. (I realise that this is somewhat of a self-feeding energy
monster.)

Regards,

Adam



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Luke Kenneth Casson Leighton
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68


On Fri, Jun 29, 2018 at 10:35 AM, Adam D. Barratt
 wrote:

>>  what is the reason why that package is not moving forward?
>
> I assume you're referring to the dpkg upload that's in proposed-updates
> waiting for the point release in two weeks time?

 i don't know: i'm an outsider who doesn't have the information in
short-term memory, which is why i cc'd the debian-riscv team as they
have current facts and knowledge foremost in their minds.  which is
why i included them.

> I'm also getting very tired of the repeated vilification of SRM over
> this, and if there were any doubt can assure you that it is not
> increasing at least my inclination to spend my already limited free
> time on Debian activity.

 ah.  so what you're saying is, you could really do with some extra help?

l.



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Adam D. Barratt
On Fri, 2018-06-29 at 10:20 +0100, Luke Kenneth Casson Leighton wrote:
[...]
>  debian-riscv has been repeatedly asking for a single zero-impact
> line
> to be included in *one* file in *one* dpkg-related package which
> would
> allow riscv to stop being a NMU architecture and become part of
> debian/unstable (and quickly beyond), for at least six months, now.
> cc'ing the debian-riscv list because they will know the details about
> this.  it's really quite ridiculous that a single one-line change
> having absolutely no effect on any other architecture whatsover is
> not
> being actioned and is holding debian-riscv back because of that.
> 
>  what is the reason why that package is not moving forward?

I assume you're referring to the dpkg upload that's in proposed-updates 
waiting for the point release in two weeks time? Please check your
facts before ranting, particularly with such a wide cross-posting.

Also, ttbomk, the dpkg change landing in stable is not likely to
magically lead to the architecture being added to unstable - a decision
which is not the release team's to make or block. Again, please ensure
you've actually done your research.

I'm also getting very tired of the repeated vilification of SRM over
this, and if there were any doubt can assure you that it is not
increasing at least my inclination to spend my already limited free
time on Debian activity.

Regards,

Adam



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Luke Kenneth Casson Leighton
On Wed, Jun 27, 2018 at 9:03 PM, Niels Thykier  wrote:

> armel/armhf:
> 
>
>  * Undesirable to keep the hardware running beyond 2020.  armhf VM
>support uncertain. (DSA)
>- Source: [DSA Sprint report]

 [other affected 32-bit architectures removed but still relevant]

 ... i'm not sure how to put this other than to just ask the question.
has it occurred to anyone to think through the consequences of not
maintaining 32-bit versions of debian for the various different
architectures?  there are literally millions of ARM-based tablets and
embedded systems out there which will basically end up in landfill if
a major distro such as debian does not take a stand and push back
against the "well everything's going 64-bit so why should *we*
bother?" meme.

 arm64 is particularly inefficient and problematic compared to
aarch32: the change in the instruction set resulted in dropping some
of the more efficiently-encoded instructions that extend a 64-bit
program size, require a whopping FIFTY PERCENT instruction-cache size
increase to compensate for, whch increased power consumption by over
15%.

 in addition, arm64 is usually speculative OoO (Cavium ThunderX V1
being a notable exception) which means it's vulnerable to spectre and
meltdown attacks, whereas 32-bit ARM is exclusively in-order.  if you
want to GUARANTEE that you've got spectre-immune hardware you need
either any 32-bit system (where even Cortex A7 has virtualisattion) or
if 64-bit is absolutely required use Cortex A53.

 basically, abandoning or planning to abandon 32-bit ARM *right now*
leaves security-conscious end-users in a really *really* dicey
position.


> We are currently unaware of any new architectures likely to be ready in
> time for inclusion in buster.

 debian-riscv has been repeatedly asking for a single zero-impact line
to be included in *one* file in *one* dpkg-related package which would
allow riscv to stop being a NMU architecture and become part of
debian/unstable (and quickly beyond), for at least six months, now.
cc'ing the debian-riscv list because they will know the details about
this.  it's really quite ridiculous that a single one-line change
having absolutely no effect on any other architecture whatsover is not
being actioned and is holding debian-riscv back because of that.

 what is the reason why that package is not moving forward?

 l.



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-29 Thread Riku Voipio
On Thu, Jun 28, 2018 at 08:11:14PM +0200, Florian Weimer wrote:
> * Niels Thykier:
> 
> > armel/armhf:
> > 
> >
> >  * Undesirable to keep the hardware running beyond 2020.  armhf VM
> >support uncertain. (DSA)
> >- Source: [DSA Sprint report]
> 
> Fedora is facing an issue running armhf under virtualization on arm64:
> 
>   

I think you mean:

https://bugzilla.redhat.com/show_bug.cgi?id=1576593

>   
> 
> Unless the discussion has moved somewhere where I can't follow it, no
> one seems to have solid idea what is going on. 

True. Looking at comment #22, the suggestion seems to be that the guest is
doing something wrong, and kvm is being terrible at pinpointing the source.

> It's also not clear that this configuration has substantial vendor or
> community support. This makes me concerned that virtualization is a viable
> path forward here.

I understand your concern. It would be surprising if this specific bug doesn't
get found and fixed. It's probably stuck because everyone thinks it's 
probably someone elses bug ;)

I still think the armhf vm on arm64 is the only reasonable path forward medium
term. The existing arm64 hw that suport arm32 vm's is still around and
infinitely better than native aarch32 builder hw, and should still be viable
for some time. 

Riku



Re: Arch qualification for buster: call for DSA, Security, toolchain concerns

2018-06-28 Thread Florian Weimer
* Niels Thykier:

> armel/armhf:
> 
>
>  * Undesirable to keep the hardware running beyond 2020.  armhf VM
>support uncertain. (DSA)
>- Source: [DSA Sprint report]

Fedora is facing an issue running armhf under virtualization on arm64:

  
  

Unless the discussion has moved somewhere where I can't follow it, no
one seems to have solid idea what is going on.  It's also not clear
that this configuration has substantial vendor or community support.
This makes me concerned that virtualization is a viable path forward
here.

(The discussion on the GCC list started off with a misdirection, sorry
about that.  The brief assumption that this was a hardware quirk is
likely quite wrong.)

>  * Concern for mips, mips64el, mipsel and ppc64el: no upstream support
>in GCC
>(Raised by the GCC maintainer; carried over from stretch)

I'm surprised to read this.  ppc64el features prominently in the
toolchain work I do (though I personally do not work on the GCC side).
>From my point of view, it's absolutely not in the same category as the
MIPS-based architectures.