Re: Porter roll call for Debian Stretch

[Adrian Bunk]

On Sun, Oct 09, 2016 at 11:13:21PM +0100, Adam D. Barratt wrote:
> On Sun, 2016-10-09 at 21:12 +0300, Adrian Bunk wrote:
> > [ adding debian-powerpc ]
> > 
> > On Sun, Oct 09, 2016 at 06:54:44PM +0200, Moritz Mühlenhoff wrote:
> > > Niels Thykier  schrieb:
> > > > If I am to support powerpc as a realease architecture for Stretch, I
> > > > need to know that there are *active* porters behind it committed to
> > > > keeping it in the working.  People who would definitely catch such
> > > > issues long before the release.  People who file bugs / submit patches 
> > > > etc.
> > > 
> > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=832931 is about
> > > a powerpc-specific build failure of mariadb in stable. The maintainer
> > > said he can't work on it, so if anyone considers himself/herself a
> > > powerpc porter, this is something to look it.
> > 
> > Can you give a hint what exactly should be looked at?
> 
> https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=powerpc&ver=10.0.27-0%2Bdeb8u1&stamp=1473621159
> 
> > The bug did not make it clear that there is any problem left at all when 
> > I looked at it recently.
> > 
> > The last message was closing the bug.
> > 
> > There was a control command reopening the bug without giving any 
> > rationale, but the last control command was
> >   fixed 832931 10.0.27-1
> 
> For unstable, yes. The stable package is still broken.

Thanks.

When skimming through that the bug I thought it was fixed in 
upstream 10.0.27, and no comment in the bug indicated otherwise.

As (pretty passive) reader of debian-powerpc, I am also puzzled by the 
fact that I cannot recall #832931 ever being mentioned there.

An email to the bug and the mailing list of the port stating that there 
is a problem, and what is known about it, can be really helpful for 
getting such a bug resolved.


> > buildd.debian.org says that 10.0.27-0+deb8u1 was installed on jessie.[1]
> 
> That's an artefact of how builds for suites with "overlays" (i.e. pu /
> tpu) are displayed. If one actually looks at the archive:
> 
> mariadb-client-10.0 | 10.0.25-0+deb8u1 | stable | powerpc
> mariadb-client-10.0 | 10.0.27-0+deb8u1 | stable | amd64, arm64, armel, 
> armhf, i386, mips, mipsel, ppc64el, s390x

Ouch.

I thought the information in the version tracking was wrong
when buildd.d.o showed green.


> Regards,
> 
> Adam

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Porter roll call for Debian Stretch

[Adam D. Barratt]

On Sun, 2016-10-09 at 21:12 +0300, Adrian Bunk wrote:
> [ adding debian-powerpc ]
> 
> On Sun, Oct 09, 2016 at 06:54:44PM +0200, Moritz Mühlenhoff wrote:
> > Niels Thykier  schrieb:
> > > If I am to support powerpc as a realease architecture for Stretch, I
> > > need to know that there are *active* porters behind it committed to
> > > keeping it in the working.  People who would definitely catch such
> > > issues long before the release.  People who file bugs / submit patches 
> > > etc.
> > 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=832931 is about
> > a powerpc-specific build failure of mariadb in stable. The maintainer
> > said he can't work on it, so if anyone considers himself/herself a
> > powerpc porter, this is something to look it.
> 
> Can you give a hint what exactly should be looked at?

https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=powerpc&ver=10.0.27-0%2Bdeb8u1&stamp=1473621159

> The bug did not make it clear that there is any problem left at all when 
> I looked at it recently.
> 
> The last message was closing the bug.
> 
> There was a control command reopening the bug without giving any 
> rationale, but the last control command was
>   fixed 832931 10.0.27-1

For unstable, yes. The stable package is still broken.

> buildd.debian.org says that 10.0.27-0+deb8u1 was installed on jessie.[1]

That's an artefact of how builds for suites with "overlays" (i.e. pu /
tpu) are displayed. If one actually looks at the archive:

mariadb-client-10.0 | 10.0.25-0+deb8u1 | stable | powerpc
mariadb-client-10.0 | 10.0.27-0+deb8u1 | stable | amd64, arm64, armel, 
armhf, i386, mips, mipsel, ppc64el, s390x

Regards,

Adam

Re: Porter roll call for Debian Stretch

[Adrian Bunk]

[ adding debian-powerpc ]

On Sun, Oct 09, 2016 at 06:54:44PM +0200, Moritz Mühlenhoff wrote:
> Niels Thykier  schrieb:
> > If I am to support powerpc as a realease architecture for Stretch, I
> > need to know that there are *active* porters behind it committed to
> > keeping it in the working.  People who would definitely catch such
> > issues long before the release.  People who file bugs / submit patches etc.
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=832931 is about
> a powerpc-specific build failure of mariadb in stable. The maintainer
> said he can't work on it, so if anyone considers himself/herself a
> powerpc porter, this is something to look it.

Can you give a hint what exactly should be looked at?

The bug did not make it clear that there is any problem left at all when 
I looked at it recently.

The last message was closing the bug.

There was a control command reopening the bug without giving any 
rationale, but the last control command was
  fixed 832931 10.0.27-1

buildd.debian.org says that 10.0.27-0+deb8u1 was installed on jessie.[1]

If there is a problem left somewhere it is well-hidden, and not visible 
immediately when looking at the bug - I thought this was already resolved.

> Cheers,
> Moritz

cu
Adrian

[1] https://buildd.debian.org/status/package.php?p=mariadb-10.0&suite=jessie

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Porter roll call for Debian Stretch

[W. Martin Borgert]

On 2016-10-03 19:32, W. Martin Borgert wrote:
> On 2016-10-03 16:52, Steve McIntyre wrote:
> > You *are* aware that the current plan is for Stretch to be the last
> > release for armel?
>
> *Now* I am. Planned obsolescence is good for my company, but bad
> for customers and environment...

OK, thanks for the heads-up, Steve!
I have read the respective discussion now.
Not much chance for v5, it seems.

(A partial release would be nice for such platforms, i.e. no
X11, no Haskell etc.)

Re: Porter roll call for Debian Stretch

[W. Martin Borgert]

On 2016-10-03 16:52, Steve McIntyre wrote:
> You *are* aware that the current plan is for Stretch to be the last
> release for armel?

*Now* I am. Planned obsolescence is good for my company, but bad
for customers and environment...

Re: Porter roll call for Debian Stretch

[W. Martin Borgert]

On 2016-10-03 10:32, Lennart Sorensen wrote:
> Well a number of those embedded systems are in fact using Debian powerpc.

Unfortunately, embedded systems are hard to count, which may
lead to wrong assumptions.

E.g. my company produces an embedded measurement device based on
Debian armel, but the devices (around 1000..1, AFAIK) will
never appear in popcon. I hope that armel will not be dropped in
favour of armhf or arm64 any time soon! :~)

Re: Porter roll call for Debian Stretch

[Lennart Sorensen]

On Sat, Oct 01, 2016 at 01:17:54PM +0100, Ben Hutchings wrote:
> This is not at all true.  My experience is that IBM doesn't even build-
> test 32-bit configurations, as evidenced by several stable updates
> causing FTBFS in Debian.
> 
> Which are very different from the Power Macs and similar platforms that
> most Debian powerpc users care about.

Well a number of those embedded systems are in fact using Debian powerpc.
They do care.

The powermacs on the other hand I would think are hardly used anymore.
But that may just be my impression because I never cared about them in
the first place.   They weren't relevant to me. :)

I would not be surprised if there are more embedded freescale based
powerpc systems running debian out there than powermacs, but maybe I
am under estimating how many powermac users are still out there running
debian.

-- 
Len Sorensen

Re: Porter roll call for Debian Stretch

[Jose E. Marchesi]

> As for SPARC, Oracle is actually now heavily investing in Linux SPARC
> support, so even SPARC is getting back into shape which is why I hope
> we can add sparc64 as an official port soon.
[...]

Oracle cares about Solaris on SPARC, not Linux on SPARC.

We certainly _do_ care about GNU/Linux on SPARC.

Re: Porter roll call for Debian Stretch

[Ben Hutchings]

On Sat, 2016-10-01 at 15:48 +0200, John Paul Adrian Glaubitz wrote:
> On 10/01/2016 02:17 PM, Ben Hutchings wrote:
> > 
> > > 
> > > This isn't the case for PowerPC32 where upstream development is still very
> > > active because it's part of the PowerPC kernel which is maintained by
> > > IBM.
> > 
> > This is not at all true.  My experience is that IBM doesn't even build-
> > test 32-bit configurations, as evidenced by several stable updates
> > causing FTBFS in Debian.
> 
> They care enough that they are fixing bugs. Just recently, a bug in the
> PowerPC kernel was fixed that affected 32-bit embedded PowerPCs only.

$ git log --author=ibm --grep='ppc-?32|powerpc-?32|32-bit' -i -E arch/powerpc

finds me fewer than ten commits per year.

> > 
> > > 
> > > As for SPARC, Oracle is actually now heavily investing in Linux SPARC
> > > support, so even SPARC is getting back into shape which is why I hope
> > > we can add sparc64 as an official port soon.
> > [...]
> > 
> > Oracle cares about Solaris on SPARC, not Linux on SPARC.
> 
> Well, then you know more than the people at Oracle that I am talking to.
[... much evidence of Oracle supporting Linux on SPARC ...]

OK, I accept this has changed, but I'm quite surprised - Oracle is
ruthlessly commercial, and I'm mystified as to who they expect to buy
it.

Ben.

-- 
Ben Hutchings
Klipstein's 4th Law of Prototyping and Production:
A fail-safe circuit will destroy
others.

signature.asc
Description: This is a digitally signed message part

Re: Porter roll call for Debian Stretch

[John Paul Adrian Glaubitz]

On 10/01/2016 02:17 PM, Ben Hutchings wrote:
>> This isn't the case for PowerPC32 where upstream development is still very
>> active because it's part of the PowerPC kernel which is maintained by
>> IBM.
> 
> This is not at all true.  My experience is that IBM doesn't even build-
> test 32-bit configurations, as evidenced by several stable updates
> causing FTBFS in Debian.

They care enough that they are fixing bugs. Just recently, a bug in the
PowerPC kernel was fixed that affected 32-bit embedded PowerPCs only.

>> PowerPC32 is also still quite popular which is why it still sees
>> quite some testing in the wild. There are still new PowerPC32 designs
>> based on embedded CPUs (FreeScale and the like).
> 
> Which are very different from the Power Macs and similar platforms that
> most Debian powerpc users care about.

According to popcon, there are 250 users running a 32-bit kernel:

>
https://qa.debian.org/popcon-graph.php?packages=linux-image-powerpc&show_installed=on&show_vote=on&show_old=on&show_recent=on&show_nofiles=on&want_legend=on&want_ticks=on&from_date=&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1

And a little more than 100 users running a 64-bit BE kernel:

>
https://qa.debian.org/popcon-graph.php?packages=linux-image-powerpc64&show_installed=on&show_vote=on&show_old=on&show_recent=on&show_nofiles=on&want_legend=on&want_ticks=on&from_date=&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1

And around 16 users running a 64-bit LE kernel:

>
https://qa.debian.org/popcon-graph.php?packages=linux-image-powerpc64le&show_installed=on&show_vote=on&show_old=on&show_recent=on&show_nofiles=on&want_legend=on&want_ticks=on&from_date=&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1

While you may be right that IBM is not actively testing powerpc32, bugs are
still getting fixed and code is improved in general as you can see with the
many mails sent to linuxppc-dev with "powerpc32" in the subject.

>> As for SPARC, Oracle is actually now heavily investing in Linux SPARC
>> support, so even SPARC is getting back into shape which is why I hope
>> we can add sparc64 as an official port soon.
> [...]
> 
> Oracle cares about Solaris on SPARC, not Linux on SPARC.

Well, then you know more than the people at Oracle that I am talking to. 
However,
in case this statement is based on outdated information, here are two related
links:

In fall 2015, Oracle announced Linux for SPARC which is a first reference
platform to be used as a basis for SPARC development in the future:

> https://oss.oracle.com/projects/linux-sparc/

And just recently, Oracle just announced their first product which runs
Linux on SPARC hardware:

> http://vanpupi.stepi.net/index.php/2016/09/20/welcome-exadata-sl6-2/

Also, when you look at the sparclinux LKML archives, you will see that most
patches are contributed by Oracle people these days:

> http://marc.info/?l=linux-sparc&r=1&b=201609&w=2

This isn't surprising as Oracle is actively ramping up Linux support for
SPARC. They have developers all over the world who are being paid to work
on the kernel and on the toolchain. Just recently, Eric Snowberg from
Oracle contributed lots of code to GRUB to make it support SPARC so users
can switch from SILO. Lots of the stuff hasn't been merged yet unfortunately.

According to my sources that Oracle - as you may or may not know, I'm quite
involved with people from Oracle directly, so I can probably claim I'm the
person in Debian who can say most about Linux on SPARC these days - there
many more patches that Oracle developers have already prepared but not merged
yet but will be in the near future.

If Oracle didn't invest so much in Linux on SPARC with the result of so many
improvements coming along, I would have never suggested to make sparc64
a Debian release architecture in the future.

PS: I will be meeting up with Oracle people at LinuxCon next week in Berlin
and therefore hopefully be able to make some progress regarding the
talks with them over hardware donations/loans to Debian.

Cheers,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



signature.asc
Description: OpenPGP digital signature

Re: Porter roll call for Debian Stretch

[Ben Hutchings]

On Sat, 2016-10-01 at 02:28 +0200, Adam Borowski wrote:
> On Fri, Sep 30, 2016 at 11:01:55PM +0200, Mathieu Malaterre wrote:
[...]
> > I have not heard from the ppc64el porters, but I suspect ppc64 will
> > not be a release arch. So you need to take into consideration that for
> > powerpc to remain a release arch, one need minimal working ppc64 port.
> > Could we solve the situation of ppc64 for Stretch, could it be moved
> > to official release arch ?
> 
> What would you need ppc64 for?  Unlike i386, powerpc includes 64-bit
> kernels so users don't need multiarch:
[...]

This is only the case because ppc64 has a lower level of support
(unofficial port) than powerpc (release architecture).  The 64-bit
kernel package should be dropped once powerpc is at the same or lower
level of support than ppc64 - just as we've done for i386, s390 and
sparc.

Ben.

-- 
Ben Hutchings
Klipstein's 4th Law of Prototyping and Production:
A fail-safe circuit will destroy
others.


signature.asc
Description: This is a digitally signed message part

Re: Porter roll call for Debian Stretch

[Ben Hutchings]

On Fri, 2016-09-30 at 22:34 +0200, John Paul Adrian Glaubitz wrote:
> On 09/30/2016 09:04 PM, Niels Thykier wrote:
> > 
> > As for "porter qualification"
> > =
> > 
> > We got burned during the Jessie release, where a person answered the
> > roll call for sparc and we kept sparc as a release architecture for
> > Jessie.  However, we ended up with a completely broken and unbootable
> > sparc kernel.
> 
> To be fair, this happened because the upstream kernel development for
> SPARC came to an almost complete stop. There was basically only David
> Miller working on the port which turned out not to be enough.
> 
> This isn't the case for PowerPC32 where upstream development is still very
> active because it's part of the PowerPC kernel which is maintained by
> IBM.

This is not at all true.  My experience is that IBM doesn't even build-
test 32-bit configurations, as evidenced by several stable updates
causing FTBFS in Debian.

> PowerPC32 is also still quite popular which is why it still sees
> quite some testing in the wild. There are still new PowerPC32 designs
> based on embedded CPUs (FreeScale and the like).

Which are very different from the Power Macs and similar platforms that
most Debian powerpc users care about.

> As for SPARC, Oracle is actually now heavily investing in Linux SPARC
> support, so even SPARC is getting back into shape which is why I hope
> we can add sparc64 as an official port soon.
[...]

Oracle cares about Solaris on SPARC, not Linux on SPARC.

Ben.

-- 
Ben Hutchings
Klipstein's 4th Law of Prototyping and Production:
A fail-safe circuit will destroy
others.


signature.asc
Description: This is a digitally signed message part

Re: Porter roll call for Debian Stretch

[Mathieu Malaterre]

On Sat, Oct 1, 2016 at 2:28 AM, Adam Borowski  wrote:
> On Fri, Sep 30, 2016 at 11:01:55PM +0200, Mathieu Malaterre wrote:
>> On Fri, Sep 30, 2016 at 10:34 PM, John Paul Adrian Glaubitz
>>  wrote:
>> [...]
>> > On the other hand, some packages dropped support for PowerPC32 like Mono
>> > but this isn't a concern for most users, I would say.
>> [...]
>>
>> However I need to mention that the specific ppc/mono issue is in fact
>> pretty interesting. The long thread is on debian-powerpc@l.d.o but the
>> short version is that this issue only happen because we build the
>> ppc32 mono version on a ppc64 kernel, I know that since I did debug
>> this issue.
>
> Which, if I read the bug correctly, is a yet another case of a bogus
> build system looking at characteristics of the machine it's compiled on
> rather than baseline of the arch.

Well the bug is really upstream: one cannot assume page size at
compilation time. But that is a different story.

> And, per your own work, it's +patch +fixed-upstream.

Wow ! In fact I just realize my patch was against git/master at the
time, and was never backported. Need to get this fixed ASAP.

>> I have not heard from the ppc64el porters, but I suspect ppc64 will
>> not be a release arch. So you need to take into consideration that for
>> powerpc to remain a release arch, one need minimal working ppc64 port.
>> Could we solve the situation of ppc64 for Stretch, could it be moved
>> to official release arch ?
>
> What would you need ppc64 for?  Unlike i386, powerpc includes 64-bit
> kernels so users don't need multiarch:
>
> powerpc has:
> linux-image-4.7.0-1-powerpc - Linux 4.7 for uniprocessor 32-bit PowerPC 
> (signed)
> linux-image-4.7.0-1-powerpc-smp - Linux 4.7 for multiprocessor 32-bit PowerPC 
> (signed)
> linux-image-4.7.0-1-powerpc64 - Linux 4.7 for 64-bit PowerPC (signed)
> i386 has:
> linux-image-4.7.0-1-686-pae-unsigned - Linux 4.7 for modern PCs
> linux-image-4.7.0-1-686-unsigned - Linux 4.7 for older PCs
> linux-image-4.7.0-1-grsec-686-pae - Linux 4.7 for modern PCs, Grsecurity 
> protection
> linux-image-4.7.0-1-686 - Linux 4.7 for older PCs (signed)
> linux-image-4.7.0-1-686-pae - Linux 4.7 for modern PCs (signed)
>
> Note the joke: "for modern PCs".  Unless you do embedded it takes some
> serious dumpster diving to find a machine not better served by an -amd64
> kernel (and thus multiarch).  The i386 architecture is not self-contained,
> powerpc is.
>
> Thus, there is no need for ppc64 (userland), as long as powerpc has the
> toolchain to build 64-bit kernels.  And that's a primary target for gcc
> upstream.

Great ! That's all I wanted to check. I was worried we would need
buildd(s) running on ppc64el.

Re: Porter roll call for Debian Stretch

[Adam Borowski]

On Fri, Sep 30, 2016 at 11:01:55PM +0200, Mathieu Malaterre wrote:
> On Fri, Sep 30, 2016 at 10:34 PM, John Paul Adrian Glaubitz
>  wrote:
> [...]
> > On the other hand, some packages dropped support for PowerPC32 like Mono
> > but this isn't a concern for most users, I would say.
> [...]
> 
> However I need to mention that the specific ppc/mono issue is in fact
> pretty interesting. The long thread is on debian-powerpc@l.d.o but the
> short version is that this issue only happen because we build the
> ppc32 mono version on a ppc64 kernel, I know that since I did debug
> this issue.

Which, if I read the bug correctly, is a yet another case of a bogus
build system looking at characteristics of the machine it's compiled on
rather than baseline of the arch.

And, per your own work, it's +patch +fixed-upstream.

> I have not heard from the ppc64el porters, but I suspect ppc64 will
> not be a release arch. So you need to take into consideration that for
> powerpc to remain a release arch, one need minimal working ppc64 port.
> Could we solve the situation of ppc64 for Stretch, could it be moved
> to official release arch ?

What would you need ppc64 for?  Unlike i386, powerpc includes 64-bit
kernels so users don't need multiarch:

powerpc has:
linux-image-4.7.0-1-powerpc - Linux 4.7 for uniprocessor 32-bit PowerPC (signed)
linux-image-4.7.0-1-powerpc-smp - Linux 4.7 for multiprocessor 32-bit PowerPC 
(signed)
linux-image-4.7.0-1-powerpc64 - Linux 4.7 for 64-bit PowerPC (signed)
i386 has:
linux-image-4.7.0-1-686-pae-unsigned - Linux 4.7 for modern PCs
linux-image-4.7.0-1-686-unsigned - Linux 4.7 for older PCs
linux-image-4.7.0-1-grsec-686-pae - Linux 4.7 for modern PCs, Grsecurity 
protection
linux-image-4.7.0-1-686 - Linux 4.7 for older PCs (signed)
linux-image-4.7.0-1-686-pae - Linux 4.7 for modern PCs (signed)

Note the joke: "for modern PCs".  Unless you do embedded it takes some
serious dumpster diving to find a machine not better served by an -amd64
kernel (and thus multiarch).  The i386 architecture is not self-contained,
powerpc is.

Thus, there is no need for ppc64 (userland), as long as powerpc has the
toolchain to build 64-bit kernels.  And that's a primary target for gcc
upstream.

-- 
A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol, 1kg
raspberries, 0.4kg sugar; put into a big jar for 1 month.  Filter out and
throw away the fruits (can dump them into a cake, etc), let the drink age
at least 3-6 months.

Re: Porter roll call for Debian Stretch

[Milan Kupcevic]

On 09/20/2016 05:46 PM, John Paul Adrian Glaubitz wrote:
> On 09/20/2016 11:16 PM, Niels Thykier wrote:
>>- powerpc: No porter (RM blocker)
> 
> I'd be happy to pick up powerpc to keep it for Stretch. I'm already
> maintaining powerpcspe which is very similar to powerpc.
> 


Thank you Adrian for stepping up. You will have my support in this
endeavor for the Stretch release.

Milan




signature.asc
Description: OpenPGP digital signature

Re: Porter roll call for Debian Stretch

[Mathieu Malaterre]

Adrian,

On Fri, Sep 30, 2016 at 10:34 PM, John Paul Adrian Glaubitz
 wrote:
[...]
> On the other hand, some packages dropped support for PowerPC32 like Mono
> but this isn't a concern for most users, I would say.
[...]

Thanks very much for stepping up as porter, you have my vote !

However I need to mention that the specific ppc/mono issue is in fact
pretty interesting. The long thread is on debian-powerpc@l.d.o but the
short version is that this issue only happen because we build the
ppc32 mono version on a ppc64 kernel, I know that since I did debug
this issue.

I have not heard from the ppc64el porters, but I suspect ppc64 will
not be a release arch. So you need to take into consideration that for
powerpc to remain a release arch, one need minimal working ppc64 port.
Could we solve the situation of ppc64 for Stretch, could it be moved
to official release arch ?

-M

Re: Porter roll call for Debian Stretch

[John Paul Adrian Glaubitz]

On 09/30/2016 09:04 PM, Niels Thykier wrote:
> As for "porter qualification"
> =
> 
> We got burned during the Jessie release, where a person answered the
> roll call for sparc and we kept sparc as a release architecture for
> Jessie.  However, we ended up with a completely broken and unbootable
> sparc kernel.

To be fair, this happened because the upstream kernel development for
SPARC came to an almost complete stop. There was basically only David
Miller working on the port which turned out not to be enough.

This isn't the case for PowerPC32 where upstream development is still very
active because it's part of the PowerPC kernel which is maintained by
IBM. PowerPC32 is also still quite popular which is why it still sees
quite some testing in the wild. There are still new PowerPC32 designs
based on embedded CPUs (FreeScale and the like).

As for SPARC, Oracle is actually now heavily investing in Linux SPARC
support, so even SPARC is getting back into shape which is why I hope
we can add sparc64 as an official port soon.

>   That was an embarrassment to the Debian stability and quality
>   reputation that I never - ever - want to repeat.

Well, mistakes happen and while I think it's good and important to learn
from mistakes, we should not dramatize such issues. We have had worse
issues like the OpenSSL entropy bug, for example, and we still managed
to cope with the fallout in a very professional manner.

> (For avoidance of doubt: I want to ensure that release architectures
> "just work(tm)" and I have no desire to blame that volunteer).

I don't think there is any concern regarding the powerpc port in this
regard. wanna-build shows almost 11800 packages that are up-to-date
which is a good indicator that the port is in good shape, both regarding
the toolchain and various source packages which need architecture-specific
adaptations like LibreOffice or JavaScript packages.

On the other hand, some packages dropped support for PowerPC32 like Mono
but this isn't a concern for most users, I would say.

> If I am to support powerpc as a realease architecture for Stretch, I
> need to know that there are *active* porters behind it committed to
> keeping it in the working.  People who would definitely catch such
> issues long before the release.  People who file bugs / submit patches etc.

I agree and I'm actually doing that all the time. I always run unstable
on my machines and constantly check wanna-build for build issues and
report them upstream whenever they occur. I have helped dozens of such
issues on "sh4" and "sparc64", for example.

> If you need inspiration: Have a look at the [automatic testing of
> ppc64el images].  Or the [arm64 machines on ci.debian.net] with
> comparable results to amd64.  This is the sort of thing that inspires
> confidence in the ports for me and I think we should have vastly more of.

I agree that would be nice to have. However, to be fair, we don't have
that type of testing for all release architectures and to my current
knowledge, automated testing of installation images is not a criteria
for an architecture to maintain release status.

My main argument for why we should keep the powerpc port is its
popularity. If we look at the numbers from popcon [1], powerpc
is still the fourth-most-popular port and I think we would disappoint
many users if we were to drop the port for Stretch. Note that while
ports like "arm64" or "ppc64el" receive lots of support, especially
from companies, they still haven't reached the same popularity as the
powerpc port for example. Heck, there are even more users for "hppa"
and "sparc64" which both are just unofficial ports architectures.

Thanks,
Adrian

> [1] http://popcon.debian.org/

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Porter roll call for Debian Stretch

[Niels Thykier]

Niels Thykier:
> [...]
> 
> As for "porter qualification"
> =
> 
> We got burned during the Jessie release, where a person answered the
> roll call for sparc and we kept sparc as a release architecture for
> Jessie.  However, we ended up with a completely broken and unbootable
> sparc kernel.
> 
>   That was an embarrassment to the Debian stability and quality
>   reputation that I never - ever - want to repeat.
> 
> (For avoidance of doubt: I want to ensure that release architectures
> "just work(tm)" and I have no desire to blame that volunteer).
> 
> 
> [...]
> 

I have been reminded of my inaccurate memory.  The broken sparc kernel
was released in Wheezy (and discovered during Jessie).  The roll call
for Jessie happened shortly after Wheezy release but before DSA
discovered the broken kernel.

Nonetheless, the core argument still stands.

Thanks,
~Niels

Re: Porter roll call for Debian Stretch

[Adam D. Barratt]

On Fri, 2016-09-30 at 19:04 +, Niels Thykier wrote:
> As for "porter qualification"
> =
> 
> We got burned during the Jessie release, where a person answered the
> roll call for sparc and we kept sparc as a release architecture for
> Jessie.  However, we ended up with a completely broken and unbootable
> sparc kernel.

fwiw, you mean wheezy.

Regards,

Adam

Re: Porter roll call for Debian Stretch

[Niels Thykier]

John Paul Adrian Glaubitz:
> On 09/30/2016 06:08 PM, Niels Thykier wrote:
>> I strongly /suspect/ that "no porters" for powerpc will imply the
>> removal of powerpc for Stretch.  It may or may not be moved to ports
>> (assuming someone is willing to support it there).
> 
> So, I take this as a "no" for the offer from me and Christoph Biedel to take
> over the powerpc port for Stretch?
> 
> [...]
> 
> Thanks,
> Adrian
> 

My statement above was made based on the assumption stated in the first
line of Mathieu's mail, which was "Assume there are no powerpc porters
for Stretch".

As for "porter qualification"
=

We got burned during the Jessie release, where a person answered the
roll call for sparc and we kept sparc as a release architecture for
Jessie.  However, we ended up with a completely broken and unbootable
sparc kernel.

  That was an embarrassment to the Debian stability and quality
  reputation that I never - ever - want to repeat.

(For avoidance of doubt: I want to ensure that release architectures
"just work(tm)" and I have no desire to blame that volunteer).


If I am to support powerpc as a realease architecture for Stretch, I
need to know that there are *active* porters behind it committed to
keeping it in the working.  People who would definitely catch such
issues long before the release.  People who file bugs / submit patches etc.


If you need inspiration: Have a look at the [automatic testing of
ppc64el images].  Or the [arm64 machines on ci.debian.net] with
comparable results to amd64.  This is the sort of thing that inspires
confidence in the ports for me and I think we should have vastly more of.


Thanks,
~Niels

[automatic testing of ppc64el images]:
 https://lists.debian.org/debian-powerpc/2016/06/msg2.html

[arm64 machines on ci.debian.net]:
 https://ci.debian.net/status/

Re: Porter roll call for Debian Stretch

[Adam Borowski]

On Fri, Sep 30, 2016 at 10:03:47AM +0200, Mathieu Malaterre wrote:
> [Let's assume that we can't find a powerpc porter in time for Stretch.]

Two potential porters stepped up, who might or might not be accepted.

> 1. Will `powperpc` automatically be downgraded to simple port ? Or is
> this also not automated and the port may simply be removed (eg. sparc)
> ?

Removal from the main archive isn't automatic, and assuming no serious
kernel or toolchain breakage, I guess there's no reason for such removal
to be hasty.

On the other hand, place in -ports (aka second-class architectures) isn't
automatic either, and relies on someone doing the work.

> 2. Apart from loosing the automatic debian-installer stuff, what else
> are we going to loose in that scenario ?

Security support and stable release.

-- 
A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol, 1kg
raspberries, 0.4kg sugar; put into a big jar for 1 month.  Filter out and
throw away the fruits (can dump them into a cake, etc), let the drink age
at least 3-6 months.

Re: Porter roll call for Debian Stretch

[John Paul Adrian Glaubitz]

On 09/30/2016 06:08 PM, Niels Thykier wrote:
> I strongly /suspect/ that "no porters" for powerpc will imply the
> removal of powerpc for Stretch.  It may or may not be moved to ports
> (assuming someone is willing to support it there).

So, I take this as a "no" for the offer from me and Christoph Biedel to take
over the powerpc port for Stretch?

I have quite some experience with working on ports and unlike what Matthias 
claimed,
I'm not just maintaining a few buildds but also getting architecture-specific 
bugs
fixed and so on.

Is there any specific qualification I am missing?

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Porter roll call for Debian Stretch

[Niels Thykier]

Mathieu Malaterre:
> Hi all,
> 
> [...]
> 
> [Let's assume that we can't find a powerpc porter in time for Stretch.]
> 
> 1. Will `powperpc` automatically be downgraded to simple port ? Or is
> this also not automated and the port may simply be removed (eg. sparc)
> ?
> 2. Apart from loosing the automatic debian-installer stuff, what else
> are we going to loose in that scenario ?
> 
> Thanks much for pointers !
> 
> -M
> 

I strongly /suspect/ that "no porters" for powerpc will imply the
removal of powerpc for Stretch.  It may or may not be moved to ports
(assuming someone is willing to support it there).

Not sure I can answer your 2nd question though.

~Niels

Re: Porter roll call for Debian Stretch

[Christian Zigotzky]

You have a porter for PowerPC. See email from Adrian. ;-)

-- Christian

Sent from my iPhone

> On 30 Sep 2016, at 10:03, Mathieu Malaterre  wrote:
> 
> Hi all,
> 
>> On Fri, Sep 23, 2016 at 3:54 PM, Matthias Klose  wrote:
>>> On 20.09.2016 23:46, John Paul Adrian Glaubitz wrote:
 On 09/20/2016 11:16 PM, Niels Thykier wrote:
   - powerpc: No porter (RM blocker)
>>> 
>>> I'd be happy to pick up powerpc to keep it for Stretch. I'm already
>>> maintaining powerpcspe which is very similar to powerpc.
>> 
>> No, you are not maintaining powerpcspe as a release architecture, and that's
>> something different than building packages for some of the ports 
>> architectures.
>> If you can get powerpcspe accepted as a release architecture, then maybe you
>> gain some credibility to maintain another release architecture ;)
> 
> [Let's assume that we can't find a powerpc porter in time for Stretch.]
> 
> 1. Will `powperpc` automatically be downgraded to simple port ? Or is
> this also not automated and the port may simply be removed (eg. sparc)
> ?
> 2. Apart from loosing the automatic debian-installer stuff, what else
> are we going to loose in that scenario ?
> 
> Thanks much for pointers !
> 
> -M
> 

Re: Porter roll call for Debian Stretch

[Mathieu Malaterre]

Hi all,

On Fri, Sep 23, 2016 at 3:54 PM, Matthias Klose  wrote:
> On 20.09.2016 23:46, John Paul Adrian Glaubitz wrote:
>> On 09/20/2016 11:16 PM, Niels Thykier wrote:
>>>- powerpc: No porter (RM blocker)
>>
>> I'd be happy to pick up powerpc to keep it for Stretch. I'm already
>> maintaining powerpcspe which is very similar to powerpc.
>
> No, you are not maintaining powerpcspe as a release architecture, and that's
> something different than building packages for some of the ports 
> architectures.
> If you can get powerpcspe accepted as a release architecture, then maybe you
> gain some credibility to maintain another release architecture ;)

[Let's assume that we can't find a powerpc porter in time for Stretch.]

1. Will `powperpc` automatically be downgraded to simple port ? Or is
this also not automated and the port may simply be removed (eg. sparc)
?
2. Apart from loosing the automatic debian-installer stuff, what else
are we going to loose in that scenario ?

Thanks much for pointers !

-M

Re: Porter roll call for Debian Stretch

[Christoph Biedl]

John Paul Adrian Glaubitz wrote...

> On 09/20/2016 11:16 PM, Niels Thykier wrote:
> >- powerpc: No porter (RM blocker)
> 
> I'd be happy to pick up powerpc to keep it for Stretch. I'm already
> maintaining powerpcspe which is very similar to powerpc.

For somewhat personal reasons I'm interested in keeping powerpc in
stretch as well. I certainly cannot take the entire role as a porter,
especially since I don't know what amount of work this implies. But I
am willing to help.

There are two powerpc boxes in my collection, used regulary. One runs
on stable, the other on testing. I haven't done d-i tests but
certainly could do.

Christoph



signature.asc
Description: Digital signature

Re: Porter roll call for Debian Stretch

[John Paul Adrian Glaubitz]

On 09/23/2016 03:54 PM, Matthias Klose wrote:
> No, you are not maintaining powerpcspe as a release architecture, and that's
> something different than building packages for some of the ports 
> architectures.
> If you can get powerpcspe accepted as a release architecture, then maybe you
> gain some credibility to maintain another release architecture ;)

So, what are the criteria to be knighted to become a maintainer of powerpc?

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Porter roll call for Debian Stretch

[Matthias Klose]

On 20.09.2016 23:46, John Paul Adrian Glaubitz wrote:
> On 09/20/2016 11:16 PM, Niels Thykier wrote:
>>- powerpc: No porter (RM blocker)
> 
> I'd be happy to pick up powerpc to keep it for Stretch. I'm already
> maintaining powerpcspe which is very similar to powerpc.

No, you are not maintaining powerpcspe as a release architecture, and that's
something different than building packages for some of the ports architectures.
If you can get powerpcspe accepted as a release architecture, then maybe you
gain some credibility to maintain another release architecture ;)

Matthias

Re: Porter roll call for Debian Stretch

[Christian Seiler]

On 09/21/2016 08:41 AM, Riku Voipio wrote:
> AFAIK Address space randomizing is not really helpful on 32 bit 
> architectures - there is just not that many places to randomize to[1].

Well, sure, but there's still a huge difference in an explot with
100% reliability, or an exploit that will just crash the program
in 95% of cases. Sure, if there's an easy way to repeatedly try
the exploit 20 times, it won't be a show-stopper, but it will
make the life of people who want to exploit a flaw just a tiny
bit harder.

> At least previously, PIE added ~10% to binary size,

At least on x86 there have been substantial improvements in
receent gcc versions when it comes to PIE support, so the impact
of PIE on executables even on 32bit is a lot smaller than it used
to be. I don't know about ARM though.

Consider the following two data points:

 - A _lot_ of code in Debian is in shared libraries, which are
   compiled with -fPIC anyway. Many executables only spend a
   fraction of their instructions in the executable code itself.

 - It's been considered best practice to enable PIE executables
   if possible (via hardening=+all or similar), so many programs
   in Debian (and e.g. all of my packages except one) already
   use that. I suspect that a lot of of code that you are
   currently running is already PIE, especially in the packages
   that are more actively maintained.

Regards,
Christian

Re: Porter roll call for Debian Stretch

[Riku Voipio]

On Tue, Sep 20, 2016 at 09:16:00PM +, Niels Thykier wrote:
> Over all, most people (who answered it) was positive towards the switch.
>  Based on this, I suspect that if we make PIE default in Stretch, then
> we will do it for all architectures.  That said, you will be notified if
> that default changes for Stretch.

Is this just for ASLR, or is ther another motivating factor for PIE?

AFAIK Address space randomizing is not really helpful on 32 bit 
architectures - there is just not that many places to randomize to[1].
At least previously, PIE added ~10% to binary size, which can have a major
performance impact on the 32-bit arm core's that don't have much cache to
begin with.

Riku
[1] https://cseweb.ucsd.edu/~hovav/dist/asrandom.pdf

Re: Porter roll call for Debian Stretch

[John Paul Adrian Glaubitz]

On 09/20/2016 11:16 PM, Niels Thykier wrote:
>- powerpc: No porter (RM blocker)

I'd be happy to pick up powerpc to keep it for Stretch. I'm already
maintaining powerpcspe which is very similar to powerpc.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913



signature.asc
Description: OpenPGP digital signature

Re: Porter roll call for Debian Stretch

[Niels Thykier]

ni...@thykier.net:
> Hi,
> 
> Like last release, we are doing a roll call for porters of all release
> architectures.  If you are an active porter behind one of the [release
> architectures] for the entire lifetime of Debian Stretch (est. end of
> 2020), please respond with a signed email containing the following
> before Friday, the 9th of September:
> 
>  [...]
> 

Thanks to all that replied to the roll call.

We got replies from all architectures except amd64 (Waived), i386
(Waived) and powerpc (problematic).  For the ports that replied, all had
at least 2 or more porters.
  We also got a parallel update from Matthias Klose on the toolchain
state[1].  Based on these we have updated the current state of the
architecture qualification for Stretch[2].

 * Please check that your name is listed as a porter on [2] (see [3]
   for how).  If I missed you, please do not hesitate to let me know.

 * For powerpc, mips and mipsel: Please note that we have added a
   (potentially) blocking issues for your architecture.
   - powerpc: No porter (RM blocker)
   - mips+mipsel: #834147 (RM concern, possibly blocker)


As for the piggy backed question about PIE by default:

 * arm64: 2 OK, 1 did not mention it
 * armel: 3 OK, 1 OK if tested first, 1 was neutral and 1 did not
   mention it
 * armhf: 4 OK, 1 OK if tested first, 1 was neutral and 2 did not
   mention it
 * mips+mipsel+mips64el: 3 OK, 1 OK later, 4 did not mention it.
 * ppc64el: 3 OK, 1 recommended further testing/research
 * s390x: 2 OK

Over all, most people (who answered it) was positive towards the switch.
 Based on this, I suspect that if we make PIE default in Stretch, then
we will do it for all architectures.  That said, you will be notified if
that default changes for Stretch.

Thanks,
~Niels

[1] https://lists.debian.org/debian-release/2016/09/msg00263.html

[2] https://release.debian.org/stretch/arch_qualify.html

[3] Hover your mouse over the number of porters for your architecture
or download the underlying data file from:
  https://release.debian.org/stretch/arch_spec.yaml




signature.asc
Description: OpenPGP digital signature

Re: Porter roll call for Debian Stretch

[Niels Thykier]

Aurelien Jarno:
> On 2016-08-17 22:05, ni...@thykier.net wrote:
>> Hi,
>>
>> Like last release, we are doing a roll call for porters of all release
>> architectures.  If you are an active porter behind one of the [release
> 

Hi,

Apologies for the tardiness on my part for this.

> Does it really concerns *all* release architectures? Traditionally amd64
> and i386 have been granted waivers as "the toolchain maintainers are
> happy to support" these architectures "as-is".

Indeed you are correct that amd64 and i386 has generally been granted a
waiver.  Said waiver has not been retracted, so the roll call was
intended for all except amd64 + i386.

> That said the toolchain
> maintainers do not fix ports specific bugs outside of the toolchain.
> 
> While I fully agree that we can have a waiver for amd64 due to being the
> de facto standard architecture, it seems that a few leaf packages do
> not build on i386 and that we have no porters to fix them. That is
> probably still fine, but I wonder how fast the number of such packages
> will increase in the future.
> 

You are not the first to race the concern for i386.  To my knowledge, it
is still a "minor" issue for Stretch.  Nonetheless, I suspect that we
should revisit that decision for Buster.

>> architectures] for the entire lifetime of Debian Stretch (est. end of
>> 2020), please respond with a signed email containing the following
> 
> What is the relation between the end of support of Stretch...
> 
>> before Friday, the 9th of September:
>  
>>  * Which architectures are you committing to be an active porter for?
>>  * Please describe recent relevant porter contributions.
>>  * Are you running/using Debian testing or sid on said port(s)?
>>  * Are you testing/patching d-i for the port(s)?
>>  * If we were to enable -fPIE/-pie by default in GCC-6, should that change
>>also apply to this port? [0]
> 
> ... and the above questions?
> 
> I fully agree that running testing/sid, fixing bugs or working on d-i up
> to the release of Stretch will improve its quality. But after the
> release it will improve the quality of Buster and later Bullseye. On the
> other hand running testing/sid after the release of Stretch will not
> help to catch bugs that can be fixed through a point release.
> 
> Aurelien
> 

Most of the questions were intended to help and inspire people in how to
report what they did.  We were also interested in concrete examples,
which unfortunately did not happen in all answers.

That said, there are some questions that were intended to catch
potential issues *before* we committed to a release architecture.  As an
example, the "running testing/sid on said port" was meant to help us
spot a potential "broken release architecture discovered post release"
in the making (as happened with sparc).

As for the d-i question, I am a bit surprised that you understood it as
implicitly limited to sid/testing.

(Finally, as mentioned elsewhere, the PIE question was simply
piggy-backed because I thought it was simpler that way).

~Niels







signature.asc
Description: OpenPGP digital signature

Re: FTBFS with PIE & bindnow (was: Re: Porter roll call for Debian Stretch)

[Matthias Klose]

Hi,

thanks for the work on this.  I'd like to defer the final decision to the
release team, however I'm not keen on having these defaults turned on
architectures which already have enough issues on their own.  In the recent
porters call people claim that turning on these "should not be a problem"
without giving any details why. I don't like this "laissez faire" attitude.
Without doing even a (limited) test rebuild on these architectures.  I don't
think this is the right time to turn on the defaults, that should be better done
when starting the development for the next release cycle.  If people are
confident to turn these on on amd64, fine.  If people are fine to trust Ubuntu's
experience to turn these on on ppc64el and s390x, fine as well.  But for any
other architecture please do a partial test rebuild before turning these on.

Matthias

On 09.09.2016 16:04, Bálint Réczey wrote:
> Hi,
> 
> First of all thanks to Lucas Nussbaum who ran the first test build!
> 
> 2016-08-31 19:21 GMT+02:00 Steve Langasek :
>> On Wed, Aug 31, 2016 at 11:26:55AM +0100, Dimitri John Ledkov wrote:
>>> Hello,
 Results are available at
 https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/
>>
 I did a full rebuild with bindnow and PIE enabled, then rebuilt all
 failed packages with a pristine unstable chroot.
>>
 You can take a look at
 https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/diff.txt
 and grep for "OK Failed" (failed with PIE+bindnow, built fine in
 unstable). (There are 1188 packages failing to build)
>>
 Logs for both builds are available in the respective subdirectories.
>>
>>> Are you sure these are correct? The numbers for PIE+bindnow are a lot
>>> higher than what we see in Ubuntu, for same unmodified packages.
>>
>>> E.g. looking at http://qa.ubuntuwire.org/ftbfs/
>>
>>> amd64/ppc64el/s390x have PIE+bindnow enabled, and
>>> i386/armhf/arm64/powerpc do not. here is nothing in the thousands
>>> range. There might be a dozen packages with PIE+bindnow fixes in
>>> ubuntu, that's not in debian, but that amount cannot be more than a
>>> dozen or two.
> 
> Is there a list available or an easy way of collecting them?
> 
>>
>> Note that enabling PIE by default is going to cause build failures for a
>> number of packages which link against static libraries, if those static
>> libraries have not been rebuilt yet with PIE/PIC.  Ubuntu has worked through
>> this transition, so a direct comparison would require a rebootstrap test in
>> Debian instead of just a rebuild test (i.e.: test rebuild packages in
>> dependency order, and build later packages against the output of the earlier
>> rebuilds).
> 
> True. Full rebootstrapping of the archive is not available
> automatically and this
> was really useful as a first test.
> 
> I have added more dpkg patches [1] to make -pie hardening flag a noop since 
> GCC
> upstream is not interested in making -no-pie easily usable [2].
> 
> I tested the packages failing to build with the previous patches and
> many of them
> could be built.
> The logs of the remaining failures can be found here:
> https://people.debian.org/~rbalint/build-logs/pie-bindnow-20160906/
> 
> If we ignore the packages having "haskell" in their name the failures are down
> to 295 packages.
> 
> I'm starting ot file bugs for the FTBFS-s.
> 
> Patched dpkg and gcc is still available for those who would like to reproduce
> the issues.
> 
> Cheers,
> Balint
> 
> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=835149
> [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77464
> [3] https://people.debian.org/~rbalint/ppa/pie-bindnow/
> 

FTBFS with PIE & bindnow (was: Re: Porter roll call for Debian Stretch)

[Bálint Réczey]

Hi,

First of all thanks to Lucas Nussbaum who ran the first test build!

2016-08-31 19:21 GMT+02:00 Steve Langasek :
> On Wed, Aug 31, 2016 at 11:26:55AM +0100, Dimitri John Ledkov wrote:
>> Hello,
>> > Results are available at
>> > https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/
>
>> > I did a full rebuild with bindnow and PIE enabled, then rebuilt all
>> > failed packages with a pristine unstable chroot.
>
>> > You can take a look at
>> > https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/diff.txt
>> > and grep for "OK Failed" (failed with PIE+bindnow, built fine in
>> > unstable). (There are 1188 packages failing to build)
>
>> > Logs for both builds are available in the respective subdirectories.
>
>> Are you sure these are correct? The numbers for PIE+bindnow are a lot
>> higher than what we see in Ubuntu, for same unmodified packages.
>
>> E.g. looking at http://qa.ubuntuwire.org/ftbfs/
>
>> amd64/ppc64el/s390x have PIE+bindnow enabled, and
>> i386/armhf/arm64/powerpc do not. here is nothing in the thousands
>> range. There might be a dozen packages with PIE+bindnow fixes in
>> ubuntu, that's not in debian, but that amount cannot be more than a
>> dozen or two.

Is there a list available or an easy way of collecting them?

>
> Note that enabling PIE by default is going to cause build failures for a
> number of packages which link against static libraries, if those static
> libraries have not been rebuilt yet with PIE/PIC.  Ubuntu has worked through
> this transition, so a direct comparison would require a rebootstrap test in
> Debian instead of just a rebuild test (i.e.: test rebuild packages in
> dependency order, and build later packages against the output of the earlier
> rebuilds).

True. Full rebootstrapping of the archive is not available
automatically and this
was really useful as a first test.

I have added more dpkg patches [1] to make -pie hardening flag a noop since GCC
upstream is not interested in making -no-pie easily usable [2].

I tested the packages failing to build with the previous patches and
many of them
could be built.
The logs of the remaining failures can be found here:
https://people.debian.org/~rbalint/build-logs/pie-bindnow-20160906/

If we ignore the packages having "haskell" in their name the failures are down
to 295 packages.

I'm starting ot file bugs for the FTBFS-s.

Patched dpkg and gcc is still available for those who would like to reproduce
the issues.

Cheers,
Balint

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=835149
[2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77464
[3] https://people.debian.org/~rbalint/ppa/pie-bindnow/

Re: Porter roll call for Debian Stretch

[Roger Shimizu]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Wed, 17 Aug 2016 22:05:06 +0200
ni...@thykier.net wrote:

> Like last release, we are doing a roll call for porters of all release
> architectures.  If you are an active porter behind one of the [release
> architectures] for the entire lifetime of Debian Stretch (est. end of
> 2020), please respond with a signed email containing the following
> before Friday, the 9th of September:
> 

Hi,

I am an active porter for the following architectures and I intend
to continue this for the lifetime of the Stretch release (est. end
of 2020):

For armel, I
 - submit device-tree patch to upstream (linux kernel), and backport to debian 
kernel to get more devices supported
 - support new device for d-i and flash-kernel package
 - test most packages on this architecture
 - run Debian stable / testing / unstable system on port that I use regularly
 - triage arch-specific bugs
 - fix arch-related bugs
 - triage d-i bugs
 - test d-i regularly
 - fix d-i bugs/issues

I am a DM.

Altough I enabled -fPIE/-pie for most of my maintaining packages, I'm not sure 
/ I don't have enough knowledge whether it's able to be applied to all packages.
Since all other ARM porters seem agree on this, I believe it definitely 
deserves a try to enable this hardening on stretch.

Cheers,
- -- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1
-BEGIN PGP SIGNATURE-

iQIcBAEBCgAGBQJXzEFiAAoJEKR4aw2nAzSoAekP/j4eNf0jKvmArPlPbhA7XkBk
/5u9Un4qOHBNcSMAU5YVLHkpnT1CX/C08W+/ctGbB9AnnRwyn8X0uailjedZ13jK
oZYW/kUSwWiPmOkRTeNgFepzuKL+TNsAGgjHOY4ZbzYKC+h9C0UNWwyA77L3GUep
3HA9eTrtxMAAvJPNN4AhOjMeCI3qXIZ+wLKjhT+u/OUETWly8MolBw8PUjjwW6yy
Va7ciRMQf8KL149+Pa/tYFaENoAOV6//3M2QkJyaGbfxJp3xiFFcrlw+kw6J4RyH
vNIewz78nZwN88Y7JWa2EdBVcJr0897REXpDPXK/OpzlWw0R0xqB86jtmHfc+rQJ
IiNGt5Uc4Y3mD04O6AEDDJFJnEQ/QLi8OR8/TuxHiBJ6JTv0m67KsJVgdFqeRnlz
wJqcIQAzTF1iixVXjreVqW6P/+pGNHDbh9APfUz+UV97sZ4tD2BV1K1Rgk7cPDCn
zcpVhkQRy5PzLmK315vb8h9juFDDS/18yzmXwGMnmIhv4+8GBJIQLm5gvk9NuEbw
V/hBC42+fqX6RzGCoV3M8V+A6aLSpG/BcIAQOx8ewVfzMHIFSJmYParCHKXdiX+z
WB8UBt2xCHuzg98jxU80UwR492X9WvKeb6xA8dKqOW22XzsLxaQTe+SLSaGQ7er2
cpuhCpYFDKj/TL6UE2f9
=Vckg
-END PGP SIGNATURE-

Re: Porter roll call for Debian Stretch

[Steve Langasek]

On Wed, Aug 31, 2016 at 11:26:55AM +0100, Dimitri John Ledkov wrote:
> Hello,
> > Results are available at
> > https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/

> > I did a full rebuild with bindnow and PIE enabled, then rebuilt all
> > failed packages with a pristine unstable chroot.

> > You can take a look at
> > https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/diff.txt
> > and grep for "OK Failed" (failed with PIE+bindnow, built fine in
> > unstable). (There are 1188 packages failing to build)

> > Logs for both builds are available in the respective subdirectories.

> Are you sure these are correct? The numbers for PIE+bindnow are a lot
> higher than what we see in Ubuntu, for same unmodified packages.

> E.g. looking at http://qa.ubuntuwire.org/ftbfs/

> amd64/ppc64el/s390x have PIE+bindnow enabled, and
> i386/armhf/arm64/powerpc do not. here is nothing in the thousands
> range. There might be a dozen packages with PIE+bindnow fixes in
> ubuntu, that's not in debian, but that amount cannot be more than a
> dozen or two.

Note that enabling PIE by default is going to cause build failures for a
number of packages which link against static libraries, if those static
libraries have not been rebuilt yet with PIE/PIC.  Ubuntu has worked through
this transition, so a direct comparison would require a rebootstrap test in
Debian instead of just a rebuild test (i.e.: test rebuild packages in
dependency order, and build later packages against the output of the earlier
rebuilds).

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: PGP signature

Re: Porter roll call for Debian Stretch

[Bálint Réczey]

Hi,

2016-08-31 12:26 GMT+02:00 Dimitri John Ledkov :
> Hello,
>
> On 30 August 2016 at 23:07, Lucas Nussbaum  wrote:
>> On 22/08/16 at 19:12 +0200, Bálint Réczey wrote:
>>> Hi Guillem,
>>>
>>> 2016-08-21 14:02 GMT+02:00 Guillem Jover :
>>> > Hi!
>>> >
>>> > On Sun, 2016-08-21 at 10:24:42 +0200, Bálint Réczey wrote:
>>> >> I'm testing a set of patches [2] for gcc and dpkg which enable bindnow 
>>> >> for all
>>> >> arches and PIE for amd64, ppc64el and s390x in sync with Ubuntu.
>>> >>
>>> >> My assumption was that this set of architectures need the least amount of
>>> >> additional work since they are tested already in Ubuntu, but I would be 
>>> >> happy
>>> >> if more ports would opt in for PIE.
>>> >>
>>> >> I plan filing wishlist bugs against dpkg and gcc with the patches
>>> >> after I rebuilt a
>>> >> few packages with the defaults.
>>> >
>>> > TBH I think PIE should in fact be safer to enable globally than
>>> > bindnow, because the latter changes the run-time behavior and things
>>> > might break (perhaps even silently) when failing to load plugins
>>> > or similar.
>>>
>>> Yes, in that sense enabling PIE is safer indeed. Regarding bindnow
>>> I don't expect too many surprises either, since other distributions
>>> already tested enabling bindnow and probably they found
>>> most issues.
>>>
>>> >
>>> > From dpkg PoV enabling both, would at least require a full-archive
>>> > rebuild, for bindnow ideally also a full autopkgtest run (as the
>>> > updated dpkg FAQ states now, after Lucas Nussbaum approached me some
>>> > weeks ago mentioning he was willing to do such archive-wide rebuild).
>>>
>>> The patches at [2] seem to work well and since you expressed that you would
>>> prefer changing both toolchain and dpkg, too, I would like to suggest 
>>> running
>>> the rebuild with those patches.
>>>
>>> I think Matthias would be OK with the patch since it is very small and 
>>> brings
>>> Debian's gcc closer to Ubuntu's.
>>>
>>> Lucas, could you please run the rebuild with the three patches?
>>
>> Hi,
>>
>> Results are available at
>> https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/
>>
>> I did a full rebuild with bindnow and PIE enabled, then rebuilt all
>> failed packages with a pristine unstable chroot.
>>
>> You can take a look at
>> https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/diff.txt
>> and grep for "OK Failed" (failed with PIE+bindnow, built fine in
>> unstable). (There are 1188 packages failing to build)
>>
>> Logs for both builds are available in the respective subdirectories.
>>
>> Lucas
>>
>
> Are you sure these are correct? The numbers for PIE+bindnow are a lot
> higher than what we see in Ubuntu, for same unmodified packages.
>
> E.g. looking at http://qa.ubuntuwire.org/ftbfs/
>
> amd64/ppc64el/s390x have PIE+bindnow enabled, and
> i386/armhf/arm64/powerpc do not. here is nothing in the thousands
> range. There might be a dozen packages with PIE+bindnow fixes in
> ubuntu, that's not in debian, but that amount cannot be more than a
> dozen or two.
>
> Looking at e.g. ace buildlog the PIE+bindnow has this:
> -fno-PIE -no-pie -Wl,-z,relro -Wl,-z,now
>
> Which is bindnow without pie, instead of with pie.

Ubuntu sets bindnow in GCC in Ubuntu while it is set by dpkg in Debian.

Ace requests those flags in d/rules:
...
export DEB_BUILD_MAINT_OPTIONS =
hardening=+format,+fortify,+stackprotector,+relro,+bindnow,-pie
...

I have just started looking at the logs and I think I have a patch to GCC which
will fix many of the FTBFS-es.
I'm also looking at the GCC diff in Ubuntu.

Cheers,
Balint

>
> And same ace build fine in Ubuntu, with PIE+bindnow on the relevant
> arches. https://launchpad.net/ubuntu/+source/ace/6.3.3+dfsg-1.1
>
> --
> Regards,
>
> Dimitri.

Re: Porter roll call for Debian Stretch

[Dimitri John Ledkov]

Hello,

On 30 August 2016 at 23:07, Lucas Nussbaum  wrote:
> On 22/08/16 at 19:12 +0200, Bálint Réczey wrote:
>> Hi Guillem,
>>
>> 2016-08-21 14:02 GMT+02:00 Guillem Jover :
>> > Hi!
>> >
>> > On Sun, 2016-08-21 at 10:24:42 +0200, Bálint Réczey wrote:
>> >> I'm testing a set of patches [2] for gcc and dpkg which enable bindnow 
>> >> for all
>> >> arches and PIE for amd64, ppc64el and s390x in sync with Ubuntu.
>> >>
>> >> My assumption was that this set of architectures need the least amount of
>> >> additional work since they are tested already in Ubuntu, but I would be 
>> >> happy
>> >> if more ports would opt in for PIE.
>> >>
>> >> I plan filing wishlist bugs against dpkg and gcc with the patches
>> >> after I rebuilt a
>> >> few packages with the defaults.
>> >
>> > TBH I think PIE should in fact be safer to enable globally than
>> > bindnow, because the latter changes the run-time behavior and things
>> > might break (perhaps even silently) when failing to load plugins
>> > or similar.
>>
>> Yes, in that sense enabling PIE is safer indeed. Regarding bindnow
>> I don't expect too many surprises either, since other distributions
>> already tested enabling bindnow and probably they found
>> most issues.
>>
>> >
>> > From dpkg PoV enabling both, would at least require a full-archive
>> > rebuild, for bindnow ideally also a full autopkgtest run (as the
>> > updated dpkg FAQ states now, after Lucas Nussbaum approached me some
>> > weeks ago mentioning he was willing to do such archive-wide rebuild).
>>
>> The patches at [2] seem to work well and since you expressed that you would
>> prefer changing both toolchain and dpkg, too, I would like to suggest running
>> the rebuild with those patches.
>>
>> I think Matthias would be OK with the patch since it is very small and brings
>> Debian's gcc closer to Ubuntu's.
>>
>> Lucas, could you please run the rebuild with the three patches?
>
> Hi,
>
> Results are available at
> https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/
>
> I did a full rebuild with bindnow and PIE enabled, then rebuilt all
> failed packages with a pristine unstable chroot.
>
> You can take a look at
> https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/diff.txt
> and grep for "OK Failed" (failed with PIE+bindnow, built fine in
> unstable). (There are 1188 packages failing to build)
>
> Logs for both builds are available in the respective subdirectories.
>
> Lucas
>

Are you sure these are correct? The numbers for PIE+bindnow are a lot
higher than what we see in Ubuntu, for same unmodified packages.

E.g. looking at http://qa.ubuntuwire.org/ftbfs/

amd64/ppc64el/s390x have PIE+bindnow enabled, and
i386/armhf/arm64/powerpc do not. here is nothing in the thousands
range. There might be a dozen packages with PIE+bindnow fixes in
ubuntu, that's not in debian, but that amount cannot be more than a
dozen or two.

Looking at e.g. ace buildlog the PIE+bindnow has this:
-fno-PIE -no-pie -Wl,-z,relro -Wl,-z,now

Which is bindnow without pie, instead of with pie.

And same ace build fine in Ubuntu, with PIE+bindnow on the relevant
arches. https://launchpad.net/ubuntu/+source/ace/6.3.3+dfsg-1.1

-- 
Regards,

Dimitri.

Re: Porter roll call for Debian Stretch

[Lucas Nussbaum]

On 22/08/16 at 19:12 +0200, Bálint Réczey wrote:
> Hi Guillem,
> 
> 2016-08-21 14:02 GMT+02:00 Guillem Jover :
> > Hi!
> >
> > On Sun, 2016-08-21 at 10:24:42 +0200, Bálint Réczey wrote:
> >> I'm testing a set of patches [2] for gcc and dpkg which enable bindnow for 
> >> all
> >> arches and PIE for amd64, ppc64el and s390x in sync with Ubuntu.
> >>
> >> My assumption was that this set of architectures need the least amount of
> >> additional work since they are tested already in Ubuntu, but I would be 
> >> happy
> >> if more ports would opt in for PIE.
> >>
> >> I plan filing wishlist bugs against dpkg and gcc with the patches
> >> after I rebuilt a
> >> few packages with the defaults.
> >
> > TBH I think PIE should in fact be safer to enable globally than
> > bindnow, because the latter changes the run-time behavior and things
> > might break (perhaps even silently) when failing to load plugins
> > or similar.
> 
> Yes, in that sense enabling PIE is safer indeed. Regarding bindnow
> I don't expect too many surprises either, since other distributions
> already tested enabling bindnow and probably they found
> most issues.
> 
> >
> > From dpkg PoV enabling both, would at least require a full-archive
> > rebuild, for bindnow ideally also a full autopkgtest run (as the
> > updated dpkg FAQ states now, after Lucas Nussbaum approached me some
> > weeks ago mentioning he was willing to do such archive-wide rebuild).
> 
> The patches at [2] seem to work well and since you expressed that you would
> prefer changing both toolchain and dpkg, too, I would like to suggest running
> the rebuild with those patches.
> 
> I think Matthias would be OK with the patch since it is very small and brings
> Debian's gcc closer to Ubuntu's.
> 
> Lucas, could you please run the rebuild with the three patches?

Hi,

Results are available at
https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/

I did a full rebuild with bindnow and PIE enabled, then rebuilt all
failed packages with a pristine unstable chroot.

You can take a look at
https://people.debian.org/~lucas/logs/2016/08/30/pie-bindnow-20160830/diff.txt
and grep for "OK Failed" (failed with PIE+bindnow, built fine in
unstable). (There are 1188 packages failing to build)

Logs for both builds are available in the respective subdirectories.

Lucas

Re: Porter roll call for Debian Stretch

[Fernando Seiti Furusato]

Hi,

I am an active porter for the following architectures and I intend
to continue this for the lifetime of the Stretch release (est. end
of 2020):

For ppc64el, I
- test most packages on this architecture
- run a Debian testing or unstable system on port that I use regularly
- triage arch-specific bugs
- fix arch-related bugs
- test d-i with some regularity
- am currently woking on providing ppc64el hardware so to include the
arch to ci.d.n

I am not a DD/DM.

I believe the port is ready to have -fPIE/-pie enabled by default.

Fernando Seiti Furusato




signature.asc
Description: OpenPGP digital signature

Re: Porter roll call for Debian Stretch

[YunQiang Su]

  Hi,

  I am an active porter for the following architectures and I intend
  to continue this for the lifetime of the Stretch release (est. end
  of 2020):

  For mips, mipsel and mips64el, I
  - test most packages on this architecture
  - run a Debian testing or unstable system on port that I use regularly
  - fix toolchain issues
  - triage arch-specific bugs
  - fix arch-related bugs
  - triage d-i bugs
  - test d-i regularly
  - fix d-i bugs/issues
  - maintain buildds
  - maintain/provide hardware for (or assist with) automated tests on ci.d.n,
jenkins.d.n (etc.)
  - run other automated tests outside the Debian QA services
 Run daily build test
 Run autopkgtest
  - ...

  I am a DD

  I believe the ports *are* ready to have -fPIE/-pie enabled by default.

  YunQiang Su

On Sun, Aug 28, 2016 at 11:53 PM, Aurelien Jarno  wrote:
> On 2016-08-17 22:05, ni...@thykier.net wrote:
>> Hi,
>>
>> Like last release, we are doing a roll call for porters of all release
>> architectures.  If you are an active porter behind one of the [release
>
> Does it really concerns *all* release architectures? Traditionally amd64
> and i386 have been granted waivers as "the toolchain maintainers are
> happy to support" these architectures "as-is". That said the toolchain
> maintainers do not fix ports specific bugs outside of the toolchain.
>
> While I fully agree that we can have a waiver for amd64 due to being the
> de facto standard architecture, it seems that a few leaf packages do
> not build on i386 and that we have no porters to fix them. That is
> probably still fine, but I wonder how fast the number of such packages
> will increase in the future.
>
>> architectures] for the entire lifetime of Debian Stretch (est. end of
>> 2020), please respond with a signed email containing the following
>
> What is the relation between the end of support of Stretch...
>
>> before Friday, the 9th of September:
>
>>  * Which architectures are you committing to be an active porter for?
>>  * Please describe recent relevant porter contributions.
>>  * Are you running/using Debian testing or sid on said port(s)?
>>  * Are you testing/patching d-i for the port(s)?
>>  * If we were to enable -fPIE/-pie by default in GCC-6, should that change
>>also apply to this port? [0]
>
> ... and the above questions?
>
> I fully agree that running testing/sid, fixing bugs or working on d-i up
> to the release of Stretch will improve its quality. But after the
> release it will improve the quality of Buster and later Bullseye. On the
> other hand running testing/sid after the release of Stretch will not
> help to catch bugs that can be fixed through a point release.
>
> Aurelien
>
> --
> Aurelien Jarno  GPG: 4096R/1DDD8C9B
> aurel...@aurel32.net http://www.aurel32.net



-- 
YunQiang Su

Re: Porter roll call for Debian Stretch

[Yunqiang Su]

Sorry for the previous post without signature.

  Hi,

  I am an active porter for the following architectures and I intend
  to continue this for the lifetime of the Stretch release (est. end
  of 2020):

  For mips, mipsel and mips64el, I
  - test most packages on this architecture
  - run a Debian testing or unstable system on port that I use regularly
  - fix toolchain issues
  - triage arch-specific bugs
  - fix arch-related bugs
  - triage d-i bugs
  - test d-i regularly
  - fix d-i bugs/issues
  - maintain buildds
  - maintain/provide hardware for (or assist with) automated tests on ci.d.n,
jenkins.d.n (etc.)
  - run other automated tests outside the Debian QA services
 Run daily build test
 Run autopkgtest
  - ...

  I am a DD

  I believe the ports *are* ready to have -fPIE/-pie enabled by default.

  YunQiang Su

> 在 2016年8月31日，00:04，YunQiang Su  写道：
> 
>  Hi,
> 
>  I am an active porter for the following architectures and I intend
>  to continue this for the lifetime of the Stretch release (est. end
>  of 2020):
> 
>  For mips, mipsel and mips64el, I
>  - test most packages on this architecture
>  - run a Debian testing or unstable system on port that I use regularly
>  - fix toolchain issues
>  - triage arch-specific bugs
>  - fix arch-related bugs
>  - triage d-i bugs
>  - test d-i regularly
>  - fix d-i bugs/issues
>  - maintain buildds
>  - maintain/provide hardware for (or assist with) automated tests on ci.d.n,
>jenkins.d.n (etc.)
>  - run other automated tests outside the Debian QA services
> Run daily build test
> Run autopkgtest
>  - ...
> 
>  I am a DD
> 
>  I believe the ports *are* ready to have -fPIE/-pie enabled by default.
> 
>  YunQiang Su
> 
> On Sun, Aug 28, 2016 at 11:53 PM, Aurelien Jarno  wrote:
>> On 2016-08-17 22:05, ni...@thykier.net wrote:
>>> Hi,
>>> 
>>> Like last release, we are doing a roll call for porters of all release
>>> architectures.  If you are an active porter behind one of the [release
>> 
>> Does it really concerns *all* release architectures? Traditionally amd64
>> and i386 have been granted waivers as "the toolchain maintainers are
>> happy to support" these architectures "as-is". That said the toolchain
>> maintainers do not fix ports specific bugs outside of the toolchain.
>> 
>> While I fully agree that we can have a waiver for amd64 due to being the
>> de facto standard architecture, it seems that a few leaf packages do
>> not build on i386 and that we have no porters to fix them. That is
>> probably still fine, but I wonder how fast the number of such packages
>> will increase in the future.
>> 
>>> architectures] for the entire lifetime of Debian Stretch (est. end of
>>> 2020), please respond with a signed email containing the following
>> 
>> What is the relation between the end of support of Stretch...
>> 
>>> before Friday, the 9th of September:
>> 
>>> * Which architectures are you committing to be an active porter for?
>>> * Please describe recent relevant porter contributions.
>>> * Are you running/using Debian testing or sid on said port(s)?
>>> * Are you testing/patching d-i for the port(s)?
>>> * If we were to enable -fPIE/-pie by default in GCC-6, should that change
>>>   also apply to this port? [0]
>> 
>> ... and the above questions?
>> 
>> I fully agree that running testing/sid, fixing bugs or working on d-i up
>> to the release of Stretch will improve its quality. But after the
>> release it will improve the quality of Buster and later Bullseye. On the
>> other hand running testing/sid after the release of Stretch will not
>> help to catch bugs that can be fixed through a point release.
>> 
>> Aurelien
>> 
>> --
>> Aurelien Jarno  GPG: 4096R/1DDD8C9B
>> aurel...@aurel32.net http://www.aurel32.net
> 
> 
> 
> --
> YunQiang Su



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Porter roll call for Debian Stretch

[Aurelien Jarno]

On 2016-08-17 22:05, ni...@thykier.net wrote:
> Hi,
> 
> Like last release, we are doing a roll call for porters of all release
> architectures.  If you are an active porter behind one of the [release

Does it really concerns *all* release architectures? Traditionally amd64
and i386 have been granted waivers as "the toolchain maintainers are
happy to support" these architectures "as-is". That said the toolchain
maintainers do not fix ports specific bugs outside of the toolchain.

While I fully agree that we can have a waiver for amd64 due to being the
de facto standard architecture, it seems that a few leaf packages do
not build on i386 and that we have no porters to fix them. That is
probably still fine, but I wonder how fast the number of such packages
will increase in the future.

> architectures] for the entire lifetime of Debian Stretch (est. end of
> 2020), please respond with a signed email containing the following

What is the relation between the end of support of Stretch...

> before Friday, the 9th of September:
 
>  * Which architectures are you committing to be an active porter for?
>  * Please describe recent relevant porter contributions.
>  * Are you running/using Debian testing or sid on said port(s)?
>  * Are you testing/patching d-i for the port(s)?
>  * If we were to enable -fPIE/-pie by default in GCC-6, should that change
>also apply to this port? [0]

... and the above questions?

I fully agree that running testing/sid, fixing bugs or working on d-i up
to the release of Stretch will improve its quality. But after the
release it will improve the quality of Buster and later Bullseye. On the
other hand running testing/sid after the release of Stretch will not
help to catch bugs that can be fixed through a point release.

Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net


signature.asc
Description: PGP signature

Re: Porter roll call for Debian Stretch

[Balint Reczey]

On 08/22/2016 07:12 PM, Bálint Réczey wrote:
> Hi Guillem,
> 
> 2016-08-21 14:02 GMT+02:00 Guillem Jover :
>> Hi!
>>
>> On Sun, 2016-08-21 at 10:24:42 +0200, Bálint Réczey wrote:
>>> I'm testing a set of patches [2] for gcc and dpkg which enable bindnow for 
>>> all
>>> arches and PIE for amd64, ppc64el and s390x in sync with Ubuntu.
>>>
>>> My assumption was that this set of architectures need the least amount of
>>> additional work since they are tested already in Ubuntu, but I would be 
>>> happy
>>> if more ports would opt in for PIE.
>>>
>>> I plan filing wishlist bugs against dpkg and gcc with the patches
>>> after I rebuilt a
>>> few packages with the defaults.
>>
>> TBH I think PIE should in fact be safer to enable globally than
>> bindnow, because the latter changes the run-time behavior and things
>> might break (perhaps even silently) when failing to load plugins
>> or similar.
> 
> Yes, in that sense enabling PIE is safer indeed. Regarding bindnow
> I don't expect too many surprises either, since other distributions
> already tested enabling bindnow and probably they found
> most issues.
> 
>>
>> From dpkg PoV enabling both, would at least require a full-archive
>> rebuild, for bindnow ideally also a full autopkgtest run (as the
>> updated dpkg FAQ states now, after Lucas Nussbaum approached me some
>> weeks ago mentioning he was willing to do such archive-wide rebuild).
> 
> The patches at [2] seem to work well and since you expressed that you would
> prefer changing both toolchain and dpkg, too, I would like to suggest running
> the rebuild with those patches.
> 
> I think Matthias would be OK with the patch since it is very small and brings
> Debian's gcc closer to Ubuntu's.
> 
> Lucas, could you please run the rebuild with the three patches?
> 
> I'll attach the patches to the bug reports.

For the record I have opened #835146, #835148 and #835149 against dpkg
and gcc-6 with the patches.

> 
> [2] https://people.debian.org/~rbalint/ppa/pie-bindnow/
> 

Re: Porter roll call for Debian Stretch

[Bálint Réczey]

Hi Guillem,

2016-08-21 14:02 GMT+02:00 Guillem Jover :
> Hi!
>
> On Sun, 2016-08-21 at 10:24:42 +0200, Bálint Réczey wrote:
>> I'm testing a set of patches [2] for gcc and dpkg which enable bindnow for 
>> all
>> arches and PIE for amd64, ppc64el and s390x in sync with Ubuntu.
>>
>> My assumption was that this set of architectures need the least amount of
>> additional work since they are tested already in Ubuntu, but I would be happy
>> if more ports would opt in for PIE.
>>
>> I plan filing wishlist bugs against dpkg and gcc with the patches
>> after I rebuilt a
>> few packages with the defaults.
>
> TBH I think PIE should in fact be safer to enable globally than
> bindnow, because the latter changes the run-time behavior and things
> might break (perhaps even silently) when failing to load plugins
> or similar.

Yes, in that sense enabling PIE is safer indeed. Regarding bindnow
I don't expect too many surprises either, since other distributions
already tested enabling bindnow and probably they found
most issues.

>
> From dpkg PoV enabling both, would at least require a full-archive
> rebuild, for bindnow ideally also a full autopkgtest run (as the
> updated dpkg FAQ states now, after Lucas Nussbaum approached me some
> weeks ago mentioning he was willing to do such archive-wide rebuild).

The patches at [2] seem to work well and since you expressed that you would
prefer changing both toolchain and dpkg, too, I would like to suggest running
the rebuild with those patches.

I think Matthias would be OK with the patch since it is very small and brings
Debian's gcc closer to Ubuntu's.

Lucas, could you please run the rebuild with the three patches?

I'll attach the patches to the bug reports.

Cheers,
Balint

[2] https://people.debian.org/~rbalint/ppa/pie-bindnow/

Re: Porter roll call for Debian Stretch

[Guillem Jover]

Hi!

On Sun, 2016-08-21 at 10:24:42 +0200, Bálint Réczey wrote:
> I'm testing a set of patches [2] for gcc and dpkg which enable bindnow for all
> arches and PIE for amd64, ppc64el and s390x in sync with Ubuntu.
>
> My assumption was that this set of architectures need the least amount of
> additional work since they are tested already in Ubuntu, but I would be happy
> if more ports would opt in for PIE.
> 
> I plan filing wishlist bugs against dpkg and gcc with the patches
> after I rebuilt a
> few packages with the defaults.

TBH I think PIE should in fact be safer to enable globally than
bindnow, because the latter changes the run-time behavior and things
might break (perhaps even silently) when failing to load plugins
or similar.

From dpkg PoV enabling both, would at least require a full-archive
rebuild, for bindnow ideally also a full autopkgtest run (as the
updated dpkg FAQ states now, after Lucas Nussbaum approached me some
weeks ago mentioning he was willing to do such archive-wide rebuild).

Thanks,
Guillem

Re: Porter roll call for Debian Stretch

[Guillem Jover]

Hi!

On Sun, 2016-08-21 at 08:22:09 +0200, Niels Thykier wrote:
> Kurt Roeckx:
> > On Wed, Aug 17, 2016 at 10:05:06PM +0200, ni...@thykier.net wrote:
> >>  * If we were to enable -fPIE/-pie by default in GCC-6, should that change
> >>also apply to this port? [0]
> > 
> > If -fPIE is the default will -fPIC override it?
> > 
> > It will also default to tell the linker to use -pie, but then
> > don't do it when you want to create a shared library?
> 
> I believe so.  At least, Ubuntu made PIE default in their compiler
> without having to change all SO packages to still build.

As I mentioned on IRC, the Gentoo people seems to have done so as well
in their Gentoo Hardened Toolchain project
.

> Admittedly, it could also be that GCC uses some built "compiler spec"
> files for this case (a la an implicit "-specs=$FILE"), which are similar
> to those Redhat uses for this purposes (see [1] for an example of such
> files).

I think there are some bugs tracked in Fedora about this
, And this
is what I based the dpkg patch on as a potential alternative instead
of modifying our toolchain, which I think would be preferable. But see
below.

> >>From what I understand, depending on what the .o file is going to
> > be used for you want different things:
> > - shared library: -fPIC
> > - executable: -fPIC or -fPIE both work, but prefer -fPIE
> > - static library: Same as executables
> > 
> > For static libraries we now have a policy to not use -fPIC,
> > should that then get replaced by using -fPIE?

> Honestly, I had not thought about that.  From the looks of it, de facto
> we will end up with -fPIE being the default for static libraries as well.
>   I would be supporting that change on the assumption that it requires
> less work from individual maintainers (and presumably has no notable
> downsides in the final result).

I think that's what we are getting right now for any package enabling
the "pie" build flags feature anyway.

> [1] Example spec files for this case seems to be something like:
> 
> pie-compile.specs
> """
> *cc1_options:
> + %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}}
> """
> 
> pie-link.specs:
> """
> *self_spec:
> + %{!shared:%{!r:-pie}}
> """
> 
> NB: I manually carved them out of a diff without testing them.

That would be
,
which neither I tested. Testing should first be done for gcc, g++, gcj,
gobjc, gobjc++, gccgo (although I've not managed to use shared libraries
with latest releases, but used to be able to) and gfortran. If someone
wants to test these first to see if it works and then later on do an
archive rebuild, that would also be appreciated.

At that point I could then at least switch the current implementation
so that we can just enable the "pie" feature per package regardless of
it building prorgrams or shared libraries. Enabling it by default
globally would require the usual process described in the dpkg FAQ.

Thanks,
Guillem

Re: Porter roll call for Debian Stretch

[Bálint Réczey]

Hi,

2016-08-21 8:22 GMT+02:00 Niels Thykier :
> Kurt Roeckx:
>> On Wed, Aug 17, 2016 at 10:05:06PM +0200, ni...@thykier.net wrote:
>>>  * If we were to enable -fPIE/-pie by default in GCC-6, should that change
>>>also apply to this port? [0]
>>
>> If -fPIE is the default will -fPIC override it?
>>
>> It will also default to tell the linker to use -pie, but then
>> don't do it when you want to create a shared library?
>>
>
> I believe so.  At least, Ubuntu made PIE default in their compiler
> without having to change all SO packages to still build.
>
> Admittedly, it could also be that GCC uses some built "compiler spec"
> files for this case (a la an implicit "-specs=$FILE"), which are similar
> to those Redhat uses for this purposes (see [1] for an example of such
> files).
>
> Regardless, it seems to "just work(TM)" if you pass the proper flags
> when compiling your SOs.
>
>>>From what I understand, depending on what the .o file is going to
>> be used for you want different things:
>> - shared library: -fPIC
>> - executable: -fPIC or -fPIE both work, but prefer -fPIE
>> - static library: Same as executables
>>
>> For static libraries we now have a policy to not use -fPIC,
>> should that then get replaced by using -fPIE?
>>
>>
>>
>> Kurt
>>
>
> Honestly, I had not thought about that.  From the looks of it, de facto
> we will end up with -fPIE being the default for static libraries as well.
>   I would be supporting that change on the assumption that it requires
> less work from individual maintainers (and presumably has no notable
> downsides in the final result).

I'm testing a set of patches [2] for gcc and dpkg which enable bindnow for all
arches and PIE for amd64, ppc64el and s390x in sync with Ubuntu.

My assumption was that this set of architectures need the least amount of
additional work since they are tested already in Ubuntu, but I would be happy
if more ports would opt in for PIE.

I plan filing wishlist bugs against dpkg and gcc with the patches
after I rebuilt a
few packages with the defaults.

Cheers,
Balint

[2] https://people.debian.org/~rbalint/ppa/pie-bindnow/


>
> Thanks,
> ~Niels
>
> [1] Example spec files for this case seems to be something like:
>
> pie-compile.specs
> """
> *cc1_options:
> + %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}}
> """
>
> pie-link.specs:
> """
> *self_spec:
> + %{!shared:%{!r:-pie}}
> """
>
> NB: I manually carved them out of a diff without testing them.
>

Re: Porter roll call for Debian Stretch

[Niels Thykier]

Kurt Roeckx:
> On Wed, Aug 17, 2016 at 10:05:06PM +0200, ni...@thykier.net wrote:
>>  * If we were to enable -fPIE/-pie by default in GCC-6, should that change
>>also apply to this port? [0]
> 
> If -fPIE is the default will -fPIC override it?
> 
> It will also default to tell the linker to use -pie, but then
> don't do it when you want to create a shared library?
> 

I believe so.  At least, Ubuntu made PIE default in their compiler
without having to change all SO packages to still build.

Admittedly, it could also be that GCC uses some built "compiler spec"
files for this case (a la an implicit "-specs=$FILE"), which are similar
to those Redhat uses for this purposes (see [1] for an example of such
files).

Regardless, it seems to "just work(TM)" if you pass the proper flags
when compiling your SOs.

>>From what I understand, depending on what the .o file is going to
> be used for you want different things:
> - shared library: -fPIC
> - executable: -fPIC or -fPIE both work, but prefer -fPIE
> - static library: Same as executables
> 
> For static libraries we now have a policy to not use -fPIC,
> should that then get replaced by using -fPIE?
> 
> 
> 
> Kurt
> 

Honestly, I had not thought about that.  From the looks of it, de facto
we will end up with -fPIE being the default for static libraries as well.
  I would be supporting that change on the assumption that it requires
less work from individual maintainers (and presumably has no notable
downsides in the final result).

Thanks,
~Niels

[1] Example spec files for this case seems to be something like:

pie-compile.specs
"""
*cc1_options:
+ %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}}
"""

pie-link.specs:
"""
*self_spec:
+ %{!shared:%{!r:-pie}}
"""

NB: I manually carved them out of a diff without testing them.

Re: Porter roll call for Debian Stretch

[Kurt Roeckx]

On Wed, Aug 17, 2016 at 10:05:06PM +0200, ni...@thykier.net wrote:
>  * If we were to enable -fPIE/-pie by default in GCC-6, should that change
>also apply to this port? [0]

If -fPIE is the default will -fPIC override it?

It will also default to tell the linker to use -pie, but then
don't do it when you want to create a shared library?

>From what I understand, depending on what the .o file is going to
be used for you want different things:
- shared library: -fPIC
- executable: -fPIC or -fPIE both work, but prefer -fPIE
- static library: Same as executables

For static libraries we now have a policy to not use -fPIC,
should that then get replaced by using -fPIE?



Kurt

Re: Porter roll call for Debian Stretch

[Niels Thykier]

Martin Michlmayr:
> * ni...@thykier.net  [2016-08-17 22:05]:
>> 2020), please respond with a signed email containing the following
>> before Friday, the 9th of September:
> 
> Can you please specify where to respond to?  I don't think dozens of
> emails to -ports and -devel make any sense.
> 

Ah, sorry I had indeed forgotten to set Reply-To. :)

> Maybe debian-release with CC debian- or to you personally and
> you'll collect the info?
> 

Please send the replies to debian-release. :)

Thanks,
~Niels

Re: Porter roll call for Debian Stretch

[Martin Michlmayr]

* ni...@thykier.net  [2016-08-17 22:05]:
> 2020), please respond with a signed email containing the following
> before Friday, the 9th of September:

Can you please specify where to respond to?  I don't think dozens of
emails to -ports and -devel make any sense.

Maybe debian-release with CC debian- or to you personally and
you'll collect the info?

-- 
Martin Michlmayr
http://www.cyrius.com/