Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Ian Campbell]

On Tue, 2016-11-01 at 22:54 +0200, Adrian Bunk wrote:
> I do not see any reason for waiting instead of sending the binNMU 
> request right now.

I didn't see any movement on the dpkg front so I went ahead and did so:
#843139.

Thanks,
Ian.

Lua [Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)]

[Peter Colberg]

On Tue, Nov 01, 2016 at 10:54:33PM +0200, Adrian Bunk wrote:
> LUA

The language is named "Lua", which means "moon" in Portuguese.

https://www.lua.org/about.html#name

LUA is an acronym for "Lua Uppercase Accident" ;-).

Peter

Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Adrian Bunk]

On Mon, Oct 31, 2016 at 03:23:51PM +0100, Bálint Réczey wrote:
> Hi Ian,
> 
> 2016-10-31 14:19 GMT+01:00 Ian Campbell :
> > On Mon, 2016-10-31 at 12:17 +0100, Bálint Réczey wrote:
> >> 2016-10-31 10:38 GMT+01:00 Ian Campbell :
> >> > If possible I'd also prefer a solution which fixed qcontrol-static
> >> > without opting out for the normal dynamic/non-udeb binary.
> >>
> >> I ran the build tests on amd64, this is why I have not caught this error.
> >> A rebuild of lua5.1 should fix the problem.
> >>
> >> Dpkg changes are on their way, too in the next days. I would ask
> >> for a binNMU after dpkg is updated.
> >
> > Thanks, to check I understand: I should wait for #835146 to be fixed in
> > sid (which is expected to happen via a dpkg upload in the next days)
> > and then ask for a binNMU of lua5.1 then qcontrol (or maybe I have
> > reason to upload a newer qcontrol anyway, we'll see)?
> 
> Yes, this should fix qcontrol and maybe you don't even have to ask for
> rebuilds.  I think there is already an archive-wide rebuild planned to
> make reproducibility-related changes in dpkg take effect thus I suggest
> waiting a few days to see if you need to ask for binNMUs.

I do not see any reason for waiting instead of sending the binNMU 
request right now.

Based on the error message the only problem for qcontrol is that the
LUA 5.1 static library has not been compiled with the toolchain that
defaults to PIE, and the one and only change required to fix the build
of qcontrol should therefore be to request a binNMU of lua5.1

> Cheers,
> Balint

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Bálint Réczey]

Hi Ian,

2016-10-31 14:19 GMT+01:00 Ian Campbell :
> On Mon, 2016-10-31 at 12:17 +0100, Bálint Réczey wrote:
>> 2016-10-31 10:38 GMT+01:00 Ian Campbell :
>> > If possible I'd also prefer a solution which fixed qcontrol-static
>> > without opting out for the normal dynamic/non-udeb binary.
>>
>> I ran the build tests on amd64, this is why I have not caught this error.
>> A rebuild of lua5.1 should fix the problem.
>>
>> Dpkg changes are on their way, too in the next days. I would ask
>> for a binNMU after dpkg is updated.
>
> Thanks, to check I understand: I should wait for #835146 to be fixed in
> sid (which is expected to happen via a dpkg upload in the next days)
> and then ask for a binNMU of lua5.1 then qcontrol (or maybe I have
> reason to upload a newer qcontrol anyway, we'll see)?

Yes, this should fix qcontrol and maybe you don't even have to ask for
rebuilds.  I think there is already an archive-wide rebuild planned to
make reproducibility-related changes in dpkg take effect thus I suggest
waiting a few days to see if you need to ask for binNMUs.

Cheers,
Balint

Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Ian Campbell]

On Mon, 2016-10-31 at 12:17 +0100, Bálint Réczey wrote:
> 2016-10-31 10:38 GMT+01:00 Ian Campbell :
> > If possible I'd also prefer a solution which fixed qcontrol-static
> > without opting out for the normal dynamic/non-udeb binary.
> 
> I ran the build tests on amd64, this is why I have not caught this error.
> A rebuild of lua5.1 should fix the problem.
> 
> Dpkg changes are on their way, too in the next days. I would ask
> for a binNMU after dpkg is updated.

Thanks, to check I understand: I should wait for #835146 to be fixed in
sid (which is expected to happen via a dpkg upload in the next days)
and then ask for a binNMU of lua5.1 then qcontrol (or maybe I have
reason to upload a newer qcontrol anyway, we'll see)?

Ian.

Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Bálint Réczey]

2016-10-31 12:52 GMT+01:00 Henrique de Moraes Holschuh :
> On Mon, 31 Oct 2016, Ian Campbell wrote:
>> On Mon, 2016-10-31 at 10:27 +0100, Jérémy Lal wrote:
>> > export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-pie
>> > ?
>>
>> Thanks, but that doesn't appear to help, I think the issue is to do
>> with the changed default in gcc rather than the hardening settings
>
> Which is actually a bug: hardening=-pie (now) needs to actually inject
> flags to disable PIE, instead.

It is indeed a bug, but not an easily solvable one:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=835149

Cheers,
Balint

Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Henrique de Moraes Holschuh]

On Mon, 31 Oct 2016, Ian Campbell wrote:
> On Mon, 2016-10-31 at 10:27 +0100, Jérémy Lal wrote:
> > export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-pie
> > ?
> 
> Thanks, but that doesn't appear to help, I think the issue is to do
> with the changed default in gcc rather than the hardening settings

Which is actually a bug: hardening=-pie (now) needs to actually inject
flags to disable PIE, instead.

-- 
  Henrique Holschuh

Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Bálint Réczey]

Hi Ian,

2016-10-31 10:38 GMT+01:00 Ian Campbell :
> On Mon, 2016-10-31 at 10:27 +0100, Jérémy Lal wrote:
>> export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-pie
>> ?
>
> Thanks, but that doesn't appear to help, I think the issue is to do
> with the changed default in gcc rather than the hardening settings
> injected into the build by dpkg-buildpackage/dpkg-buildflags (or
> whatever it is which does it).

Yes, this is the case.

>
> If possible I'd also prefer a solution which fixed qcontrol-static
> without opting out for the normal dynamic/non-udeb binary.

I ran the build tests on amd64, this is why I have not caught this error.
A rebuild of lua5.1 should fix the problem.

Dpkg changes are on their way, too in the next days. I would ask
for a binNMU after dpkg is updated.

Cheers,
Balint

Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Ian Campbell]

On Mon, 2016-10-31 at 10:27 +0100, Jérémy Lal wrote:
> export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-pie
> ?

Thanks, but that doesn't appear to help, I think the issue is to do
with the changed default in gcc rather than the hardening settings
injected into the build by dpkg-buildpackage/dpkg-buildflags (or
whatever it is which does it).

If possible I'd also prefer a solution which fixed qcontrol-static
without opting out for the normal dynamic/non-udeb binary.

Ian.

Re: Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Jérémy Lal]

export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-pie
?

2016-10-31 10:21 GMT+01:00 Ian Campbell <i...@debian.org>:

> Hi,
>
> I maintain qcontrol[0] which builds on armel and armhf only (it's a
> tool specific to certain NAS machines). It links a version against
> liblua statically for use in a udeb (the resulting binary is dynamic,
> it is only liblua5.1 which is linked statically).
>
> It seems that on armhf my latest upload has failed to build for reasons
> which look to be related to the PIE by default transition[1]:
>
> cc -Wl,-z,relro -g qcontrol.o system.o qnap-pic.o ts209.o ts219.o
> ts409.o ts41x.o evdev.o a125.o synology.o /usr/lib/$(dpkg-architecture
> -qDEB_HOST_MULTIARCH)/liblua5.1.a -lpthread -lm -ldl -o qcontrol-static
> /usr/bin/ld: /usr/lib/arm-linux-gnueabihf/liblua5.1.a(lapi.o):
> relocation R_ARM_THM_MOVW_ABS_NC against `luaO_nilobject_' can not be used
> when making a shared object; recompile with -fPIC
>
> I've checked the wiki and the recent '"PIE by default" transition is
> underway -- wiki needs updating' thread on d-devel but I'm none the
> wiser on what my next course of action should be.
>
> Should I be filing a bug against liblua asking for specific changes?
> asking for a binNMU? Making some change to qcontrol's build (maybe
> -fno-pic or -fno-pie, I've played with this a little with no luck so
> far, I'll keep poking at it though)?
>
> I don't see any preexisting bugs against liblua5.1 around the need for
> changes or rebuild.
>
> Thanks for any advice,
>
> Ian.
>
> [0] https://tracker.debian.org/pkg/qcontrol
> [1] https://buildd.debian.org/status/fetch.php?pkg=qcontrol;
> arch=armhf=0.5.5-1=1477849051
> [2] https://wiki.debian.org/Hardening/PIEByDefaultTransition
>
>

Static linking and fPIC (Was: Re: "PIE by default" transition is underway -- wiki needs updating)

[Ian Campbell]

Hi,

I maintain qcontrol[0] which builds on armel and armhf only (it's a
tool specific to certain NAS machines). It links a version against
liblua statically for use in a udeb (the resulting binary is dynamic,
it is only liblua5.1 which is linked statically).

It seems that on armhf my latest upload has failed to build for reasons
which look to be related to the PIE by default transition[1]:

cc -Wl,-z,relro -g qcontrol.o system.o qnap-pic.o ts209.o ts219.o ts409.o 
ts41x.o evdev.o a125.o synology.o /usr/lib/$(dpkg-architecture 
-qDEB_HOST_MULTIARCH)/liblua5.1.a -lpthread -lm -ldl -o qcontrol-static
/usr/bin/ld: /usr/lib/arm-linux-gnueabihf/liblua5.1.a(lapi.o): relocation 
R_ARM_THM_MOVW_ABS_NC against `luaO_nilobject_' can not be used when making a 
shared object; recompile with -fPIC

I've checked the wiki and the recent '"PIE by default" transition is
underway -- wiki needs updating' thread on d-devel but I'm none the
wiser on what my next course of action should be.

Should I be filing a bug against liblua asking for specific changes?
asking for a binNMU? Making some change to qcontrol's build (maybe
-fno-pic or -fno-pie, I've played with this a little with no luck so
far, I'll keep poking at it though)?

I don't see any preexisting bugs against liblua5.1 around the need for
changes or rebuild.

Thanks for any advice,

Ian.

[0] https://tracker.debian.org/pkg/qcontrol
[1] 
https://buildd.debian.org/status/fetch.php?pkg=qcontrol=armhf=0.5.5-1=1477849051
[2] https://wiki.debian.org/Hardening/PIEByDefaultTransition

Re: "PIE by default" transition is underway -- wiki needs updating

[Bálint Réczey]

Hi,

2016-10-27 4:03 GMT+02:00 Steve M. Robbins :
> On Wed, Oct 26, 2016 at 05:26:24AM +0200, Guillem Jover wrote:
>
>> My point was that, yes we have changed to generating relocatable code
>> but that is still targetted for executables only, which preserves the
>> current behavior, [...]
>
> But something must have changed with how a static lib is now compiled,
> because (a) I see bug reports saying "foo broke because static libbar
> is not -fPIC" and (b) the recommended fix is to rebuild libbar with
> the new toolchain -- with no source changes.
>
> So what's going on with static libs?

As a preparation for the transition I suggested using PIC for static
libraries for some packages and also suggested changing the Policy
to be more liberal about that.  (#837478)
I followed the current Policy and opened discussion on debian-devel
before enabling PIC in the affected packages:
https://lists.debian.org/debian-devel/2016/09/msg00277.html

I have updated some of the affected packages according the current
Policy.

A few weeks later Adrian shared his concerns with this approach
and preferred simple binNMU-s for affected static librarie because
it also fixes the build issues. The reasoning can be read in the same
bug.

Some binNMU-s already took place.
Adrian also filed bugs for reverting the changes enabling PIC for
static libraries.

This is where we are now. I guess the rest of the binNMUs will take
place soonish. The Policy may or may not be changed depending on
new inputs in #837478.

Cheers,
Balint

Re: "PIE by default" transition is underway -- wiki needs updating

[Steve M. Robbins]

On Wed, Oct 26, 2016 at 05:26:24AM +0200, Guillem Jover wrote:

> My point was that, yes we have changed to generating relocatable code
> but that is still targetted for executables only, which preserves the
> current behavior, [...]

But something must have changed with how a static lib is now compiled,
because (a) I see bug reports saying "foo broke because static libbar
is not -fPIC" and (b) the recommended fix is to rebuild libbar with
the new toolchain -- with no source changes.

So what's going on with static libs?

Thanks,
-Steve


signature.asc
Description: PGP signature

Re: "PIE by default" transition is underway -- wiki needs updating

[Andreas Cadhalpun]

Hi,

On 26.10.2016 05:37, Adam Borowski wrote:
> What's your reason for building something as big and with as extensive
> dependencies statically?  

Some parts of the test suite use private functions not exposed in the shared
libraries, so they need the static libraries.

> Let's delegate static libraries to special needs only, as limited as possible.

Yeah, I'll just not include the static libraries in the lib*-dev packages
anymore.

Best regards,
Andreas

Re: Bug#837478: "PIE by default" transition is underway -- wiki needs updating

[Andreas Cadhalpun]

Hi,

On 26.10.2016 05:26, Guillem Jover wrote:
> On Wed, 2016-10-26 at 00:37:18 +0200, Andreas Cadhalpun wrote:
>> On 25.10.2016 13:55, Guillem Jover wrote:
>>> For many static libraries,
>>> making them embeddable into other shared libraries is really not
>>> desirable. And those should be using the shared libraries instead.
>>
>> If that's the reason why it shouldn't be done, policy should mention it.
>> The current policy does not list this as reason not to use -fPIC, merely:
>> "since there is no benefit"
> 
> I don't think it's "the reason", but personally I think it's one
> important reason.

If there are more, they could be mentioned in policy, too.

> Embedding static libraries into shared libraries can be very
> problematic if the shared libraries do not take precautions, such as
> explicit symbol visibility or symbol versioning, etc. Which most
> shared libraries do not do. And even then it's still prone to symbol
> conflicts, etc, even inside the shared library being linked itself in
> case of namespace issues, if the static library is sloppy.

A (possibly shortened) version of this paragraph would be a good addition
to the policy. :)

> So I think this should be in general discouraged.
> 
>>> I still think the current policy is fine, and if someone wants to build
>>> a static library with PIC it should be brought up here.
>>
>> The current ffmpeg packages builds shared and static libraries, the
>> latter because they are used in the test suite. Both are built from
>> the same object files compiled with -fPIC.
>> Do you really think those static libraries should not be included
>> in the binary lib*-dev packages just because they are not incompatible
>> with including in other shared libraries?
> 
> Well, I guess depends on how "clean" they are, what's the intended
> usage, etc. But given that in this case the usage is inside the same
> project, that seems pretty safe!

There might be some confusion here. The ffmpeg package uses these
static libraries only for build-time testing, so doesn't need to
have them included in the binary packages.
I don't know if anyone is using these shipped static libraries.

> I'd personally probably not ship them, 
> and would instead provide non-PIC ones there. Or at most ship them in
> addition as _pic.a libraries, to require explicit invocation.

I'd rather not built everything twice, so I think I'll just drop the
static libraries in the next upload and only worry about this again,
when/if someone files a bug about missing the static libraries.

Best regards,
Andreas

Re: "PIE by default" transition is underway -- wiki needs updating

[Henrique de Moraes Holschuh]

On Wed, 26 Oct 2016, Adrian Bunk wrote:
> On Wed, Oct 26, 2016 at 05:37:06AM +0200, Adam Borowski wrote:
> > On Wed, Oct 26, 2016 at 12:37:18AM +0200, Andreas Cadhalpun wrote:
> > > The current policy says:
> > > "As to the static libraries, the common case is not to have relocatable 
> > > code"
> > > 
> > > As of gcc-6 version 6.2.0-7 this is factually wrong, because the compiler
> > > enables PIE by default, which means it produces relocatable code.
> > > This should definitely be updated to reflect reality.
> > 
> > It's not that simple.  The compiler enables PIE by default only:
> > * if you use "gcc" or "gcc-6".  gcc-5, clang, even gcc-snapshot still
> >   default to non-PIE.
> 
> The intention is that gcc-6 will be the only gcc version in stretch.

It needs to be able to compile the kernel correctly, first.  Which it
might not be able to right now (gcc 6.2.0), if one goes by code
generation issues reported in LKML (for other distros).  Hopefully it
will be fixed upstream by the time stretch is out, but it *is* a current
concern.

Also, the upstream kernel source needs to be fixed to deal with PIE
enabled by default (it is highly allergic to that).  This is already
ongoing AFAIK.

-- 
  Henrique Holschuh

Re: "PIE by default" transition is underway -- wiki needs updating

[Adrian Bunk]

On Wed, Oct 26, 2016 at 05:37:06AM +0200, Adam Borowski wrote:
> On Wed, Oct 26, 2016 at 12:37:18AM +0200, Andreas Cadhalpun wrote:
> > The current policy says:
> > "As to the static libraries, the common case is not to have relocatable 
> > code"
> > 
> > As of gcc-6 version 6.2.0-7 this is factually wrong, because the compiler
> > enables PIE by default, which means it produces relocatable code.
> > This should definitely be updated to reflect reality.
> 
> It's not that simple.  The compiler enables PIE by default only:
> * if you use "gcc" or "gcc-6".  gcc-5, clang, even gcc-snapshot still
>   default to non-PIE.

The intention is that gcc-6 will be the only gcc version in stretch.

> * on most first-class (release) architectures: amd64 arm64 armel armhf i386
>   mips mipsel mips64el ppc64el s390x.  This leaves out powerpc.
> * on none of second-class architectures: alpha hppa hurd-i386
>   kfreebsd-{amd64,i386} m68k powerpcspe ppc64 sh4 sparc64 x32
> * on none of unofficial architectures
> 
> Points 2-4 mean you need to bear with extra joy of supporting both variants
> on different archs.

As maintainer of an average package, it shouldn't make any difference 
for you.

In these special cases where it does, you might often already pass 
-fno-pie on all architectures now.

> So while indeed the current wording of the policy is wrong, the reality is
> currently... complicated.
>...

The "must not be compiled with the `-fPIC' flag" unless there is an 
exceptional case is still true.

So only a slight update in the wording is required regarding PIE.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: "PIE by default" transition is underway -- wiki needs updating

[Simon McVittie]

On Wed, 26 Oct 2016 at 05:37:06 +0200, Adam Borowski wrote:
> On Wed, Oct 26, 2016 at 12:37:18AM +0200, Andreas Cadhalpun wrote:
> > The current ffmpeg packages builds shared and static libraries
> 
> What's your reason for building something as big and with as extensive
> dependencies statically?  Let's delegate static libraries to special needs
> only, as limited as possible.

Are you suggesting that the normal way in which libraries are packaged in
Debian should change from "shared+static unless there is a good reason
to prefer shared-only or static-only" to "shared-only unless there is a
good reason to prefer shared+static or static-only"?

(In Autotools packages, this would mean adding --disable-static
to most configure invocations, and dropping the .a file from
debian/libwhatever.install.)

I don't think that's a bad idea, but it's certainly not the project's
conventional practice right now (e.g. Policy §8.3 says the static library
"is usually provided"); so it should probably be treated as a separate
change to the distribution, related to but distinct from the switch to
PIE-by-default.

S

Re: "PIE by default" transition is underway -- wiki needs updating

[Adam Borowski]

On Wed, Oct 26, 2016 at 12:37:18AM +0200, Andreas Cadhalpun wrote:
> The current policy says:
> "As to the static libraries, the common case is not to have relocatable code"
> 
> As of gcc-6 version 6.2.0-7 this is factually wrong, because the compiler
> enables PIE by default, which means it produces relocatable code.
> This should definitely be updated to reflect reality.

It's not that simple.  The compiler enables PIE by default only:
* if you use "gcc" or "gcc-6".  gcc-5, clang, even gcc-snapshot still
  default to non-PIE.
* on most first-class (release) architectures: amd64 arm64 armel armhf i386
  mips mipsel mips64el ppc64el s390x.  This leaves out powerpc.
* on none of second-class architectures: alpha hppa hurd-i386
  kfreebsd-{amd64,i386} m68k powerpcspe ppc64 sh4 sparc64 x32
* on none of unofficial architectures

Points 2-4 mean you need to bear with extra joy of supporting both variants
on different archs.  Point 1 means you might need to link libraries with
both into one binary.

So while indeed the current wording of the policy is wrong, the reality is
currently... complicated.


> > I still think the current policy is fine, and if someone wants to build
> > a static library with PIC it should be brought up here.
> 
> The current ffmpeg packages builds shared and static libraries, the
> latter because they are used in the test suite. Both are built from
> the same object files compiled with -fPIC.
> Do you really think those static libraries should not be included
> in the binary lib*-dev packages just because they are not incompatible
> with including in other shared libraries?

What's your reason for building something as big and with as extensive
dependencies statically?  Let's delegate static libraries to special needs
only, as limited as possible.


Meow!
-- 
A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol, 1kg
raspberries, 0.4kg sugar; put into a big jar for 1 month.  Filter out and
throw away the fruits (can dump them into a cake, etc), let the drink age
at least 3-6 months.

Re: "PIE by default" transition is underway -- wiki needs updating

[Guillem Jover]

Hi!

On Wed, 2016-10-26 at 00:37:18 +0200, Andreas Cadhalpun wrote:
> On 25.10.2016 13:55, Guillem Jover wrote:
> > I don't think the reasoning there is sound (as I've mentioned
> > elsewhere), and the policy bug should be closed.
> > 
> > Switching from no-PIE to PIE by default preserves our current behavior
> > WRT static libraries vs shared libraries.
> 
> The current policy says:
> "As to the static libraries, the common case is not to have relocatable code"
> 
> As of gcc-6 version 6.2.0-7 this is factually wrong, because the compiler
> enables PIE by default, which means it produces relocatable code.
> This should definitely be updated to reflect reality.

Right, that should be updated, but the bug was not about that. :)

My point was that, yes we have changed to generating relocatable code
but that is still targetted for executables only, which preserves the
current behavior, sorry if the wording was confusing.

> > For many static libraries,
> > making them embeddable into other shared libraries is really not
> > desirable. And those should be using the shared libraries instead.
> 
> If that's the reason why it shouldn't be done, policy should mention it.
> The current policy does not list this as reason not to use -fPIC, merely:
> "since there is no benefit"

I don't think it's "the reason", but personally I think it's one
important reason.

Embedding static libraries into shared libraries can be very
problematic if the shared libraries do not take precautions, such as
explicit symbol visibility or symbol versioning, etc. Which most
shared libraries do not do. And even then it's still prone to symbol
conflicts, etc, even inside the shared library being linked itself in
case of namespace issues, if the static library is sloppy.

So I think this should be in general discouraged.

> > I still think the current policy is fine, and if someone wants to build
> > a static library with PIC it should be brought up here.
> 
> The current ffmpeg packages builds shared and static libraries, the
> latter because they are used in the test suite. Both are built from
> the same object files compiled with -fPIC.
> Do you really think those static libraries should not be included
> in the binary lib*-dev packages just because they are not incompatible
> with including in other shared libraries?

Well, I guess depends on how "clean" they are, what's the intended
usage, etc. But given that in this case the usage is inside the same
project, that seems pretty safe! I'd personally probably not ship them,
and would instead provide non-PIC ones there. Or at most ship them in
addition as _pic.a libraries, to require explicit invocation.

Thanks,
Guillem

Re: "PIE by default" transition is underway -- wiki needs updating

[Andreas Cadhalpun]

Hi,

On 25.10.2016 13:55, Guillem Jover wrote:
> I don't think the reasoning there is sound (as I've mentioned
> elsewhere), and the policy bug should be closed.
> 
> Switching from no-PIE to PIE by default preserves our current behavior
> WRT static libraries vs shared libraries.

The current policy says:
"As to the static libraries, the common case is not to have relocatable code"

As of gcc-6 version 6.2.0-7 this is factually wrong, because the compiler
enables PIE by default, which means it produces relocatable code.
This should definitely be updated to reflect reality.

> For many static libraries,
> making them embeddable into other shared libraries is really not
> desirable. And those should be using the shared libraries instead.

If that's the reason why it shouldn't be done, policy should mention it.
The current policy does not list this as reason not to use -fPIC, merely:
"since there is no benefit"

> I still think the current policy is fine, and if someone wants to build
> a static library with PIC it should be brought up here.

The current ffmpeg packages builds shared and static libraries, the
latter because they are used in the test suite. Both are built from
the same object files compiled with -fPIC.
Do you really think those static libraries should not be included
in the binary lib*-dev packages just because they are not incompatible
with including in other shared libraries?

Best regards,
Andreas

Re: "PIE by default" transition is underway -- wiki needs updating

[Guillem Jover]

Hi!

On Tue, 2016-10-25 at 09:44:44 +0200, Bálint Réczey wrote:
> 2016-10-25 5:31 GMT+02:00 Steve M. Robbins :
> > I haven't been paying close attention to the "PIE by default" [1] 
> > discussions,
> > so I may have missed the memo, but: it seems the transition is underway?
> 
> GCC have been changed to enable PIE by default but dpkg has not been
> changed yet.

Yes, this will be included in the next dpkg upload which should happen
in a couple of days.

> > I've seen two bugs already claiming "static library foo must be compiled 
> > with
> > -fPIC" -- because some reverse dependency now fails to build.  But I think
> > this advice is misplaced.  The Ubuntu page [2] says that all you need to do 
> > is
> > rebuild the library foo with the PIE-enabled compiler, then rebuild the
> > depending code:
> >
> > Relocation Linking Failure
> >
> > A dynamically linked program that pulls in a static library that 
> > was not
> > built with -fPIC. These give an error like:
> >
> > relocation R_X86_64_32 against '[SYMBOL]' can not be used when 
> > making a
> > shared object; recompile with -fPIC
> >
> > To address these types of issues, the package providing the static 
> > object
> > needs to be rebuilt (usually just a no-change rebuild against the 
> > pie-by-
> > default compiler) before rebuilding the failed package.
> >
> >
> > So it seems to me that this should be emphasized on the wiki [1].  Secondly,
> 
> I filed the original bugs with the following template, which contains
> "Please", not
> "must": "Please build .a with -fPIC"
> It seems it was a mistake not emphasizing that a rebuild can also solve most 
> of
> the FTBFS bugs, and I have now updated the wiki, too.

I don't think the reasoning there is sound (as I've mentioned
elsewhere), and the policy bug should be closed.

Switching from no-PIE to PIE by default preserves our current behavior
WRT static libraries vs shared libraries. For many static libraries,
making them embeddable into other shared libraries is really not
desirable. And those should be using the shared libraries instead. I
still think the current policy is fine, and if someone wants to build
a static library with PIC it should be brought up here.

> > it seems that the proposal to change policy to encourage -fPIC on static
> > libraries [3] is misplaced and should be withdrawn.Are both these
> > statements accurate?
> 
> It have updated the wiki making it clear, that the Policy may not be
> changed.

I'd personally see no point in all those bug reports, TBH. :)

Thanks,
Guillem

Re: "PIE by default" transition is underway -- wiki needs updating

[Bálint Réczey]

Hi Steve,

2016-10-25 5:31 GMT+02:00 Steve M. Robbins :
> Hi,
>
> I haven't been paying close attention to the "PIE by default" [1] discussions,
> so I may have missed the memo, but: it seems the transition is underway?

GCC have been changed to enable PIE by default but dpkg has not been
changed yet.

>
> I've seen two bugs already claiming "static library foo must be compiled with
> -fPIC" -- because some reverse dependency now fails to build.  But I think
> this advice is misplaced.  The Ubuntu page [2] says that all you need to do is
> rebuild the library foo with the PIE-enabled compiler, then rebuild the
> depending code:
>
> Relocation Linking Failure
>
> A dynamically linked program that pulls in a static library that was 
> not
> built with -fPIC. These give an error like:
>
> relocation R_X86_64_32 against '[SYMBOL]' can not be used when 
> making a
> shared object; recompile with -fPIC
>
> To address these types of issues, the package providing the static 
> object
> needs to be rebuilt (usually just a no-change rebuild against the 
> pie-by-
> default compiler) before rebuilding the failed package.
>
>
> So it seems to me that this should be emphasized on the wiki [1].  Secondly,

I filed the original bugs with the following template, which contains
"Please", not
"must": "Please build .a with -fPIC"
It seems it was a mistake not emphasizing that a rebuild can also solve most of
the FTBFS bugs, and I have now updated the wiki, too.


> it seems that the proposal to change policy to encourage -fPIC on static
> libraries [3] is misplaced and should be withdrawn.Are both these
> statements accurate?

It have updated the wiki making it clear, that the Policy may not be changed.

Thanks,
Balint

>
> Thanks,
> -Steve
>
> [1] https://wiki.debian.org/Hardening/PIEByDefaultTransition
> [2] https://wiki.ubuntu.com/SecurityTeam/PIE
> [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=837478

"PIE by default" transition is underway -- wiki needs updating

[Steve M. Robbins]

Hi,

I haven't been paying close attention to the "PIE by default" [1] discussions, 
so I may have missed the memo, but: it seems the transition is underway?  

I've seen two bugs already claiming "static library foo must be compiled with 
-fPIC" -- because some reverse dependency now fails to build.  But I think 
this advice is misplaced.  The Ubuntu page [2] says that all you need to do is 
rebuild the library foo with the PIE-enabled compiler, then rebuild the 
depending code:

Relocation Linking Failure

A dynamically linked program that pulls in a static library that was 
not 
built with -fPIC. These give an error like: 

relocation R_X86_64_32 against '[SYMBOL]' can not be used when 
making a 
shared object; recompile with -fPIC

To address these types of issues, the package providing the static 
object 
needs to be rebuilt (usually just a no-change rebuild against the 
pie-by-
default compiler) before rebuilding the failed package. 


So it seems to me that this should be emphasized on the wiki [1].  Secondly, 
it seems that the proposal to change policy to encourage -fPIC on static 
libraries [3] is misplaced and should be withdrawn.Are both these 
statements accurate?

Thanks,
-Steve

[1] https://wiki.debian.org/Hardening/PIEByDefaultTransition
[2] https://wiki.ubuntu.com/SecurityTeam/PIE
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=837478


signature.asc
Description: This is a digitally signed message part.