Re: Harden_all_packages_with_position-independent_code + guile modules
On Wed, 2015-03-18 at 11:37 -0700, Moez Roy wrote: FULL RELRO http://tk-blog.blogspot.co.at/2009/02/relro-not-so-well-known-memory.html If that's all we got I suggest to remove this flag or (better) provide a way for applications that use modules to compile themselves, without removing the whole set of hardening flags. Any advise from the change owners? How should applications that use modules with undefined systems should handle that? Should they add % undefine _hardened_build by default? I was doing some research last night but not tested it yet: nonow 1) add -nonow to the CFLAGS 2) or add -z nonow to the LDFLAGS doing the koji builds now to test and see if it works. Also need to test if there is a -lazy option. Why are you using -Wl,--no-add-needed in the LD flags? I don't see the reason for it. Added Tomas (the previous maintainer) in case he remembers. I am able to get much further ahead in the build process when I remove this. Do you mean the package was fully built by removing the no-add-needed flag? Previously the make check process was failing. regards, Nikos -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 19.3.2015 08:16, Nikos Mavrogiannopoulos wrote: On Wed, 2015-03-18 at 11:37 -0700, Moez Roy wrote: FULL RELRO http://tk-blog.blogspot.co.at/2009/02/relro-not-so-well-known-memory.html If that's all we got I suggest to remove this flag or (better) provide a way for applications that use modules to compile themselves, without removing the whole set of hardening flags. Any advise from the change owners? How should applications that use modules with undefined systems should handle that? Should they add % undefine _hardened_build by default? I was doing some research last night but not tested it yet: nonow 1) add -nonow to the CFLAGS 2) or add -z nonow to the LDFLAGS doing the koji builds now to test and see if it works. Also need to test if there is a -lazy option. Why are you using -Wl,--no-add-needed in the LD flags? I don't see the reason for it. Added Tomas (the previous maintainer) in case he remembers. If I remember correctly it was added to get rid of some unneeded dependency that was otherwise pulled in. But it might be unnecessary now. You could try to build the rpm (without hardening) with and without the -Wl,--no-add-needed and see whether there are some additional Requires added to the resulting rpms. Regards, Tomas -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
Here are 2 solutions: - Solution 1: Keeps -z now (Full RELRO) but removes guile: I changed the first 2 lines in the spec file to: %global without dane %global without guile and the latest version builds successfully: https://koji.fedoraproject.org/koji/taskinfo?taskID=9276305 -- Solution 2: Keep the guile bindings at the cost of disabling -z now: I added the following under %build in the spec file: CFLAGS=$RPM_OPT_FLAGS -Wl,-z,lazy CXXFLAGS=$RPM_OPT_FLAGS -Wl,-z,lazy export CFLAGS export CXXFLAGS and the latest version builds successfully: https://koji.fedoraproject.org/koji/taskinfo?taskID=9276929 -- Regards, Moez Roy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Thu, 2015-03-12 at 10:41 -0400, Adam Jackson wrote: We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. I've reopened the ticket and added a patch: https://fedorahosted.org/fesco/ticket/1384#comment:13 Basically this changes things so executables are always PIE and -z now, and that _hardened_build is back to 0 by default. Changing it to 1 just adds -z now to shared libs as well. As a result, most packages that set it to 0 in response to the F23 change can probably go back to not overriding it. One other change I might like to sneak in there is linking PIEs with -z nodlopen by default. It's a thing that _can_ work, but it really requires that the executable be designed with it in mind which most aren't. It'd be nice not to expose ABIs we don't mean to. - ajax -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Mon, 2015-03-16 at 10:57 +0100, Nikos Mavrogiannopoulos wrote: Am 16.03.2015 um 09:47 schrieb Nikos Mavrogiannopoulos: What was the rationale of adding -z now to the hardening flags? Looking its description doesn't reveal any hardening features, and the gnutls guile module failure to build seems to be directly related to that flag: https://bugzilla.redhat.com/show_bug.cgi?id=1196556 FULL RELRO http://tk-blog.blogspot.co.at/2009/02/relro-not-so-well-known-memory.html If that's all we got I suggest to remove this flag or (better) provide a way for applications that use modules to compile themselves, without removing the whole set of hardening flags. Any advise from the change owners? How should applications that use modules with undefined systems should handle that? Should they add % undefine _hardened_build by default? -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Wed, Mar 18, 2015 at 6:54 AM, Nikos Mavrogiannopoulos n...@redhat.com wrote: On Mon, 2015-03-16 at 10:57 +0100, Nikos Mavrogiannopoulos wrote: Am 16.03.2015 um 09:47 schrieb Nikos Mavrogiannopoulos: What was the rationale of adding -z now to the hardening flags? Looking its description doesn't reveal any hardening features, and the gnutls guile module failure to build seems to be directly related to that flag: https://bugzilla.redhat.com/show_bug.cgi?id=1196556 FULL RELRO http://tk-blog.blogspot.co.at/2009/02/relro-not-so-well-known-memory.html If that's all we got I suggest to remove this flag or (better) provide a way for applications that use modules to compile themselves, without removing the whole set of hardening flags. Any advise from the change owners? How should applications that use modules with undefined systems should handle that? Should they add % undefine _hardened_build by default? I was doing some research last night but not tested it yet: nonow 1) add -nonow to the CFLAGS 2) or add -z nonow to the LDFLAGS doing the koji builds now to test and see if it works. Also need to test if there is a -lazy option. -Moez -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Wed, Mar 18, 2015 at 7:21 AM, Moez Roy moez@gmail.com wrote: On Wed, Mar 18, 2015 at 6:54 AM, Nikos Mavrogiannopoulos n...@redhat.com wrote: On Mon, 2015-03-16 at 10:57 +0100, Nikos Mavrogiannopoulos wrote: Am 16.03.2015 um 09:47 schrieb Nikos Mavrogiannopoulos: What was the rationale of adding -z now to the hardening flags? Looking its description doesn't reveal any hardening features, and the gnutls guile module failure to build seems to be directly related to that flag: https://bugzilla.redhat.com/show_bug.cgi?id=1196556 FULL RELRO http://tk-blog.blogspot.co.at/2009/02/relro-not-so-well-known-memory.html If that's all we got I suggest to remove this flag or (better) provide a way for applications that use modules to compile themselves, without removing the whole set of hardening flags. Any advise from the change owners? How should applications that use modules with undefined systems should handle that? Should they add % undefine _hardened_build by default? I was doing some research last night but not tested it yet: nonow 1) add -nonow to the CFLAGS 2) or add -z nonow to the LDFLAGS doing the koji builds now to test and see if it works. Also need to test if there is a -lazy option. Why are you using -Wl,--no-add-needed in the LD flags? I am able to get much further ahead in the build process when I remove this. I was not successful with -Wl,-z -Wl,nonow Kept getting /usr/bin/ld: warning: -z nonow ignored. Maybe there is no option as -z nonow. Or maybe -z now takes precedence based on the RPM flags. Adding '%global _hardened_build 1' to the spec file and setting the target to F21 caused it to fail: https://koji.fedoraproject.org/koji/taskinfo?taskID=9264983 If I used the F20 source and set target for F23 it succeeded with the default hardening flags: https://koji.fedoraproject.org/koji/taskinfo?taskID=9265633 From the changelog I see F20 does not have guile bindings. * Mon Jan 05 2015 Nikos Mavrogiannopoulos n...@redhat.com 3.3.11-2 - enabled guile bindings (#1177847) So maybe you should considering reverting the above change, and sticking with the default hardening flags? Regards, Moez -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
Am 18.03.2015 um 19:37 schrieb Moez Roy: I was not successful with -Wl,-z -Wl,nonow FULL RELO: -Wl,-z,now YOU: -Wl,-z -Wl,nonow TRY: -Wl,-z,nonow what you did was change semantics for unknown reasons signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 03/18/2015 02:54 PM, Nikos Mavrogiannopoulos wrote: Any advise from the change owners? How should applications that use modules with undefined systems should handle that? What semantics do you expect from the undefined symbol? -- Florian Weimer / Red Hat Product Security -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 03/18/2015 03:21 PM, Moez Roy wrote:
I was doing some research last night but not tested it yet:
nonow
That's not a thing. The opposite of “-z now” is “-z lazy”:
else if (strcmp (optarg, now) == 0)
{
link_info.flags |= (bfd_vma) DF_BIND_NOW;
link_info.flags_1 |= (bfd_vma) DF_1_NOW;
}
else if (strcmp (optarg, lazy) == 0)
{
link_info.flags = ~(bfd_vma) DF_BIND_NOW;
link_info.flags_1 = ~(bfd_vma) DF_1_NOW;
}
--
Florian Weimer / Red Hat Product Security
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Mon, 2015-03-16 at 18:31 +0100, Jakub Jelinek wrote: On Mon, Mar 16, 2015 at 01:24:19PM -0400, Adam Jackson wrote: On Fri, 2015-03-13 at 12:14 +0100, Florian Weimer wrote: PIE does alter symbol resolution, though not in a particularly big way. In a normal executable, taking the address of a global function takes the address of the definition found in the executable itself, if any; in a PIE, you take the address of the first definition found by the runtime linker. Normally that's not a correctness issue since the executable usually ends up as the first object searched, but it does mean an LD_PRELOAD can override more symbols than it used to because the (now-PIE) executable is importing more symbols than before. ??? PIE is the first object in symbol search scope, before LD_PRELOAD, identically to normal executables. Yeah, sorry, I was reading readelf's output wrong. Those symbols get emitted as relocations (where they wouldn't for an executable), but they resolve to the providing object. My error, I apologize. - ajax -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Mon, 2015-03-16 at 10:19 +0100, Reindl Harald wrote: Am 16.03.2015 um 09:47 schrieb Nikos Mavrogiannopoulos: What was the rationale of adding -z now to the hardening flags? Looking its description doesn't reveal any hardening features, and the gnutls guile module failure to build seems to be directly related to that flag: https://bugzilla.redhat.com/show_bug.cgi?id=1196556 FULL RELRO http://tk-blog.blogspot.co.at/2009/02/relro-not-so-well-known-memory.html If that's all we got I suggest to remove this flag or (better) provide a way for applications that use modules to compile themselves, without removing the whole set of hardening flags. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Mon, Mar 16, 2015 at 01:24:19PM -0400, Adam Jackson wrote: On Fri, 2015-03-13 at 12:14 +0100, Florian Weimer wrote: On 03/12/2015 03:41 PM, Adam Jackson wrote: We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. People tell conflicting things about PIE. I have asked essentially the same thing, and I was told, no, PIE itself alters symbol resolution. Is this true or not? PIE does alter symbol resolution, though not in a particularly big way. In a normal executable, taking the address of a global function takes the address of the definition found in the executable itself, if any; in a PIE, you take the address of the first definition found by the runtime linker. Normally that's not a correctness issue since the executable usually ends up as the first object searched, but it does mean an LD_PRELOAD can override more symbols than it used to because the (now-PIE) executable is importing more symbols than before. ??? PIE is the first object in symbol search scope, before LD_PRELOAD, identically to normal executables. Jakub -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Fri, 2015-03-13 at 12:14 +0100, Florian Weimer wrote: On 03/12/2015 03:41 PM, Adam Jackson wrote: We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. People tell conflicting things about PIE. I have asked essentially the same thing, and I was told, no, PIE itself alters symbol resolution. Is this true or not? PIE does alter symbol resolution, though not in a particularly big way. In a normal executable, taking the address of a global function takes the address of the definition found in the executable itself, if any; in a PIE, you take the address of the first definition found by the runtime linker. Normally that's not a correctness issue since the executable usually ends up as the first object searched, but it does mean an LD_PRELOAD can override more symbols than it used to because the (now-PIE) executable is importing more symbols than before. - ajax -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Thu, 2015-03-12 at 10:41 -0400, Adam Jackson wrote: On Thu, 2015-03-12 at 13:45 +, Petr Pisar wrote: On 2015-03-12, Nikos Mavrogiannopoulos n...@redhat.com wrote: In rawhide building the gnutls guile bindings fails, and that's related to the new hardening flags being enabled with [0]. The failure is quite peculiar since the loading of a dynamic module fails [1] which already is position independent. [...] [1]. https://bugzilla.redhat.com/show_bug.cgi?id=1196556 The test-suite.log reads file not found which is far from loading DSO failed. However I can add my recent story: After hardening perl, loading a DSO by perl failed. I believe the reason was the DSO had an undefined symbol which was not defined in any SO_NEEDed libraries. But because the symbol was never used at run-time, before hardening the executable, run-time linking passed. But after hardening, the -znow feature caused resolving all symbols at link time, including the missing symbol, so dlopen(3) failed. We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. What was the rationale of adding -z now to the hardening flags? Looking its description doesn't reveal any hardening features, and the gnutls guile module failure to build seems to be directly related to that flag: https://bugzilla.redhat.com/show_bug.cgi?id=1196556 regards, Nikos -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
Am 16.03.2015 um 09:47 schrieb Nikos Mavrogiannopoulos: What was the rationale of adding -z now to the hardening flags? Looking its description doesn't reveal any hardening features, and the gnutls guile module failure to build seems to be directly related to that flag: https://bugzilla.redhat.com/show_bug.cgi?id=1196556 FULL RELRO http://tk-blog.blogspot.co.at/2009/02/relro-not-so-well-known-memory.html signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Fri, 13 Mar 2015 19:24:58 +0100 Ralf Corsepius rc040...@freenet.de wrote: My conclusion is that gcc5 is at least sometimes using a lot more memory than older versions. I haven't noticed a significant increase in memory usage in my local mock builds, but I haven't tried to investigate. Yeah, me either, was just a conclusion based on the OOM. Perhaps reducing smp_mflags could help? Dunno ;) If you set it always to 4 do a number of scratch builds in a row work? buildhw boxes have 20GB of memory and 16CPUs... so perhaps 16 threads is causing it to hit OOM much more often than in the past? buildvm boxes have 10GB of memory, but only 4 (faster) cpus. arm boxes have 4GB of memory and 4 cpus. Does this explain why my yesterday's scratch-builds didn't fail but real builds failed? If it's memory related and the builders with 4 cpus always work, but the ones on the builders with 16 cpus always fail it would. Does this explain why real builds failed yesterday but succeed today? Did you change something about this configuration or has something related to g++ changed within the last 24hrs? ... too many unknowns for my taste. Such is life. ;) kevin pgpaYxnv2DKoP.pgp Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 03/13/2015 01:58 PM, Kevin Fenzi wrote: On Fri, 13 Mar 2015 07:46:04 +0100 Ralf Corsepius rc040...@freenet.de wrote: On 03/12/2015 05:06 PM, Ralf Corsepius wrote: Well, all I can say, building rawhide packages (esp. C++) seems to have evolved into a lottery and rendered working one Fedora packages into an adventure. E.g. I am facing packages which one day build, but fail to build the other day. Today, I have been facing a package which builds in local rawhide mocks, but fails in official koji. Let me provide some facts, as they happened to me, yesterday (2015-03-12): Trying to build k3d[1] for rawhide - ca. 10:00 UTC: local x86_64-mockbuild succeeds - 10:15 UTC: Building in koji koji fails on x86_64: c++: internal compiler error: Killed (program cc1plus) https://koji.fedoraproject.org/koji/taskinfo?taskID=9208659 builder: buildhw-06 Mar 12 10:22:17 buildhw-06.phx2.fedoraproject.org kernel: Out of memory: Kill process 10227 (cc1plus) score 29 or sacrifice child Mar 12 10:22:17 buildhw-06.phx2.fedoraproject.org kernel: Killed process 10227 (cc1plus) total-vm:635536kB, anon-rss:578384kB, file-rss:17300kB Mar 12 10:22:17 buildhw-06.phx2.fedoraproject.org kernel: cc1plus: page allocation failure: order:0, mode:0x280da - 10:36 UTC: Presuming a temporary hickup, retrying koji koji fails on x86_64: c++: internal compiler error: Killed (program cc1plus) https://koji.fedoraproject.org/koji/taskinfo?taskID=9208790 builder: buildhw-06 again Mar 12 10:43:46 buildhw-06.phx2.fedoraproject.org kernel: Out of memory: Kill process 13650 (cc1plus) score 29 or sacrifice child Mar 12 10:43:46 buildhw-06.phx2.fedoraproject.org kernel: Killed process 13650 (cc1plus) total-vm:647752kB, anon-rss:594536kB, file-rss:12628kB Mar 12 10:43:57 buildhw-06.phx2.fedoraproject.org kernel: as invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 ...snip... ?!? My conclusion is that gcc5 is at least sometimes using a lot more memory than older versions. I haven't noticed a significant increase in memory usage in my local mock builds, but I haven't tried to investigate. Perhaps reducing smp_mflags could help? Dunno ;) buildhw boxes have 20GB of memory and 16CPUs... so perhaps 16 threads is causing it to hit OOM much more often than in the past? buildvm boxes have 10GB of memory, but only 4 (faster) cpus. arm boxes have 4GB of memory and 4 cpus. Does this explain why my yesterday's scratch-builds didn't fail but real builds failed? Does this explain why real builds failed yesterday but succeed today? Did you change something about this configuration or has something related to g++ changed within the last 24hrs? ... too many unknowns for my taste. Ralf -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 03/12/2015 05:06 PM, Ralf Corsepius wrote: Well, all I can say, building rawhide packages (esp. C++) seems to have evolved into a lottery and rendered working one Fedora packages into an adventure. E.g. I am facing packages which one day build, but fail to build the other day. Today, I have been facing a package which builds in local rawhide mocks, but fails in official koji. Let me provide some facts, as they happened to me, yesterday (2015-03-12): Trying to build k3d[1] for rawhide - ca. 10:00 UTC: local x86_64-mockbuild succeeds - 10:15 UTC: Building in koji koji fails on x86_64: c++: internal compiler error: Killed (program cc1plus) https://koji.fedoraproject.org/koji/taskinfo?taskID=9208659 - 10:36 UTC: Presuming a temporary hickup, retrying koji koji fails on x86_64: c++: internal compiler error: Killed (program cc1plus) https://koji.fedoraproject.org/koji/taskinfo?taskID=9208790 - 14:29 UTC: Retrying koji, again: koji fails on x86_64: c++: internal compiler error: Killed (program cc1plus) https://koji.fedoraproject.org/koji/taskinfo?taskID=9210278 - 16:04 UTC: Trying x86_64 koji scratch-build koji succeeds (on x86_64): http://koji.fedoraproject.org/koji/taskinfo?taskID=9210901 - 16:59 UTC: Trying full koji scratch-build koji succeeds on all arches: http://koji.fedoraproject.org/koji/taskinfo?taskID=9211124 - 20:07 UTC: Retrying full koji-build koji fails on i386: c++: internal compiler error: Killed (program cc1plus) https://koji.fedoraproject.org/koji/taskinfo?taskID=9212121 - 2015-03-13 04:05 UTC: Retrying full koji-build: koji succeeded. http://koji.fedoraproject.org/koji/taskinfo?taskID=9217030 ?!? Ralf [1] k3d: A fairly complex c++-application, with a long and complex dep-chain involving many c/c++ packages, comprising boost, gtkmm24, ImageMagic++ and libstdc++. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Thu, 2015-03-12 at 18:49 +0100, Till Maas wrote: On Thu, Mar 12, 2015 at 10:41:49AM -0400, Adam Jackson wrote: We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. Yes, if it is causing problems in a majority of packages, I agree that it would be better to make this opt-in again. I think it is important to document what a hardened build means, in the change request as well as provide a pointer from the packaging guidelines. It's no much point mentioning hardened builds but no-one can find out what are these flags and the rationale of being added to this set. Currently it was implied they were flags to enable position independent code, but as it seems there are other flags in this set too. regards, Nikos -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Fri, Mar 13, 2015 at 12:34 AM, Nikos Mavrogiannopoulos
n...@redhat.com wrote:
I think it is important to document what a hardened build means, in the
change request as well as provide a pointer from the packaging
guidelines. It's no much point mentioning hardened builds but no-one can
find out what are these flags and the rationale of being added to this
set. Currently it was implied they were flags to enable position
independent code, but as it seems there are other flags in this set too.
regards,
Nikos
https://pkgs.fedoraproject.org/cgit/redhat-rpm-config.git/plain/redhat-hardened-cc1
*cc1_options:
+ %{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}
--
https://pkgs.fedoraproject.org/cgit/redhat-rpm-config.git/plain/redhat-hardened-ld
*self_spec:
+ %{!shared:-pie}
*link:
+ -z now
I added the above information to the wiki as requested:
https://fedoraproject.org/wiki/Changes/Harden_all_packages_with_position-independent_code#Detailed_Harden_Flags_Description
Regards,
Moez
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 03/13/2015 10:14 AM, Moez Roy wrote: I added the above information to the wiki as requested: https://fedoraproject.org/wiki/Changes/Harden_all_packages_with_position-independent_code#Detailed_Harden_Flags_Description I think people who can read GCC specs file syntax will not have to consult this page, so I tried to explain what the specs files do in some detail. -- Florian Weimer / Red Hat Product Security -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 03/12/2015 03:41 PM, Adam Jackson wrote: We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. People tell conflicting things about PIE. I have asked essentially the same thing, and I was told, no, PIE itself alters symbol resolution. Is this true or not? -- Florian Weimer / Red Hat Product Security -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Fri, 13 Mar 2015 07:46:04 +0100 Ralf Corsepius rc040...@freenet.de wrote: On 03/12/2015 05:06 PM, Ralf Corsepius wrote: Well, all I can say, building rawhide packages (esp. C++) seems to have evolved into a lottery and rendered working one Fedora packages into an adventure. E.g. I am facing packages which one day build, but fail to build the other day. Today, I have been facing a package which builds in local rawhide mocks, but fails in official koji. Let me provide some facts, as they happened to me, yesterday (2015-03-12): Trying to build k3d[1] for rawhide - ca. 10:00 UTC: local x86_64-mockbuild succeeds - 10:15 UTC: Building in koji koji fails on x86_64: c++: internal compiler error: Killed (program cc1plus) https://koji.fedoraproject.org/koji/taskinfo?taskID=9208659 builder: buildhw-06 Mar 12 10:22:17 buildhw-06.phx2.fedoraproject.org kernel: Out of memory: Kill process 10227 (cc1plus) score 29 or sacrifice child Mar 12 10:22:17 buildhw-06.phx2.fedoraproject.org kernel: Killed process 10227 (cc1plus) total-vm:635536kB, anon-rss:578384kB, file-rss:17300kB Mar 12 10:22:17 buildhw-06.phx2.fedoraproject.org kernel: cc1plus: page allocation failure: order:0, mode:0x280da - 10:36 UTC: Presuming a temporary hickup, retrying koji koji fails on x86_64: c++: internal compiler error: Killed (program cc1plus) https://koji.fedoraproject.org/koji/taskinfo?taskID=9208790 builder: buildhw-06 again Mar 12 10:43:46 buildhw-06.phx2.fedoraproject.org kernel: Out of memory: Kill process 13650 (cc1plus) score 29 or sacrifice child Mar 12 10:43:46 buildhw-06.phx2.fedoraproject.org kernel: Killed process 13650 (cc1plus) total-vm:647752kB, anon-rss:594536kB, file-rss:12628kB Mar 12 10:43:57 buildhw-06.phx2.fedoraproject.org kernel: as invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 ...snip... ?!? My conclusion is that gcc5 is at least sometimes using a lot more memory than older versions. Perhaps reducing smp_mflags could help? buildhw boxes have 20GB of memory and 16CPUs... so perhaps 16 threads is causing it to hit OOM much more often than in the past? buildvm boxes have 10GB of memory, but only 4 (faster) cpus. arm boxes have 4GB of memory and 4 cpus. kevin pgpEQFfiVwEJm.pgp Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Harden_all_packages_with_position-independent_code + guile modules
In rawhide building the gnutls guile bindings fails, and that's related to the new hardening flags being enabled with [0]. The failure is quite peculiar since the loading of a dynamic module fails [1] which already is position independent. Could someone explain what do the new flags in rawhide do (and possibly add this description in [0]). Using the same hardened flags in F21 has no negative effect in building the module. regards, Nikos [0]. https://fedoraproject.org/wiki/Changes/Harden_all_packages_with_position-independent_code [1]. https://bugzilla.redhat.com/show_bug.cgi?id=1196556 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 12/03/15 14:41, Adam Jackson wrote: On Thu, 2015-03-12 at 13:45 +, Petr Pisar wrote: However I can add my recent story: After hardening perl, loading a DSO by perl failed. I believe the reason was the DSO had an undefined symbol which was not defined in any SO_NEEDed libraries. But because the symbol was never used at run-time, before hardening the executable, run-time linking passed. But after hardening, the -znow feature caused resolving all symbols at link time, including the missing symbol, so dlopen(3) failed. We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. On top of which there seems, despite a number of questions posted here since the change went live, very little assistance from the proposal owners with fixing packages that have been broken by it. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 2015-03-12, Nikos Mavrogiannopoulos n...@redhat.com wrote: In rawhide building the gnutls guile bindings fails, and that's related to the new hardening flags being enabled with [0]. The failure is quite peculiar since the loading of a dynamic module fails [1] which already is position independent. [...] [1]. https://bugzilla.redhat.com/show_bug.cgi?id=1196556 The test-suite.log reads file not found which is far from loading DSO failed. However I can add my recent story: After hardening perl, loading a DSO by perl failed. I believe the reason was the DSO had an undefined symbol which was not defined in any SO_NEEDed libraries. But because the symbol was never used at run-time, before hardening the executable, run-time linking passed. But after hardening, the -znow feature caused resolving all symbols at link time, including the missing symbol, so dlopen(3) failed. -- Petr -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Thu, 2015-03-12 at 13:45 +, Petr Pisar wrote: On 2015-03-12, Nikos Mavrogiannopoulos n...@redhat.com wrote: In rawhide building the gnutls guile bindings fails, and that's related to the new hardening flags being enabled with [0]. The failure is quite peculiar since the loading of a dynamic module fails [1] which already is position independent. [...] [1]. https://bugzilla.redhat.com/show_bug.cgi?id=1196556 The test-suite.log reads file not found which is far from loading DSO failed. However I can add my recent story: After hardening perl, loading a DSO by perl failed. I believe the reason was the DSO had an undefined symbol which was not defined in any SO_NEEDed libraries. But because the symbol was never used at run-time, before hardening the executable, run-time linking passed. But after hardening, the -znow feature caused resolving all symbols at link time, including the missing symbol, so dlopen(3) failed. We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. - ajax -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 03/12/2015 04:39 PM, Moez Roy wrote: On Thu, Mar 12, 2015 at 7:49 AM, Tom Hughes t...@compton.nu wrote: On 12/03/15 14:41, Adam Jackson wrote: We may want to revisit this, honestly. +1 I believe most of the issues should get resolved on its own after the mass rebuild of all the packages. Rel-Eng will do the mass rebuild as soon as they get the green light from Jakub. Well, all I can say, building rawhide packages (esp. C++) seems to have evolved into a lottery and rendered working one Fedora packages into an adventure. E.g. I am facing packages which one day build, but fail to build the other day. Today, I have been facing a package which builds in local rawhide mocks, but fails in official koji. To be honest, this is very non-satisfactory! I tried suggesting 2 mass rebuilds; 1 mass rebuild of all packages right now, and another mass rebuild of all the packages 1 month later when gcc5 becomes more stable, but they said no as it costs too much resources. Not sure - IMO, we need to minimize the causes of breakdowns, because at the moment too many issues seem to be interacting at the same time ( GCC-5.0, harding, usual package upgrade churn). A non hardened mass rebuild, would have been an intermediate step which would have been helpful, but ... it's too late, IMO. Ralf -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Thu, Mar 12, 2015 at 7:49 AM, Tom Hughes t...@compton.nu wrote: On 12/03/15 14:41, Adam Jackson wrote: On Thu, 2015-03-12 at 13:45 +, Petr Pisar wrote: However I can add my recent story: After hardening perl, loading a DSO by perl failed. I believe the reason was the DSO had an undefined symbol which was not defined in any SO_NEEDed libraries. But because the symbol was never used at run-time, before hardening the executable, run-time linking passed. But after hardening, the -znow feature caused resolving all symbols at link time, including the missing symbol, so dlopen(3) failed. We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. On top of which there seems, despite a number of questions posted here since the change went live, very little assistance from the proposal owners with fixing packages that have been broken by it. Tom I believe most of the issues should get resolved on its own after the mass rebuild of all the packages. Rel-Eng will do the mass rebuild as soon as they get the green light from Jakub. I tried suggesting 2 mass rebuilds; 1 mass rebuild of all packages right now, and another mass rebuild of all the packages 1 month later when gcc5 becomes more stable, but they said no as it costs too much resources. -Moez -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Thu, Mar 12, 2015 at 05:26:43PM +, Tom Hughes wrote: Well I haven't figured out how to fix libdwarf (see message this morning) nor do I have any idea what I should do next to try and figure it out. So unless I ignore the rawhide first rule it seems I can't update that for F22 until the mass rebuild which might apparently fix it in rawhide. Alternatively I can disable hardened builds, but I assumed that needed permission from somebody... Sorry that this was not clear. You can go ahead and disable the check for now. According to my memory it is in general permitted to adjust the build flags if there are important reasons for this. It would be great if you can try to include the PIE build flags if only the -z now build flag is causing problems. Btw. why are you setting the CFLAGS manually before running %configure? %configure should already take care of them. Regards Till -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Thu, Mar 12, 2015 at 10:41:49AM -0400, Adam Jackson wrote: We may want to revisit this, honestly. The actual proposal was just to build executables as PIE, right? Forcing -z now is a bit more than maybe was expected. Yes, if it is causing problems in a majority of packages, I agree that it would be better to make this opt-in again. Regards Till -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 12/03/15 17:14, Till Maas wrote: On Thu, Mar 12, 2015 at 02:49:03PM +, Tom Hughes wrote: On top of which there seems, despite a number of questions posted here since the change went live, very little assistance from the proposal owners with fixing packages that have been broken by it. I do not see much value in repeating responses if others have already figured something out. So to workaround build failures because of it for individual packages is to disable the flags for the respective package. This allows to build a package for now so the actual problem can be analysed without disturbing updates for the package. Well I haven't figured out how to fix libdwarf (see message this morning) nor do I have any idea what I should do next to try and figure it out. So unless I ignore the rawhide first rule it seems I can't update that for F22 until the mass rebuild which might apparently fix it in rawhide. Alternatively I can disable hardened builds, but I assumed that needed permission from somebody... Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On Thu, Mar 12, 2015 at 02:49:03PM +, Tom Hughes wrote: On top of which there seems, despite a number of questions posted here since the change went live, very little assistance from the proposal owners with fixing packages that have been broken by it. I do not see much value in repeating responses if others have already figured something out. So to workaround build failures because of it for individual packages is to disable the flags for the respective package. This allows to build a package for now so the actual problem can be analysed without disturbing updates for the package. Regards Till -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 12/03/15 17:48, Till Maas wrote: On Thu, Mar 12, 2015 at 05:26:43PM +, Tom Hughes wrote: Well I haven't figured out how to fix libdwarf (see message this morning) nor do I have any idea what I should do next to try and figure it out. So unless I ignore the rawhide first rule it seems I can't update that for F22 until the mass rebuild which might apparently fix it in rawhide. Alternatively I can disable hardened builds, but I assumed that needed permission from somebody... Sorry that this was not clear. You can go ahead and disable the check for now. According to my memory it is in general permitted to adjust the build flags if there are important reasons for this. It would be great if you can try to include the PIE build flags if only the -z now build flag is causing problems. Btw. why are you setting the CFLAGS manually before running %configure? %configure should already take care of them. I think the CFLAGS things was inherited from the previous owner of the package. I've removed it now, and disabled the hardened build for now. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Harden_all_packages_with_position-independent_code + guile modules
On 03/12/2015 06:14 PM, Till Maas wrote: On Thu, Mar 12, 2015 at 02:49:03PM +, Tom Hughes wrote: On top of which there seems, despite a number of questions posted here since the change went live, very little assistance from the proposal owners with fixing packages that have been broken by it. The feature owner should have performed a mass rebuilt prior to pushing this feature into the wild. I don't recall such thing to have happened. I do not see much value in repeating responses if others have already figured something out. So to workaround build failures because of it for individual packages is to disable the flags for the respective package. For this to be applicable, you have to know that hardening is the actual cause - Provided the shape rawhide currently is in, this often does not apply. Ralf -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
