libdwarf build failure with hardened builds
Does anybody have any clue what's going on here: https://kojipkgs.fedoraproject.org//work/tasks/8137/9208137/build.log It's an update to libdwarf, but the actual cause appears to be that it doesn't like the new hardened build options. It's a glibc symbol that it seems to be complaing about though: /usr/bin/ld: dwgetopt.o: relocation R_386_GOTOFF against undefined symbol `optopt@@GLIBC_2.0' can not be used when making a shared object The dwgetopt.o file appears to have been been correctly build with the hardening flags: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib -c ./dwgetopt.c Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: libdwarf build failure with hardened builds
On Thu, Mar 12, 2015 at 10:55 AM, Moez Roy moez@gmail.com wrote: On Thu, Mar 12, 2015 at 1:46 AM, Tom Hughes t...@compton.nu wrote: Does anybody have any clue what's going on here: https://kojipkgs.fedoraproject.org//work/tasks/8137/9208137/build.log It's an update to libdwarf, but the actual cause appears to be that it doesn't like the new hardened build options. It's a glibc symbol that it seems to be complaing about though: /usr/bin/ld: dwgetopt.o: relocation R_386_GOTOFF against undefined symbol `optopt@@GLIBC_2.0' can not be used when making a shared object The dwgetopt.o file appears to have been been correctly build with the hardening flags: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib -c ./dwgetopt.c Tom What is interesting about this is it built successfully for x86_64 but failed for i686. https://koji.fedoraproject.org/koji/taskinfo?taskID=9208115 I added '%global _hardened_build 1' to the spec file and it builds fine with the hardening flags for f21. https://koji.fedoraproject.org/koji/taskinfo?taskID=9211417 So it is a gcc5 regression. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: libdwarf build failure with hardened builds
On Thu, Mar 12, 2015 at 1:46 AM, Tom Hughes t...@compton.nu wrote: Does anybody have any clue what's going on here: https://kojipkgs.fedoraproject.org//work/tasks/8137/9208137/build.log It's an update to libdwarf, but the actual cause appears to be that it doesn't like the new hardened build options. It's a glibc symbol that it seems to be complaing about though: /usr/bin/ld: dwgetopt.o: relocation R_386_GOTOFF against undefined symbol `optopt@@GLIBC_2.0' can not be used when making a shared object The dwgetopt.o file appears to have been been correctly build with the hardening flags: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -I. -I. -I./../libdwarf -DCONFPREFIX=/usr/local/lib -c ./dwgetopt.c Tom What is interesting about this is it built successfully for x86_64 but failed for i686. https://koji.fedoraproject.org/koji/taskinfo?taskID=9208115 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
On Fri, Mar 6, 2015 at 2:01 PM, Jerry James loganje...@gmail.com wrote: Oops, sorry, got distracted. It is polymake. That package has multiple problems. First, it invokes undefined behavior in one bit of code. That happened to work out with gcc 4.x, but gcc 5.x compiles the code a bit differently, resulting in a scrambled data structure that causes a crash sometime later. Tracing that down was sure a pain, but I finally found it, and will definitely tell upstream to fix their code. Can you post a link here for that gcc regression bug? Thanks. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
On Sat, Mar 7, 2015 at 12:52 PM, Moez Roy moez@gmail.com wrote: Can you post a link here for that gcc regression bug? Thanks. There is no regression. The source code invoked undefined behavior. Under such circumstances, the compiler is free to do anything at all. Things just happened to work out right with gcc 4.x, but the fact that it doesn't work out right with gcc 5.x is not gcc's fault. -- Jerry James http://www.jamezone.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
On 02/24/2015 06:44 PM, Jerry James wrote: How is this really supposed to be done? If I'm doing it the right way, then the current method is broken. %undefine _hardened_build -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
On Thu, Feb 26, 2015 at 11:00 AM, Rex Dieter rdie...@math.unl.edu wrote: which package is this again? I can try experimenting a bit. The one that worked for me was lightdm, fwiw. Oops, sorry, got distracted. It is polymake. That package has multiple problems. First, it invokes undefined behavior in one bit of code. That happened to work out with gcc 4.x, but gcc 5.x compiles the code a bit differently, resulting in a scrambled data structure that causes a crash sometime later. Tracing that down was sure a pain, but I finally found it, and will definitely tell upstream to fix their code. Second, polymake queries perl::Config for the flags perl was built with, and perl was built with the hardening flags. No wonder I couldn't get rid of them. Third, polymake has a number of plugins (apps in polymake parlance), some of which depend on each other, but they are not linked to one another. The main application knows about the dependencies between them and dlopens them in the correct order to avoid unresolved symbols. However, it also has a documentation building step, which dlopens them one by one to extract documentation strings. This is fine, because no code in the plugins is actually executed. At least, it used to be fine back when RTLD_LAZY actually worked. Now it causes the documentation building step to fail due to unresolved symbols. The question is whether I can fix this by altering polymake alone, or whether a non-hardened perl is needed to make this work. I'm still experimenting to discover the answer to that question. (Polymake builds take awhile and gobble memory like it was doughnuts, so this isn't a fast process.) -- Jerry James http://www.jamezone.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
On Saturday, February 28, 2015, Till Maas opensou...@till.name wrote: I am going to get this updated once the mass rebuild is done and I got the details worked out about what to do best, for example for allowing exceptions from this. You gonna allow many exceptions as some packages on x86_64 failed to build. -- Yours sincerely, Christopher Meng http://cicku.me -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
On Tue, Feb 24, 2015 at 05:44:18PM -0700, Jerry James wrote: Also, http://fedoraproject.org/wiki/Hardened_Packages seems to be entirely useless at this point. Perhaps it could be replaced with a page that discusses the current state of the hardening flags. I am going to get this updated once the mass rebuild is done and I got the details worked out about what to do best, for example for allowing exceptions from this. Regards Till -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
Jerry James wrote: On Wed, Feb 25, 2015 at 9:53 AM, Rex Dieter rdie...@math.unl.edu wrote: Mamoru TASAKA wrote: Does %undefine _hardened_build help? that version works for me Hmmm, am I doing something wrong? I've tried both: %undefine _hardened_build and: %undefine _hardened_build %global _hardened_build 0 at the top of the spec file, and in both cases, I still see the hardened specs in CFLAGS and LDFLAGS when %configure is invoked. I've got the right spec file inside the mock root. I don't understand why this worked for you and isn't working for me. which package is this again? I can try experimenting a bit. The one that worked for me was lightdm, fwiw. -- Rex -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
RE: Hardened builds
Mamoru TASAKA wrote: Hello: I've got a package that is currently failing to build in Rawhide. It has a home-brewed garbage collector inside, and it appears the GC is confused about which objects are on the stack, and which are on the heap. I want to try building without the hardening flags to see if that has something to do with the problem. According to https://fedorahosted.org/fesco/ticket/1384 the way to do this is to put: %global _hardened_build 0 at the top of the spec file. But that doesn't work with a local mock build. I still see -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 in the compiler flags and -specs=/usr/lib/rpm/redhat/redhat-hardened-ld in the linker flags. Does %undefine _hardened_build help? that version works for me -- Rex -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
On Wed, Feb 25, 2015 at 9:53 AM, Rex Dieter rdie...@math.unl.edu wrote: Mamoru TASAKA wrote: Does %undefine _hardened_build help? that version works for me Hmmm, am I doing something wrong? I've tried both: %undefine _hardened_build and: %undefine _hardened_build %global _hardened_build 0 at the top of the spec file, and in both cases, I still see the hardened specs in CFLAGS and LDFLAGS when %configure is invoked. I've got the right spec file inside the mock root. I don't understand why this worked for you and isn't working for me. -- Jerry James http://www.jamezone.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
RE: Hardened builds
Hello: I've got a package that is currently failing to build in Rawhide. It has a home-brewed garbage collector inside, and it appears the GC is confused about which objects are on the stack, and which are on the heap. I want to try building without the hardening flags to see if that has something to do with the problem. According to https://fedorahosted.org/fesco/ticket/1384 the way to do this is to put: %global _hardened_build 0 at the top of the spec file. But that doesn't work with a local mock build. I still see -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 in the compiler flags and -specs=/usr/lib/rpm/redhat/redhat-hardened-ld in the linker flags. Does %undefine _hardened_build help? Regards, Mamoru -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Hardened builds
I've got a package that is currently failing to build in Rawhide. It
has a home-brewed garbage collector inside, and it appears the GC is
confused about which objects are on the stack, and which are on the
heap. I want to try building without the hardening flags to see if
that has something to do with the problem. According to
https://fedorahosted.org/fesco/ticket/1384 the way to do this is to
put:
%global _hardened_build 0
at the top of the spec file. But that doesn't work with a local mock
build. I still see -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 in
the compiler flags and -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
in the linker flags. So I tried adding the lines:
%global _hardened_cflags %{nil}
%global _hardened_ldflags %{nil}
just below the _hardened_build line. Now mock explodes:
Traceback (most recent call last):
File /usr/sbin/mock, line 829, in module
main()
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/sbin/mock, line 650, in main
run_command(options, args, config_opts, commands, buildroot, state)
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/sbin/mock, line 725, in run_command
do_rebuild(config_opts, commands, buildroot, args)
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/sbin/mock, line 496, in do_rebuild
post=post_build, clean=clean)
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/sbin/mock, line 440, in rebuild_generic
commands.init(prebuild=not config_opts.get('short_circuit'))
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/lib/python2.7/site-packages/mockbuild/backend.py, line
122, in init
self.buildroot.initialize(**kwargs)
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/lib/python2.7/site-packages/mockbuild/buildroot.py, line
80, in initialize
self._init(prebuild=prebuild, do_log=do_log)
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/lib/python2.7/site-packages/mockbuild/buildroot.py, line
117, in _init
self.plugins.call_hooks('preinit')
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/lib/python2.7/site-packages/mockbuild/plugin.py, line
65, in call_hooks
hook(*args, **kwargs)
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/lib/python2.7/site-packages/mockbuild/plugins/ccache.py,
line 60, in _ccachePreInitHook
self.buildroot.uid_manager.changeOwner(self.ccachePath, recursive=True)
File /usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py,
line 84, in trace
result = func(*args, **kw)
File /usr/lib/python2.7/site-packages/mockbuild/uid.py, line 84,
in changeOwner
os.chown(os.path.join(root, f), uid, gid)
OSError: [Errno 2] No such file or directory:
'/var/cache/mock/fedora-rawhide-x86_64/ccache/u1000/9/stats.lock'
How is this really supposed to be done? If I'm doing it the right
way, then the current method is broken.
Also, http://fedoraproject.org/wiki/Hardened_Packages seems to be
entirely useless at this point. Perhaps it could be replaced with a
page that discusses the current state of the hardening flags.
Thank you,
--
Jerry James
http://www.jamezone.org/
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Hardened builds
On Tue, Feb 24, 2015 at 6:26 PM, Mamoru TASAKA
mtas...@fedoraproject.org wrote:
Does
%undefine _hardened_build
help?
Thanks for the suggestion, but regrettably, now %configure does this:
checking C++ compiler ... ok (g++ is GCC 5.0)
determining architecture ... ok (x86_64)
determining compiler flags ... ok
CFLAGS=%{optflags} -I%{_includedir}/eigen3 -Wno-unused-local-typedefs -Wall
CXXFLAGS=%{optflags} -I%{_includedir}/eigen3
-Wno-unused-local-typedefs -ftemplate-depth-200 -Wall
-Wno-strict-aliasing -Wno-parentheses -fwrapv -fopenmp
checking gmp installation ... Could not compile a test program
checking for the GNU Multiprecision Library (GMP).
The complete error log follows:
g++: error: %{optflags}: No such file or directory
I'm going to sleep on it and see what I can come up with tomorrow.
The mock explosion, by the way, appears to be unrelated. I think what
happened is that I did one mock run with -r fedora-rawhide-x86_64,
then wanted to do another without blowing away the first, so I added a
--uniqueext argument. Then later, I did a mock --clean
--uniqueext=, and after that the next mock --rebuild ...
--uniqueext=... threw that error. I will try to reproduce later and
file a bug report.
Regards,
--
Jerry James
http://www.jamezone.org/
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Autodetecting insufficiently hardened builds (was: New hardened build support (coming) in F16)
Matthew Garrett wrote: Unless the checking is part of autoqa this simply isn't sufficient. There's a huge benefit to implementing it in the way that's easiest for maintainers. The earlier a problem is detected, the cheaper it is to fix. If I have understood AutoQA right, it gets involved only after I submit a package to updates-testing. Currently we (the AutoQA) are able to run the test either after you submit it to Bodhi, or even right after the build is finished in Koji (which is probably better for your use case). Feel free to discuss details with us in autoqa-devel [1]. [1] https://fedorahosted.org/mailman/listinfo/autoqa-devel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Autodetecting insufficiently hardened builds
The earlier a problem is detected, the cheaper it is to fix. If I have understood AutoQA right, it gets involved only after I submit a package to updates-testing. Kamil Paral wrote: Currently we (the AutoQA) are able to run the test either after you submit it to Bodhi, or even right after the build is finished in Koji (which is probably better for your use case). Thanks for the correction. Yes that's better, but better still is to also run the test from RPMbuild on my local workstation, for example when I do fedpkg local or fedpkg mockbuild. Björn Persson signature.asc Description: This is a digitally signed message part. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Autodetecting insufficiently hardened builds (was: New hardened build support (coming) in F16)
Matthew Garrett wrote: Unless the checking is part of autoqa this simply isn't sufficient. There's a huge benefit to implementing it in the way that's easiest for maintainers. The earlier a problem is detected, the cheaper it is to fix. If I have understood AutoQA right, it gets involved only after I submit a package to updates-testing. Even better, when possible, is to detect problems during the build, so that I notice them when I test-build locally and can fix them before I even commit the changes to Git. Some checks are already done at build time, and Steve's points 1, 2 and 4 look like they should be possible to detect at build time too. Björn Persson signature.asc Description: This is a digitally signed message part. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Autodetecting insufficiently hardened builds (was: New hardened build support (coming) in F16)
On Tue, Aug 09, 2011 at 04:53:16PM +0200, Björn Persson wrote: Matthew Garrett wrote: Unless the checking is part of autoqa this simply isn't sufficient. There's a huge benefit to implementing it in the way that's easiest for maintainers. The earlier a problem is detected, the cheaper it is to fix. If I have understood AutoQA right, it gets involved only after I submit a package to updates-testing. Even better, when possible, is to detect problems during the build, so that I notice them when I test-build locally and can fix them before I even commit the changes to Git. Some checks are already done at build time, and Steve's points 1, 2 and 4 look like they should be possible to detect at build time too. Checking at build time is a developer aid, but doesn't prevent mistakes from slipping into the distribution. Having both would certainly be helpful. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
