Re: Introducing security hardening features for Lenny
On Wed, Mar 05, 2008 at 06:16:33AM +, Kees Cook wrote: Hi, I finally got some time to run some benchmarks. I checked the results[1] into the hardening svn tree, in case other people want to contribute more stuff. On Wed, Jan 30, 2008 at 08:46:55PM +0100, Moritz Muehlenhoff wrote: Video: mplayer with the -benchmark option in conjunction with -nosound and -vo. mplayer doesn't compile with PIE due to the various ASM routines. (I've noted this failure mode in the wiki[2] now.) However, with everything else enabled (including FORTIFY_SOURCE), there was no measurable difference (it was below the percentage difference between runs): runtime in seconds Mplayer Normal Hardened 110.87 10.807 210.873 10.824 310.854 10.963 410.809 10.84 510.877 10.838 avg 10.8566 10.8544 diff: -0.02% error 0.19% 1.00% HTML rendering: Mike Hommey once blogged about benchmarking the ACID test: http://web.glandium.org/blog/?cat=17 I followed that to: http://celtickane.com/webdesign/jsspeed2007.php The differences between runs were too high for me to use, so I skipped this for now. Nexuiz: | To run the benchmark: start Nexuiz open the console (`) issuing: | timedemo demos/demo1.dem The results will be stored in: | ~/.nexuiz/data/benchmark.log This one showed a possible difference: nexuiz Normal Hardened 1 66.68 68.113 2 66.802 66.93 3 66.758 67.03 4 66.728 67.051 5 66.859 67.037 avg 66.7654 67.2322 diff: 0.70% error0.14% 1.31% So, for nexuiz, with all hardening enabled in i386, there was a less-than-1-percent reduction in speed. Though the error margin for the hardened runs were still larger than the measured slow-down. Not sure about XML benchmarks. I did parse/render tests with inkscape on i386. Some of that is XML, but I figured it was heavy CPU, which might be worth something. Note that inkscape already compiles with all hardening options (excepting PIE), so the hardened time differences are entirely due to PIE. This one turned out similar to nexuiz, but with less error. Again, less than 1 percent slow-down was seen. inkscapeNormal Hardened 1 48.163 48.503 2 48.227 48.535 3 48.267 48.647 4 48.335 48.431 5 48.199 48.587 avg 48.2382 48.5406 diff: 0.63% error0.20% 0.22% I also ran inkscape and nexuiz tests on x86_64, and there was no measurable difference. I'm unclear if this was due to the extra registers, or just that that CPU was much faster and the difference vanished into the noise. Thank you very much for those. Though what did you built using -fPIE FORTIFY_SOURCES and so on ? only the tested applications ? or their build-deps as well ? Because I don't expect mplayer to be slowed a lot if you don't rebuild its ogg/mp3/mpg/... as well :) Same goes for inkscape. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpR2vBvGymLM.pgp Description: PGP signature
Re: Introducing security hardening features for Lenny
On Tue, 04 Mar 2008, Kees Cook wrote: mplayer doesn't compile with PIE due to the various ASM routines. (I've noted this failure mode in the wiki[2] now.) However, with everything else enabled (including FORTIFY_SOURCE), there was no measurable difference (it was below the percentage difference between runs): runtime in seconds Mplayer Normal Hardened 110.87 10.807 210.873 10.824 310.854 10.963 410.809 10.84 510.877 10.838 Just for future reference, it'd probably be better to run more than 5 tests of each population in the future, as 5 tests means you'll only detect very large differences in performance at any reasonable level of signifigance. FE: t.test(x=c(10.87,10.873,10.854,10.809,10.877),y=c(10.807,10.824,10.963,10.84,10.838)) Welch Two Sample t-test data: c(10.87, 10.873, 10.854, 10.809, 10.877) and c(10.807, 10.824, 10.963, 10.84, 10.838) t = 0.0722, df = 5.561, p-value = 0.945 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -0.07382831 0.07822831 sample estimates: mean of x mean of y 10.8566 10.8544 This one showed a possible difference: nexuiz Normal Hardened 1 66.68 68.113 2 66.802 66.93 3 66.758 67.03 4 66.728 67.051 5 66.859 67.037 While there may be a possible difference here, it's not significant, given the sample size: t.test(x=c(66.68,66.802,66.758,66.728,66.859),y=c(68.113,66.93,67.03,67.051,67.037)) Welch Two Sample t-test data: c(66.68, 66.802, 66.758, 66.728, 66.859) and c(68.113, 66.93, 67.03, 67.051, 67.037) t = -2.0899, df = 4.154, p-value = 0.1023 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -1.0779888 0.1443888 sample estimates: mean of x mean of y 66.7654 67.2322 But useful data nevertheless.[1] Don Armstrong 1: I won't even begin to discuss how many times I see benchmarks without SEM or sd reported. -- I'd sign up in a hot second for any cellular company whose motto was: We're less horrible than a root canal with a cold chisel. -- Cory Doctorow http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Wed, Mar 05, 2008 at 10:16:52AM +0100, Pierre Habouzit wrote: On Wed, Mar 05, 2008 at 06:16:33AM +, Kees Cook wrote: I finally got some time to run some benchmarks. I checked the results[1] into the hardening svn tree, in case other people want to contribute more stuff. Thank you very much for those. Though what did you built using -fPIE FORTIFY_SOURCES and so on ? only the tested applications ? or their build-deps as well ? Because I don't expect mplayer to be slowed a lot if you don't rebuild its ogg/mp3/mpg/... as well :) Same goes for inkscape. Well, libraries are already -fPIC so there's no need to recompile those. As for FORTIFY_SOURCE, that's true, I didn't rebuild the libraries with it for these tests. Getting all libs rebuilt may take a lot longer. :) -- Kees Cook@outflux.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Wed, Mar 05, 2008 at 01:29:01AM -0800, Don Armstrong wrote: Just for future reference, it'd probably be better to run more than 5 tests of each population in the future, as 5 tests means you'll only detect very large differences in performance at any reasonable level of signifigance. I suspected. :) I'm not much of a statistician, so I saved the raw data, hoping other folks would hop in to help out on this one. (Which you have!) Getting larger data sets will be rather time-consuming -- especially for nexuiz which I didn't figure out how to automate. t.test(x=c(10.87,10.873,10.854,10.809,10.877),y=c(10.807,10.824,10.963,10.84,10.838)) What tool is this you're using? Welch Two Sample t-test data: c(10.87, 10.873, 10.854, 10.809, 10.877) and c(10.807, 10.824, 10.963, 10.84, 10.838) t = 0.0722, df = 5.561, p-value = 0.945 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -0.07382831 0.07822831 sample estimates: mean of x mean of y 10.8566 10.8544 Which of these outputs should be paid attention to? But useful data nevertheless.[1] 1: I won't even begin to discuss how many times I see benchmarks without SEM or sd reported. Heh, well I know of the ideas, but haven't had any practice actually calculating them. Thanks! -Kees -- Kees Cook@outflux.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Wed, Mar 05, 2008 at 09:55:37AM -0800, Kees Cook wrote: t.test(x=c(10.87,10.873,10.854,10.809,10.877),y=c(10.807,10.824,10.963,10.84,10.838)) What tool is this you're using? GNU R. Takes a while to get into, but hard to beat for statistics. data: c(10.87, 10.873, 10.854, 10.809, 10.877) and c(10.807, 10.824, 10.963, 10.84, 10.838) t = 0.0722, df = 5.561, p-value = 0.945 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -0.07382831 0.07822831 sample estimates: mean of x mean of y 10.8566 10.8544 Which of these outputs should be paid attention to? The p-value is a good idea. If it's about 0.05, people tend to say the result is statistically significant. In your case it's 0.945, which means that the result is for all practical purposes worthless. /* Steinar */ -- Homepage: http://www.sesse.net/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
While these benchmarks should show any differences in raw processing performance, there's also the question of what differences the hardening measures make to application start-up times. PIE in particular should cause some slowdown when the executables are first run, but it would take some other sort of benchmark to determine the impact. PIE also makes prelinking ineffective, so it would probably be worth testing the difference between prelinked and PIE executables. On Wednesday 05 March 2008, Kees Cook wrote: I also ran inkscape and nexuiz tests on x86_64, and there was no measurable difference. I'm unclear if this was due to the extra registers, or just that that CPU was much faster and the difference vanished into the noise. The x86_64 arch has special hardware to better support PIE, so the lack of any noticeable difference in performance is likely due to that. signature.asc Description: This is a digitally signed message part.
Re: Introducing security hardening features for Lenny
On Wed, 05 Mar 2008, Kees Cook wrote: On Wed, Mar 05, 2008 at 01:29:01AM -0800, Don Armstrong wrote: Just for future reference, it'd probably be better to run more than 5 tests of each population in the future Getting larger data sets will be rather time-consuming -- especially for nexuiz which I didn't figure out how to automate. Yeah; the converse is that since you're really interested in major differences, 5 samples should be able to detect those.[1] t = 0.0722, df = 5.561, p-value = 0.945 95 percent confidence interval: -0.07382831 0.07822831 sample estimates: mean of x mean of y 10.8566 10.8544 Which of these outputs should be paid attention to? The p value and the convidence interval are the two things that are really useful. The first tells you that assuming the true difference between the means is zero, you would expect to see a difference like this or larger 94.5% of the time. The second tells you that with 95% confidence the true difference between the means is between -0.07 and 0.07. Don Armstrong 1: There are sample size calculators where given power (or 1-beta), alpha, and the difference you wish to detect will give you the number of samples required. -- When bad men combine, the good must associate; else they will fall one by one, an unpitied sacrifice in a contemptible struggle. -- Edmund Burke Thoughts on the Cause of Present Discoontents http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Wed, Mar 05, 2008 at 05:48:57PM +, Kees Cook wrote: On Wed, Mar 05, 2008 at 10:16:52AM +0100, Pierre Habouzit wrote: On Wed, Mar 05, 2008 at 06:16:33AM +, Kees Cook wrote: I finally got some time to run some benchmarks. I checked the results[1] into the hardening svn tree, in case other people want to contribute more stuff. Thank you very much for those. Though what did you built using -fPIE FORTIFY_SOURCES and so on ? only the tested applications ? or their build-deps as well ? Because I don't expect mplayer to be slowed a lot if you don't rebuild its ogg/mp3/mpg/... as well :) Same goes for inkscape. Well, libraries are already -fPIC so there's no need to recompile those. As for FORTIFY_SOURCE, that's true, I didn't rebuild the libraries with it for these tests. Getting all libs rebuilt may take a lot longer. :) Well then sadly it makes the benchmarks a lot less interesting :/ -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpG5sYdndlVB.pgp Description: PGP signature
Re: Introducing security hardening features for Lenny
Hi, I finally got some time to run some benchmarks. I checked the results[1] into the hardening svn tree, in case other people want to contribute more stuff. On Wed, Jan 30, 2008 at 08:46:55PM +0100, Moritz Muehlenhoff wrote: Video: mplayer with the -benchmark option in conjunction with -nosound and -vo. mplayer doesn't compile with PIE due to the various ASM routines. (I've noted this failure mode in the wiki[2] now.) However, with everything else enabled (including FORTIFY_SOURCE), there was no measurable difference (it was below the percentage difference between runs): runtime in seconds Mplayer Normal Hardened 110.87 10.807 210.873 10.824 310.854 10.963 410.809 10.84 510.877 10.838 avg 10.8566 10.8544 diff: -0.02% error 0.19% 1.00% HTML rendering: Mike Hommey once blogged about benchmarking the ACID test: http://web.glandium.org/blog/?cat=17 I followed that to: http://celtickane.com/webdesign/jsspeed2007.php The differences between runs were too high for me to use, so I skipped this for now. Nexuiz: | To run the benchmark: start Nexuiz open the console (`) issuing: | timedemo demos/demo1.dem The results will be stored in: | ~/.nexuiz/data/benchmark.log This one showed a possible difference: nexuiz Normal Hardened 1 66.68 68.113 2 66.802 66.93 3 66.758 67.03 4 66.728 67.051 5 66.859 67.037 avg 66.7654 67.2322 diff: 0.70% error0.14% 1.31% So, for nexuiz, with all hardening enabled in i386, there was a less-than-1-percent reduction in speed. Though the error margin for the hardened runs were still larger than the measured slow-down. Not sure about XML benchmarks. I did parse/render tests with inkscape on i386. Some of that is XML, but I figured it was heavy CPU, which might be worth something. Note that inkscape already compiles with all hardening options (excepting PIE), so the hardened time differences are entirely due to PIE. This one turned out similar to nexuiz, but with less error. Again, less than 1 percent slow-down was seen. inkscapeNormal Hardened 1 48.163 48.503 2 48.227 48.535 3 48.267 48.647 4 48.335 48.431 5 48.199 48.587 avg 48.2382 48.5406 diff: 0.63% error0.20% 0.22% I also ran inkscape and nexuiz tests on x86_64, and there was no measurable difference. I'm unclear if this was due to the extra registers, or just that that CPU was much faster and the difference vanished into the noise. -Kees [1] http://svn.debian.org/wsvn/hardening/benchmarks/ [2] http://wiki.debian.org/Hardening -- Kees Cook@outflux.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Sun, Feb 03, 2008 at 08:53:00PM +0100, Moritz Muehlenhoff wrote: Did you followup with upstream on the SSP problems we've seen on ARM? Basicly their response was basicly why would anyone want 5-10% bigger and slower binaries on arm. It was also discussed the possibility of --disable-ssp we use on our current arm/armel toolchain being broken. Once I have a bit more time I'll try seeing what happens if you build gcc-4.3 with ssp enabled. -- rm -rf only sounds scary if you don't have backups -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/04/08 05:45, Riku Voipio wrote: On Sun, Feb 03, 2008 at 08:53:00PM +0100, Moritz Muehlenhoff wrote: Did you followup with upstream on the SSP problems we've seen on ARM? Basicly their response was basicly why would anyone want 5-10% bigger and slower binaries on arm. It was also discussed Just MNSHO: Because ARM systems are almost always embedded, and don't get updated very often, so from the start should be as hardened as possible against attack. And if that means a 10% slowdown, so be it. the possibility of --disable-ssp we use on our current arm/armel toolchain being broken. Once I have a bit more time I'll try seeing what happens if you build gcc-4.3 with ssp enabled. - -- Ron Johnson, Jr. Jefferson LA USA PETA - People Eating Tasty Animals -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHpwaBS9HxQb37XmcRAqReAKDB0EjCnQ7eOA9/D2Ni0A4OpXyAcACdHXoz XUDrWQyvB8uFH4rwJEI27fY= =Iwd3 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
Riku Voipio wrote: In kernels that support text ASLR, programs compiled for PIE will gain full position randomization. For which architectures is text ASLR available? does it require external kernel patches? PIE means considerable system overhead and fatter binaries, especially for systems without large caches. I'm only aware of x86 and amd64. I don't think it's necessary on other archs. Did you followup with upstream on the SSP problems we've seen on ARM? Cheers, Moritz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
John Goerzen wrote: However, I am concerned that is appears to be limited in scope to packages that: * Are written in C or C++ * Can have hardening achieved through technical changes to the build process I think it is important to remember that other languages can have security problems too, perhaps just as easy as these (shell). Sure, but we're trying to optimise for the common case here. Everyone is welcome to start systematic security enhancement efforts for other languages (like, checking all existing Python code for insecure sub process invocation or something similar). An important reason is that some features (SSP and FORTIFIED_SOURCE) allow us to lower the amount of needed work to fix security issues. There have been several vulnerabilities which are non-issues on e.g. RHEL5, which has both enabled. The ASLR changes are icing on the cake. Cheers, Moritz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 11:47:57PM +, brian m. carlson wrote: In conclusion, there is no appreciable performance hit on any algorithm. Note that these are all hash algorithms, but they all make heavy use of memcpy, and are extremely CPU-intensive. OTOH the memcpy they use are statically checkable, so it's not a good test. the _chk versions of memcpy that are used are builtins in gcc that reduce to a simple memcpy if gcc was able to say statically if the bound was OK. Hash algorithms are not that good for checks for those features because their stack depth is usually thin, and boundaries easily checked. That's why I suggested testing encoding/decoding multimedia streams instead, or heavy text processors (xslt, docbook, ...) come to mind. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpXifsFLd6as.pgp Description: PGP signature
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 11:47:57PM +, brian m. carlson wrote: And now for some numbers, on an amd64 machine (/proc/cpuinfo at [0]), Oh and I missed that at the first read, but … With -fPIE -pie only: MD4: 335544320 bytes in 0.741s (431.703 MiB/s) MD5: 335544320 bytes in 1.052s (304.173 MiB/s) RMD160: 335544320 bytes in 1.755s (182.381 MiB/s) SHA1: 335544320 bytes in 1.797s (178.104 MiB/s) SHA256: 335544320 bytes in 3.517s (90.996 MiB/s) Of course it has no hit on amd64, it has all the relative operations needed to support PIE properly, and is not register starved. Benches have to be performed on x86 to have any valuable sense. With everything enabled, I expect on many applications on x86 to get on a regular basis 10% to 15% performances hit. Of course, on an architecture like amd64, I rather expect it to rarely be over a few percent. And as despise-able x86 is, it's still by a definite margin one of the most used architecture in Debian. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpFCh2xZEPRy.pgp Description: PGP signature
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 10:16:24PM +0100, Moritz Muehlenhoff wrote: In kernels that support text ASLR, programs compiled for PIE will gain full position randomization. For which architectures is text ASLR available? does it require external kernel patches? PIE means considerable system overhead and fatter binaries, especially for systems without large caches. -- rm -rf only sounds scary if you don't have backups -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 04:15:27PM -0800, Kees Cook wrote: On Tue, Jan 29, 2008 at 11:17:37PM +0100, sean finney wrote: In trying to not duplicate effort, I've been working both in Debian and Ubuntu to help get these options enabled globally. I have to repeat the question that tfheen asked on that list... why DEB_BUILD_HARDENING=1, and not DEB_BUILD_OPTS=hardening (thus the same as nostrip,noopt,etc). I'm all for making it as easy as possible to enable the flags. (Like I said in the other thread: patches welcome.) I'd probably want it to be nohardening, making compiles hardened by default. :) I also think it makes more sense to use DEB_BUILD_OPTIONS. OTOH, this might introduce some transition problems, when we move to opt-in for hardening to having a hardened toolchain by default and thus opt-out. On the other hand, maybe the set of packages is orthogonal, i.e. packages which might use hardening before the toolchain does by default is probably a different set to the packages which want to disable hardening after the move, due to some issues, not sure. Michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Wed, Jan 30, 2008 at 07:46:55PM +, Moritz Muehlenhoff wrote: Kees Cook wrote: Does anyone have any good test harnesses we can try this on? I'd be more than happy to run them on some modern hardware. Video: mplayer with the -benchmark option in conjunction with -nosound and -vo. HTML rendering: Mike Hommey once blogged about benchmarking the ACID test: http://web.glandium.org/blog/?cat=17 Nexuiz: | To run the benchmark: start Nexuiz open the console (`) issuing: | timedemo demos/demo1.dem The results will be stored in: | ~/.nexuiz/data/benchmark.log Not sure about XML benchmarks. About XML benchmarks you can use any big-enough xslt thingy. I'm sure people know docbook things that take long time to build, and that can give good numbers :) -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpotVmHsJtJH.pgp Description: PGP signature
Re: Introducing security hardening features for Lenny
On Wed, Jan 30, 2008 at 11:41:41AM +0200, Riku Voipio wrote: On Tue, Jan 29, 2008 at 10:16:24PM +0100, Moritz Muehlenhoff wrote: In kernels that support text ASLR, programs compiled for PIE will gain full position randomization. For which architectures is text ASLR available? does it require external kernel patches? PIE means considerable system overhead and fatter binaries, especially for systems without large caches. I can only speak for the Linux kernels, but sitting in the planned -x86 git queue for mm, text ASLR was enabled[1] for x86 and x86_64. AFAIK, the similar RedHat and SuSE kernel patches also carry these changes only for x86 and x86_64. -Kees [1] http://git.kernel.org/?p=linux/kernel/git/x86/linux-2.6-x86.git;a=history;f=arch/x86/kernel/sys_x86_64.c;hb=mm -- Kees Cook -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 09:48:43PM +, William Pitcock wrote: On Tue, 2008-01-29 at 22:37 +0100, Pierre Habouzit wrote: On Tue, Jan 29, 2008 at 09:16:24PM +, Moritz Muehlenhoff wrote: Fortify Source == This feature adds validation for internal C functions such as strcpy for buffer sizes known during compile time. While vulnerabilities in the functions it protects have become uncommon in high-profile apps, it will be useful for fringe packages we have in the archive. This feature is present in glibc since version 2.5, and is enabled through the use of -D_FORTIFY_SOURCE=2 and -O2 or higher. Well, -D_FORTIFY_SOURCE=2 is a severe performance loss in many applications, and I wouldn't recommend activating it by default. =1 has not the drawback with that regard though, but is less useful security wise (though it catch many programmatic issues, and full archive rebuild with -D_FORTIFY_SOURCE=1 would be worthwile independently of this). Out of curiosity, what applications in particular does -D_FORTIFY_SOURCE=2 cause issues in? It may be worthwhile to profile this feature and correct it's behaviour if the performance loss is that big of a deal. Basically any application that uses memcpy/memmove and some other common string.h functions heavily. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpAAJDLbEy4p.pgp Description: PGP signature
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 09:16:24PM +, Moritz Muehlenhoff wrote: Fortify Source == This feature adds validation for internal C functions such as strcpy for buffer sizes known during compile time. While vulnerabilities in the functions it protects have become uncommon in high-profile apps, it will be useful for fringe packages we have in the archive. This feature is present in glibc since version 2.5, and is enabled through the use of -D_FORTIFY_SOURCE=2 and -O2 or higher. Well, -D_FORTIFY_SOURCE=2 is a severe performance loss in many applications, and I wouldn't recommend activating it by default. =1 has not the drawback with that regard though, but is less useful security wise (though it catch many programmatic issues, and full archive rebuild with -D_FORTIFY_SOURCE=1 would be worthwile independently of this). pgp9nVbroP9o8.pgp Description: PGP signature
Re: Introducing security hardening features for Lenny
On Tue January 29 2008 3:16:24 pm Moritz Muehlenhoff wrote: Scope of this proposal == The target for Lenny is to enable these features in all applications with potential security impact, specifically: - Your application is written in C / C++ - If your package was subject to a DSA in the recent years - If your package parses files from untrusted sources - If your package communicates over a network I am very glad to see what you have been proposing so far. This is a great start. However, I am concerned that is appears to be limited in scope to packages that: * Are written in C or C++ * Can have hardening achieved through technical changes to the build process I think it is important to remember that other languages can have security problems too, perhaps just as easy as these (shell). Also there seems to be a bloat recently of the number of daemons running on the average Debian system. It seems to be just about impossible to have a desktop with sid without having avahi, dbus, hal, etc, etc, etc. running. How secure do we feel about all of this? I notice, for instance, that the latest cups requires avahi. Can we build it without that and install it without that by default for those that don't need it, to eliminate Yet Another Daemon? -- John -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Tue, 2008-01-29 at 22:37 +0100, Pierre Habouzit wrote: On Tue, Jan 29, 2008 at 09:16:24PM +, Moritz Muehlenhoff wrote: Fortify Source == This feature adds validation for internal C functions such as strcpy for buffer sizes known during compile time. While vulnerabilities in the functions it protects have become uncommon in high-profile apps, it will be useful for fringe packages we have in the archive. This feature is present in glibc since version 2.5, and is enabled through the use of -D_FORTIFY_SOURCE=2 and -O2 or higher. Well, -D_FORTIFY_SOURCE=2 is a severe performance loss in many applications, and I wouldn't recommend activating it by default. =1 has not the drawback with that regard though, but is less useful security wise (though it catch many programmatic issues, and full archive rebuild with -D_FORTIFY_SOURCE=1 would be worthwile independently of this). Out of curiosity, what applications in particular does -D_FORTIFY_SOURCE=2 cause issues in? It may be worthwhile to profile this feature and correct it's behaviour if the performance loss is that big of a deal. William signature.asc Description: This is a digitally signed message part
Re: Introducing security hardening features for Lenny
Pierre Habouzit wrote: Fortify Source =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 This feature adds validation for internal C functions such as strcpy for buffer sizes known during compile time. While vulnerabilities in the functions it protects have become uncommon in high-profile apps, it will be useful for fringe packages we have in the archive. =20 This feature is present in glibc since version 2.5, and is enabled through the use of -D_FORTIFY_SOURCE=3D2 and -O2 or higher. =20 Well, -D_FORTIFY_SOURCE=3D2 is a severe performance loss in many applications, and I wouldn't recommend activating it by default. =3D1 has not the drawback with that regard though, but is less useful security wise (though it catch many programmatic issues, and full archive rebuild with -D_FORTIFY_SOURCE=3D1 would be worthwile independently of this). There are certainly performance trade-offs involved and the final selection of features will depend on the testing of the respective maintainers (testing should be eased by hardening-wrapper). hardening-wrapper makes it simple to enable/disable selective single features, if anyone wants to run specific benchmarks on the overhead, please post them to the Wiki. We're mostly trying to bootstrap a discussion here, the details on how to put this into effect archive-wide will depend heavily on the toolchain configuration proposal by Matthias Klose. Maybe classes of security-sensitivity of applications can be defined, which specify a set of selected options. Cheers, Moritz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
hi moritz, On Tuesday 29 January 2008 10:16:24 pm Moritz Muehlenhoff wrote: A group of people have been working on introducing advanced security hardening features into our archive: http://alioth.debian.org/projects/hardening/ We recommend to activate the following features in individual packages for now and discuss how to enable them system-wide later. (Matthias Klose proposed a mechanism in debian-devel, which could be used for it: http://lists.debian.org/debian-devel/2007/12/msg00090.html). Some maintainers have already pro-actively enabled these features, e.g. in the sendmail and openssh packages, but we're heading for full archive coverage now. i guess you're aware of the discussions going on with ubuntu-devel as well? https://lists.ubuntu.com/archives/ubuntu-devel/2008-January/024958.html (and further posts where some implementation details are debated) I have to repeat the question that tfheen asked on that list... why DEB_BUILD_HARDENING=1, and not DEB_BUILD_OPTS=hardening (thus the same as nostrip,noopt,etc). otherwise, bravo for the effort! sean signature.asc Description: This is a digitally signed message part.
Re: Introducing security hardening features for Lenny
On Tue, 2008-01-29 at 23:31 +0100, Moritz Muehlenhoff wrote: Pierre Habouzit wrote: There are certainly performance trade-offs involved and the final selection of features will depend on the testing of the respective maintainers (testing should be eased by hardening-wrapper). What bothers me is that this kind of analysis should have preceded your announcement. I think that hardening is extremely important, but it is not the only important thing. It would be very helpful if your team would consider thinking about the tradeoffs, describing them so people can make some judgments. But that's not what you did: you instead posted a note, designed to sound as official as possible, asking every maintainer to add these flags. That's not right! We should instead discuss it. We're mostly trying to bootstrap a discussion here, the details on how to put this into effect archive-wide will depend heavily on the toolchain configuration proposal by Matthias Klose. Maybe classes of security-sensitivity of applications can be defined, which specify a set of selected options. For my money, you blew it. You don't bootstrap a discussion by presenting a pseudo-official email like the one you posted. But we can get back to that discussion: cancel the email by saying whoops, we're not ready yet and then having the discussion first. Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 10:31:48PM +, Moritz Muehlenhoff wrote: There are certainly performance trade-offs involved and the final selection of features will depend on the testing of the respective maintainers (testing should be eased by hardening-wrapper). I understand. To be fair, I'm worried in the implications of the SSP, FORTITY_SOURCES and PIE proposals. Others looks fine, but those three may have very important performance issues embedded. * SSP has a cost proportional to the number of calls an application performs (If I'm correct), which in CPU intensive tasks may become an issue. * FORTITY_SOURCES=2 checks memcpy and memmove, though other functions it checks should just not be used and applications beeing too slow because of them should just be shot down. * PIE is just IMHO not an option on x86 :/ Though probably someone should come up with some benchmarks. The usual culprits (multimedia libraries, html renderers, xml processors, …) all provide easily deployed bench, and before we go any further I'd like to see some numbers. If it's say less than a percent, okay I'm all for it. If we have more than 10% performance losses because of that, then we implicitely ask our users to sometimes buy faster machines (I know many people having installations where their multimedia player eats 80% CPU while decoding a film because they run it on old hardware, we may just kill this kind of use, and I would be sorry). -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpmkWRVZqdd3.pgp Description: PGP signature
Re: Introducing security hardening features for Lenny
On mar, 2008-01-29 at 15:47 -0600, John Goerzen wrote: I notice, for instance, that the latest cups requires avahi. Can we build it without that and install it without that by default for those that don't need it, to eliminate Yet Another Daemon? You do know that it depends on the lib, not the daemon? -- Yves-Alexis -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
Thomas Bushnell BSG wrote: For my money, you blew it. You don't bootstrap a discussion by presenting a pseudo-official email like the one you posted. But we can get back to that discussion: cancel the email by saying whoops, we're not ready yet and then having the discussion first. Of course we've discussed this in depth internally before before proposing it and there was no intention to make it sound official. There is no need to become aggressive. To resolve potential confusions I've sent a clarifying followup. Cheers, Moritz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Moritz Muehlenhoff wrote: The Debian archive is the biggest of all distributions and although there's security support for all security issues being found, there's still room for improvement and a need for increased resilience against flaws not yet discovered. A group of people have been working on introducing advanced security hardening features into our archive: http://alioth.debian.org/projects/hardening/ We recommend to activate the following features in individual packages for now and discuss how to enable them system-wide later. (Matthias Klose proposed a mechanism in debian-devel, which could be used for it: http://lists.debian.org/debian-devel/2007/12/msg00090.html). Concern was voiced that this proposal may sound like a call to maintainers to enable all these features directly. This was not our intention. At this point we mostly want to introduce the available solutions. Whether there will be an archive-wide global pre-selection of features and with which features will of course depend on the testing of the respective maintainers and discussions on debian-devel. Please participate in the discussion if you're intested in that matter. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHn7KTXm3vHE4uyloRAommAJwKyNlc4B/+Gkc8SsY4EuIoP3WzAACeN4XL Tych5TntCEobLJH+fKttpiI= =l7DO -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Wed, 2008-01-30 at 00:21 +0100, Moritz Muehlenhoff wrote: Thomas Bushnell BSG wrote: For my money, you blew it. You don't bootstrap a discussion by presenting a pseudo-official email like the one you posted. But we can get back to that discussion: cancel the email by saying whoops, we're not ready yet and then having the discussion first. Of course we've discussed this in depth internally before before proposing it and there was no intention to make it sound official. There is no need to become aggressive. I'm sorry for my tone. I know that it was discussed internally; but what I mean is that it needs to be discussed externally as the next step, long before it becomes the expected practice. If there were not important trade-offs, then it wouldn't matter, but the problem is that some of these options do impose significant costs. Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 11:45:32PM +0100, Pierre Habouzit wrote: On Tue, Jan 29, 2008 at 10:31:48PM +, Moritz Muehlenhoff wrote: There are certainly performance trade-offs involved and the final selection of features will depend on the testing of the respective maintainers (testing should be eased by hardening-wrapper). I understand. To be fair, I'm worried in the implications of the SSP, FORTITY_SOURCES and PIE proposals. Others looks fine, but those three may have very important performance issues embedded. * PIE is just IMHO not an option on x86 :/ If you use shared libraries, you already have this performance hit, and if you want to use %ebx in non-PIC code, you have to save and restore it around each shared-library function call. Also note that x86 has other problems (namely the 387) that make it unsuitable as a serious architecture. Though probably someone should come up with some benchmarks. The usual culprits (multimedia libraries, html renderers, xml processors, …) all provide easily deployed bench, and before we go any further I'd like to see some numbers. And now for some numbers, on an amd64 machine (/proc/cpuinfo at [0]), crypto code with heavy optimization in a shared library. All object files (including for the executable) are compiled with -fPIC, because I'm lazy. Performance without protection: MD4: 335544320 bytes in 0.745s (429.667 MiB/s) MD5: 335544320 bytes in 1.055s (303.452 MiB/s) RMD160: 335544320 bytes in 1.757s (182.169 MiB/s) SHA1: 335544320 bytes in 1.801s (177.693 MiB/s) SHA256: 335544320 bytes in 3.519s (90.928 MiB/s) With -fstack-protector only: MD4: 335544320 bytes in 0.748s (427.999 MiB/s) MD5: 335544320 bytes in 1.056s (302.979 MiB/s) RMD160: 335544320 bytes in 1.751s (182.744 MiB/s) SHA1: 335544320 bytes in 1.804s (177.348 MiB/s) SHA256: 335544320 bytes in 3.515s (91.028 MiB/s) With -D_FORTIFY_SOURCE=2 only: MD4: 335544320 bytes in 0.745s (429.670 MiB/s) MD5: 335544320 bytes in 1.053s (303.846 MiB/s) RMD160: 335544320 bytes in 1.756s (182.225 MiB/s) SHA1: 335544320 bytes in 1.794s (178.330 MiB/s) SHA256: 335544320 bytes in 3.514s (91.057 MiB/s) With -fPIE -pie only: MD4: 335544320 bytes in 0.741s (431.703 MiB/s) MD5: 335544320 bytes in 1.052s (304.173 MiB/s) RMD160: 335544320 bytes in 1.755s (182.381 MiB/s) SHA1: 335544320 bytes in 1.797s (178.104 MiB/s) SHA256: 335544320 bytes in 3.517s (90.996 MiB/s) With all three features: MD4: 335544320 bytes in 0.744s (429.978 MiB/s) MD5: 335544320 bytes in 1.058s (302.479 MiB/s) RMD160: 335544320 bytes in 1.758s (182.071 MiB/s) SHA1: 335544320 bytes in 1.793s (178.471 MiB/s) SHA256: 335544320 bytes in 3.520s (90.896 MiB/s) In conclusion, there is no appreciable performance hit on any algorithm. Note that these are all hash algorithms, but they all make heavy use of memcpy, and are extremely CPU-intensive. Code available upon request. [0] https://crustytoothpaste.ath.cx/machines/lakeview -- brian m. carlson / brian with sandals: Houston, Texas, US +1 713 440 7475 | http://crustytoothpaste.ath.cx/~bmc | My opinion only troff on top of XML: http://crustytoothpaste.ath.cx/~bmc/code/thwack OpenPGP: RSA v4 4096b 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187 signature.asc Description: Digital signature
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 11:45:32PM +0100, Pierre Habouzit wrote: * SSP has a cost proportional to the number of calls an application performs (If I'm correct), which in CPU intensive tasks may become an issue. In testing I've done, this is trivial overhead. Note that the extra code is only generated for functions with 8 byte of stack. (-fstack-protector-all is for doing _all_ functions, which doesn't make any sense.) * FORTITY_SOURCES=2 checks memcpy and memmove, though other functions it checks should just not be used and applications beeing too slow because of them should just be shot down. I still haven't found a comprehensive list of what -D_FORTIFY_SOURCE does, and at which levels various features are enabled. I've dug up various bits and pieces, but I'd love to see a pointer to good documentation. Without that, I suspect attempting to develop a reasonable test workload is bound to miss certain things. * PIE is just IMHO not an option on x86 :/ I disagree here -- the bulk of applications tend to use shared libraries, and those are all PIC. I have a hard time believing that adding the same relocation overhead for the main program is an issue. Of course, without numbers, we're all just waving our arms. :) Though probably someone should come up with some benchmarks. The usual culprits (multimedia libraries, html renderers, xml processors, …) all provide easily deployed bench, and before we go any further I'd like to see some numbers. Does anyone have any good test harnesses we can try this on? I'd be more than happy to run them on some modern hardware. -Kees -- Kees Cook -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Tue, Jan 29, 2008 at 11:17:37PM +0100, sean finney wrote: On Tuesday 29 January 2008 10:16:24 pm Moritz Muehlenhoff wrote: A group of people have been working on introducing advanced security hardening features into our archive: http://alioth.debian.org/projects/hardening/ i guess you're aware of the discussions going on with ubuntu-devel as well? https://lists.ubuntu.com/archives/ubuntu-devel/2008-January/024958.html (and further posts where some implementation details are debated) In trying to not duplicate effort, I've been working both in Debian and Ubuntu to help get these options enabled globally. I have to repeat the question that tfheen asked on that list... why DEB_BUILD_HARDENING=1, and not DEB_BUILD_OPTS=hardening (thus the same as nostrip,noopt,etc). I'm all for making it as easy as possible to enable the flags. (Like I said in the other thread: patches welcome.) I'd probably want it to be nohardening, making compiles hardened by default. :) -Kees -- Kees Cook -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On Tuesday 29 January 2008 4:31:51 pm Yves-Alexis Perez wrote: On mar, 2008-01-29 at 15:47 -0600, John Goerzen wrote: I notice, for instance, that the latest cups requires avahi. Can we build it without that and install it without that by default for those that don't need it, to eliminate Yet Another Daemon? You do know that it depends on the lib, not the daemon? -- Yves-Alexis It wound up pulling in the daemon on my box. Though it could be that the daemon was already installed because kde required it, and upgrading cups required the upgraded lib, and the daemon wouldn't work with the upgraded lib, so it too had to be upgraded... I didn't track that one all the way back. (I think it's broken that KDE requires it too) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Introducing security hardening features for Lenny
On mar, 2008-01-29 at 21:31 -0600, John Goerzen wrote: It wound up pulling in the daemon on my box. Though it could be that the daemon was already installed because kde required it, and upgrading cups required the upgraded lib, and the daemon wouldn't work with the upgraded lib, so it too had to be upgraded... I didn't track that one all the way back. Did you try to remove it? (I think it's broken that KDE requires it too) Think “Recommends”. -- Yves-Alexis signature.asc Description: This is a digitally signed message part
Re: Introducing security hardening features for Lenny
On Tuesday 29 January 2008, Moritz Muehlenhoff wrote: The Debian archive is the biggest of all distributions and although there's security support for all security issues being found, there's still room for improvement and a need for increased resilience against flaws not yet discovered. A group of people have been working on introducing advanced security hardening features into our archive: http://alioth.debian.org/projects/hardening/ We recommend to activate the following features in individual packages for now and discuss how to enable them system-wide later. (Matthias Klose proposed a mechanism in debian-devel, which could be used for it: http://lists.debian.org/debian-devel/2007/12/msg00090.html). Some maintainers have already pro-actively enabled these features, e.g. in the sendmail and openssh packages, but we're heading for full archive coverage now. There are two general classes of enhancements we'd like to apply to Debian: 1. Tool chain features preventing the exploitation of some vulnerability classes Stack protector === For a general introduction please see Wikipedia: http://en.wikipedia.org/wiki/Stack-smashing_protection This is relatively straight-forward. While it only addresses classic stack buffer overflows, we still have a lot of poorly-reviewed special case legacy code in our archive, so this will still be useful in practice. It's included in stock GCC since 4.1 onwards, so people would only need to add the compile flags to their packages. If there are packages which don't work with stack protection, it can be overridden with a compile flag. (We would need a lintian test to catch these, so that maintainers rather fix bugs in their packages than circumventing it with disabling SSP.) To enable, make sure that -fstack-protector ends up in the compiler flags. Fortify Source == This feature adds validation for internal C functions such as strcpy for buffer sizes known during compile time. While vulnerabilities in the functions it protects have become uncommon in high-profile apps, it will be useful for fringe packages we have in the archive. This feature is present in glibc since version 2.5, and is enabled through the use of -D_FORTIFY_SOURCE=2 and -O2 or higher. Format warnings === For a general introduction please see Wikipedia: http://en.wikipedia.org/wiki/Format_string_attack This feature adds a higher level of warning reporting for functions using format strings. To enable, add -Wformat and -Wformat-security flags, and pay attention to compile-time warnings. 2. Tool chain features enhancing the effectiveness of Address Space Layout Randomization, which raises the bar for successful exploitation of vulnerabilities. For a general introduction please see Wikipedia: http://en.wikipedia.org/wiki/Address_space_layout_randomization relro = This feature marks certain sections of the executable memory space read-only after the linker has finished its work. It serves as a measure against GOT overwrites, which can make exploits more difficult. This is enabled via -Wl,zrelro. Position Independent Executables Currently, modern kernels randomize the location of mmap and stack allocation, but the text segment (and subsequent brk memory) is always in the same place. In kernels that support text ASLR, programs compiled for PIE will gain full position randomization. This has some known problems on our more exotic archs, specifically hppa and m68k. These tool chains should be patched, so that enabling PIE is a NOP instead of forcing every maintainer to jump through hoops. The flag -fPIE is very similar to -fPIC, but it applies to objects linked to form the final executable binary. PIE is enabled by passing -fPIE to all object builds, and passing -pie to the final link. Experimental wrapper package An experimental wrapper has been written, which is available in unstable: http://packages.qa.debian.org/h/hardening-wrapper.html It contains basic usage information. You can use it to test compilation w/o much overhead. Lucas Nussbaum made a complete archive rebuild and about 700 packages failed to build, mostly due to problems with PIE: http://people.debian.org/~lucas/logs/2007/11/26.hardening/00lists/ Once you've verified that your package builds and runs correctly, you should add the necessary compiler/linker flags to your package's build system. Once a distribution-wide way to add these flags is defined, you can switch your package to it. Scope of this proposal == The target for Lenny is to enable these features in all applications with potential security impact, specifically: - Your application is written in C / C++ - If your package was subject to a DSA in the recent years - If your package parses files from untrusted sources - If your package
