Re: anti-ROP mechanism in libc
On Wed, May 04, 2016 at 08:28:41PM -0400, Ian Sutton wrote: > This gives me an idea for how to solve the lack of a first-stage > bootloader (like biosboot(8)) on armv7. Currently U-Boot loads the > kernel image directly into memory and jmp's to its entry point without > an intermediary stage to read /etc/random.seed from disk and provide it > to the kernel to kickstart the random subsystem. I tried to solve this > problem by writing a standalone sd/mmc driver that would operate similar > to biosboot(8) in that it would first be loaded and run by U-Boot, then > read /etc/random.seed and the kernel image from the disk, load the > entropy-supplied kernel into memory and execute it. I found that each > armv7 machine's sd/mmc stack is implemented differently enough to > require a standalone driver for every vendor's armv7 toy despite SD & > MMC supposedly being "standard" interfaces. Trying to keep up with the > ever-expanding list of armv7 machines seems like a futile and inelegant > solution. > > Perhaps the entropy normally written to /etc/random.seed upon shutdown > can now be written directly to the kernel image itself, precluding the > need for such standalone drivers. I believe the reason this wasn't done > in the past was to allow checksum verification of the kernel image > between boots. With this new patch, libc.so would change between boots > precluding the possibility of such checksum verification, a contingency > I believe was properly realized and accounted for. If we can stand to > forgo the checksum verification of a big target like libc.so, perhaps we > can forgo it with the kernel image itself too and solve the boot-time > entropy problem. > > Ian > In u-boot 2016.05 it is possible to enable EFI entry points. So the amd64 efiboot code could be adapted to read blocks that way.
Re: anti-ROP mechanism in libc
This gives me an idea for how to solve the lack of a first-stage bootloader (like biosboot(8)) on armv7. Currently U-Boot loads the kernel image directly into memory and jmp's to its entry point without an intermediary stage to read /etc/random.seed from disk and provide it to the kernel to kickstart the random subsystem. I tried to solve this problem by writing a standalone sd/mmc driver that would operate similar to biosboot(8) in that it would first be loaded and run by U-Boot, then read /etc/random.seed and the kernel image from the disk, load the entropy-supplied kernel into memory and execute it. I found that each armv7 machine's sd/mmc stack is implemented differently enough to require a standalone driver for every vendor's armv7 toy despite SD & MMC supposedly being "standard" interfaces. Trying to keep up with the ever-expanding list of armv7 machines seems like a futile and inelegant solution. Perhaps the entropy normally written to /etc/random.seed upon shutdown can now be written directly to the kernel image itself, precluding the need for such standalone drivers. I believe the reason this wasn't done in the past was to allow checksum verification of the kernel image between boots. With this new patch, libc.so would change between boots precluding the possibility of such checksum verification, a contingency I believe was properly realized and accounted for. If we can stand to forgo the checksum verification of a big target like libc.so, perhaps we can forgo it with the kernel image itself too and solve the boot-time entropy problem. Ian
Re: anti-ROP mechanism in libc
26 Apr. 2016 19:58 "Theo de Raadt"wrote: > > Here is a new version that does a more comprehensive test of the new > libc.so before installing it, and uses install -S > > Index: etc/rc > === > RCS file: /cvs/src/etc/rc,v > retrieving revision 1.474 > diff -u -p -u -r1.474 rc > --- etc/rc 29 Dec 2015 19:41:24 - 1.474 > +++ etc/rc 26 Apr 2016 11:56:46 - > @@ -158,6 +158,35 @@ make_keys() { > ssh-keygen -A > } > > +rebuildlibs() { > + local _l _liba _libas _tmpdir > + > + # Only choose newest > + for _liba in /usr/lib/libc.so.*.a; do > + _liba=$(ls ${_liba%%.[0-9]*}*.a | sort -n | tail -1) > + for _l in $_libas; do > + [[ $_l == $_liba ]] && continue 2 > + done > + _libas="$_libas $_liba" > + done I'm afraid sort -n would not behave the way you probably think: $ (echo 10.2; echo 10.10; echo 10.50) | sort -n 10.10 10.2 10.50 Also, you code does something strange, because $_liba will be always the same thing in the loop. > + for _liba in $_libas; do > + _tmpdir=$(mktemp -dq /tmp/_librebuild.) || return > + ( > + set -o errexit > + _lib=${_liba#/usr/lib/} > + _lib=${_lib%.a} > + cd $_tmpdir > + ar x ${_liba} > + cc -shared -o $_lib $(ls *.so | sort -R) $(cat .ldadd) > + [[ -s $_lib ]] && file $_lib | fgrep -q 'shared > object' > + LD_BIND_NOW=1 LD_LIBRARY_PATH=$_tmpdir awk 'BEGIN > {exit 0}' > + install -S -o root -g bin -m 0444 $_lib /usr/lib/$_lib > + ) > + rm -rf /tmp/_librebuild.${_tmpdir#*.} > + done > +} So I propose something like that instead: find_newest() { set -x local _l _ls _bestmaj _bestmin _maj _min for _l in /usr/lib/lib$1.so.+([0-9]).+([0-9]); do _ls=${_l%.*} _maj=${_ls##*.} _min=${_l##*.} if [ _maj -gt _bestmaj -o \ _maj -eq _bestmaj -a _min -gt _bestmin ]; then _bestmaj=$_maj _bestmin=$_min fi done if [ -n $_bestmaj ]; then echo $_bestmaj.$_bestmin else return 1 fi } rebuildlibs() { local _lib _tmpdir _v _v=$(find_newest c) || return _lib=libc.so.$_v _tmpdir=$(mktemp -dq /tmp/_librebuild.) || return ( set -o errexit cd $_tmpdir ar x ${_lib}.a cc -shared -o $_lib $(ls *.so | sort -R) $(cat .ldadd) [[ -s $_lib ]] && file $_lib | fgrep -q 'shared object' LD_BIND_NOW=1 LD_LIBRARY_PATH=$_tmpdir awk 'BEGIN {exit 0}' install -S -o root -g bin -m 0444 $_lib /usr/lib/$_lib ) } -- WBR, Vadim Zhukov
Re: anti-ROP mechanism in libc
Here is a new version that does a more comprehensive test of the new libc.so before installing it, and uses install -S Index: etc/rc === RCS file: /cvs/src/etc/rc,v retrieving revision 1.474 diff -u -p -u -r1.474 rc --- etc/rc 29 Dec 2015 19:41:24 - 1.474 +++ etc/rc 26 Apr 2016 11:56:46 - @@ -158,6 +158,35 @@ make_keys() { ssh-keygen -A } +rebuildlibs() { + local _l _liba _libas _tmpdir + + # Only choose newest + for _liba in /usr/lib/libc.so.*.a; do + _liba=$(ls ${_liba%%.[0-9]*}*.a | sort -n | tail -1) + for _l in $_libas; do + [[ $_l == $_liba ]] && continue 2 + done + _libas="$_libas $_liba" + done + + for _liba in $_libas; do + _tmpdir=$(mktemp -dq /tmp/_librebuild.) || return + ( + set -o errexit + _lib=${_liba#/usr/lib/} + _lib=${_lib%.a} + cd $_tmpdir + ar x ${_liba} + cc -shared -o $_lib $(ls *.so | sort -R) $(cat .ldadd) + [[ -s $_lib ]] && file $_lib | fgrep -q 'shared object' + LD_BIND_NOW=1 LD_LIBRARY_PATH=$_tmpdir awk 'BEGIN {exit 0}' + install -S -o root -g bin -m 0444 $_lib /usr/lib/$_lib + ) + rm -rf /tmp/_librebuild.${_tmpdir#*.} + done +} + # Check filesystems, optionally by using a fsck(8) flag. # Usage: do_fsck [-flag] do_fsck() { @@ -337,6 +366,8 @@ mount -s /usr >/dev/null 2>&1 mount -s /var >/dev/null 2>&1 random_seed + +rebuildlibs # Clean up left-over files. rm -f /etc/nologin /var/spool/lock/LCK.* /var/spool/uucp/STST/* Index: share/mk/bsd.lib.mk === RCS file: /cvs/src/share/mk/bsd.lib.mk,v retrieving revision 1.74 diff -u -p -u -r1.74 bsd.lib.mk --- share/mk/bsd.lib.mk 26 Oct 2015 10:43:42 - 1.74 +++ share/mk/bsd.lib.mk 25 Apr 2016 08:58:26 - @@ -174,6 +174,15 @@ FULLSHLIBNAME=lib${LIB}.so.${SHLIB_MAJOR _LIBS+=${FULLSHLIBNAME} .endif +.if defined(LIBREBUILD) +_LIBS+=${FULLSHLIBNAME}.a + +.if exists(${.CURDIR}/Symbols.list) +SYMBOLSMAP=Symbols.map +.endif + +.endif + .if defined(VERSION_SCRIPT) ${FULLSHLIBNAME}: ${VERSION_SCRIPT} LDADD+=-Wl,--version-script=${VERSION_SCRIPT} @@ -209,7 +218,13 @@ ${FULLSHLIBNAME}: ${SOBJS} ${DPADD} @echo building shared ${LIB} library \(version ${SHLIB_MAJOR}.${SHLIB_MINOR}\) @rm -f ${.TARGET} ${CC} -shared ${PICFLAG} -o ${.TARGET} \ - `${LORDER} ${SOBJS}|tsort -q` ${LDADD} + `echo ${SOBJS} | tr ' ' '\n' | sort -R` ${LDADD} + +${FULLSHLIBNAME}.a: ${SOBJS} + @echo building shared ${LIB} library \(version ${SHLIB_MAJOR}.${SHLIB_MINOR}\) ar + @rm -f ${.TARGET} + @echo ${PICFLAG} ${LDADD} > .ldadd + ar cq ${FULLSHLIBNAME}.a ${SOBJS} .ldadd ${SYMBOLSMAP} # all .do files... DOBJS+=${OBJS:.o=.do} @@ -290,6 +305,10 @@ realinstall: .if !defined(NOPIC) && defined(SHLIB_MAJOR) && defined(SHLIB_MINOR) ${INSTALL} ${INSTALL_COPY} -S -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${FULLSHLIBNAME} ${DESTDIR}${LIBDIR} +.if defined(LIBREBUILD) + ${INSTALL} ${INSTALL_COPY} -S -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ + ${FULLSHLIBNAME}.a ${DESTDIR}${LIBDIR} +.endif .endif .if defined(LINKS) && !empty(LINKS) . for lnk file in ${LINKS} Index: lib/libc/Makefile === RCS file: /cvs/src/lib/libc/Makefile,v retrieving revision 1.38 diff -u -p -u -r1.38 Makefile --- lib/libc/Makefile 10 Nov 2015 04:14:03 - 1.38 +++ lib/libc/Makefile 28 Mar 2016 04:08:34 - @@ -6,6 +6,7 @@ .include LIB=c +LIBREBUILD=y CLEANFILES+=tags Symbols.map CFLAGS+=-Wimplicit #CFLAGS+=-Werror Index: distrib/sets/lists/base/md.alpha === RCS file: /cvs/src/distrib/sets/lists/base/md.alpha,v retrieving revision 1.1097 diff -u -p -u -r1.1097 md.alpha --- distrib/sets/lists/base/md.alpha26 Apr 2016 05:54:20 - 1.1097 +++ distrib/sets/lists/base/md.alpha26 Apr 2016 07:02:07 - @@ -61,6 +61,11 @@ ./sbin/kbd ./sbin/mount_ntfs ./sbin/wsconsctl +./usr/lib/gcc-lib/alpha-unknown-openbsd5.9 +./usr/lib/gcc-lib/alpha-unknown-openbsd5.9/4.2.1 +./usr/lib/gcc-lib/alpha-unknown-openbsd5.9/4.2.1/collect2 +./usr/lib/gcc-lib/alpha-unknown-openbsd5.9/4.2.1/libgcc.a +./usr/lib/gcc-lib/alpha-unknown-openbsd5.9/4.2.1/specs ./usr/libdata/perl5/alpha-openbsd ./usr/libdata/perl5/alpha-openbsd/5.20.2 ./usr/libdata/perl5/alpha-openbsd/5.20.2/.packlist Index: distrib/sets/lists/base/md.amd64 === RCS
Re: anti-ROP mechanism in libc
On Mon, Apr 25, 2016 at 03:23:47PM +, Robert Peichaer wrote: > On Mon, Apr 25, 2016 at 10:57:37AM -0400, Ted Unangst wrote: > > Theo de Raadt wrote: > > > + cp -p /usr/lib/$_lib /usr/lib/$_tmplib > > > + install -o root -g bin -m 0444 $_lib /usr/lib/$_lib && > > > + rm -f /usr/lib/$_tmplib || > > > + mv /usr/lib/$_tmplib /usr/lib/$_lib > > > > I'm a little confused by what's going on here. If the install fails, do we > > still want to overwrite the lib? > > > If the install fails, the original library file is restored. > > The "install .. && rm .. || mv ..." is identical to if-then-else and could > be written like this too. Nitpicking: nope, It is not identical, see: https://github.com/koalaman/shellcheck/wiki/SC2015 Though, may not matter here. > if install -o root -g bin -m 0444 $_lib /usr/lib/$_lib; then > rm -f /usr/lib/$_tmplib > else > mv /usr/lib/$_tmplib /usr/lib/$_lib > fi >
Re: anti-ROP mechanism in libc
>> Wait! Does that mean there is a moment where there is not a valid >> libc.so installed? That would be wrong wouldn't it? >> >> Doesn't the install command guarantee atomicity? > > Well, this is the same procedure we use during every make build, > and it works. It had been fixed to use install -S some years ago because it was not atomic enough otherwise.
Re: anti-ROP mechanism in libc
On Mon, 25 Apr 2016 10:18:56 -0600, "Todd C. Miller" wrote: > On Mon, 25 Apr 2016 18:04:58 +0200, Mark Kettenis wrote: > > > Wait! Does that mean there is a moment where there is not a valid > > libc.so installed? That would be wrong wouldn't it? > > > > Doesn't the install command guarantee atomicity? > > Seems like it would be safer to just install as /usr/lib/$_tmplib > and then rename to /usr/lib/$_lib. Trying to recover from a failed > copy to /usr/lib/$_lib is potentially nasty (yes, I am aware that > mv is statically linked). A better approach is to use install(1)'s -S option (safe mode) which will copy the file to a temp for you and rename it to the target name if the copy is successful. That way install(1) does the cleanup. This is how we install libraries via bsd.lib.mk too. So instead of: cp -p /usr/lib/$_lib /usr/lib/$_tmplib install -o root -g bin -m 0444 $_lib /usr/lib/$_lib && rm -f /usr/lib/$_tmplib || mv /usr/lib/$_tmplib /usr/lib/$_lib All you need is: install -c -S -o root -g bin -m 0444 $_lib /usr/lib/$_lib - todd
Re: anti-ROP mechanism in libc
> On Mon, Apr 25, 2016 at 8:23 AM, Robert Peichaerwrote: > > If the install fails, the original library file is restored. > > > > The "install .. && rm .. || mv ..." is identical to if-then-else and could > > be written like this too. > > > > if install -o root -g bin -m 0444 $_lib /usr/lib/$_lib; then > > rm -f /usr/lib/$_tmplib > > else > > mv /usr/lib/$_tmplib /usr/lib/$_lib > > fi > > Shouldn't this use install -S like a normal src build does? Then the > need for cleanup on failure goes away, no? Yes, perhaps.
Re: anti-ROP mechanism in libc
Robert Peichaer wrote: > On Mon, Apr 25, 2016 at 10:57:37AM -0400, Ted Unangst wrote: > > Theo de Raadt wrote: > > > + cp -p /usr/lib/$_lib /usr/lib/$_tmplib > > > + install -o root -g bin -m 0444 $_lib /usr/lib/$_lib && > > > + rm -f /usr/lib/$_tmplib || > > > + mv /usr/lib/$_tmplib /usr/lib/$_lib > > > > I'm a little confused by what's going on here. If the install fails, do we > > still want to overwrite the lib? > > > If the install fails, the original library file is restored. ok, i didn't realize what tmplib was.
Re: anti-ROP mechanism in libc
On Mon, Apr 25, 2016 at 8:23 AM, Robert Peichaerwrote: > If the install fails, the original library file is restored. > > The "install .. && rm .. || mv ..." is identical to if-then-else and could > be written like this too. > > if install -o root -g bin -m 0444 $_lib /usr/lib/$_lib; then > rm -f /usr/lib/$_tmplib > else > mv /usr/lib/$_tmplib /usr/lib/$_lib > fi Shouldn't this use install -S like a normal src build does? Then the need for cleanup on failure goes away, no?
Re: anti-ROP mechanism in libc
> Wait! Does that mean there is a moment where there is not a valid > libc.so installed? That would be wrong wouldn't it? > > Doesn't the install command guarantee atomicity? Well, this is the same procedure we use during every make build, and it works.
Re: anti-ROP mechanism in libc
On Mon, 25 Apr 2016 18:04:58 +0200, Mark Kettenis wrote: > Wait! Does that mean there is a moment where there is not a valid > libc.so installed? That would be wrong wouldn't it? > > Doesn't the install command guarantee atomicity? Seems like it would be safer to just install as /usr/lib/$_tmplib and then rename to /usr/lib/$_lib. Trying to recover from a failed copy to /usr/lib/$_lib is potentially nasty (yes, I am aware that mv is statically linked). - todd
Re: anti-ROP mechanism in libc
On Mon, Apr 25, 2016 at 10:57:37AM -0400, Ted Unangst wrote: > Theo de Raadt wrote: > > + cp -p /usr/lib/$_lib /usr/lib/$_tmplib > > + install -o root -g bin -m 0444 $_lib /usr/lib/$_lib && > > + rm -f /usr/lib/$_tmplib || > > + mv /usr/lib/$_tmplib /usr/lib/$_lib > > I'm a little confused by what's going on here. If the install fails, do we > still want to overwrite the lib? If the install fails, the original library file is restored. The "install .. && rm .. || mv ..." is identical to if-then-else and could be written like this too. if install -o root -g bin -m 0444 $_lib /usr/lib/$_lib; then rm -f /usr/lib/$_tmplib else mv /usr/lib/$_tmplib /usr/lib/$_lib fi
Re: anti-ROP mechanism in libc
Theo de Raadt wrote: > + cp -p /usr/lib/$_lib /usr/lib/$_tmplib > + install -o root -g bin -m 0444 $_lib /usr/lib/$_lib && > + rm -f /usr/lib/$_tmplib || > + mv /usr/lib/$_tmplib /usr/lib/$_lib I'm a little confused by what's going on here. If the install fails, do we still want to overwrite the lib?
anti-ROP mechanism in libc
This change randomizes the order of symbols in libc.so at boot time. This is done by saving all the independent .so sub-files into an ar archive, and then relinking them into a new libc.so in random order, at each boot. The cost is less than a second on the systems I am using. For now, this is only done for libc, because it is generally the most gadget heavy library; spilled registers are more likely to point within the libc segment; and also the gadgets are close to system call stubs. As a result of the change, gadgets are no longer found at fixed offsets from spilled registers. (I have run this on my systems for all base/X libraries, which exposed no strange behaviour roughly 3 seconds of rebuild time at boot) I have included the sets changes, to show that a few compile tools must move into base. This should allow comp-less installs to continue working. My horrible shell scripts were improved by rpe, who also did other testing. Index: share/mk/bsd.lib.mk === RCS file: /cvs/src/share/mk/bsd.lib.mk,v retrieving revision 1.74 diff -u -p -u -r1.74 bsd.lib.mk --- share/mk/bsd.lib.mk 26 Oct 2015 10:43:42 - 1.74 +++ share/mk/bsd.lib.mk 25 Apr 2016 08:58:26 - @@ -174,6 +174,15 @@ FULLSHLIBNAME=lib${LIB}.so.${SHLIB_MAJOR _LIBS+=${FULLSHLIBNAME} .endif +.if defined(LIBREBUILD) +_LIBS+=${FULLSHLIBNAME}.a + +.if exists(${.CURDIR}/Symbols.list) +SYMBOLSMAP=Symbols.map +.endif + +.endif + .if defined(VERSION_SCRIPT) ${FULLSHLIBNAME}: ${VERSION_SCRIPT} LDADD+=-Wl,--version-script=${VERSION_SCRIPT} @@ -209,7 +218,13 @@ ${FULLSHLIBNAME}: ${SOBJS} ${DPADD} @echo building shared ${LIB} library \(version ${SHLIB_MAJOR}.${SHLIB_MINOR}\) @rm -f ${.TARGET} ${CC} -shared ${PICFLAG} -o ${.TARGET} \ - `${LORDER} ${SOBJS}|tsort -q` ${LDADD} + `echo ${SOBJS} | tr ' ' '\n' | sort -R` ${LDADD} + +${FULLSHLIBNAME}.a: ${SOBJS} + @echo building shared ${LIB} library \(version ${SHLIB_MAJOR}.${SHLIB_MINOR}\) ar + @rm -f ${.TARGET} + @echo ${PICFLAG} ${LDADD} > .ldadd + ar cq ${FULLSHLIBNAME}.a ${SOBJS} .ldadd ${SYMBOLSMAP} # all .do files... DOBJS+=${OBJS:.o=.do} @@ -290,6 +305,10 @@ realinstall: .if !defined(NOPIC) && defined(SHLIB_MAJOR) && defined(SHLIB_MINOR) ${INSTALL} ${INSTALL_COPY} -S -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${FULLSHLIBNAME} ${DESTDIR}${LIBDIR} +.if defined(LIBREBUILD) + ${INSTALL} ${INSTALL_COPY} -S -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ + ${FULLSHLIBNAME}.a ${DESTDIR}${LIBDIR} +.endif .endif .if defined(LINKS) && !empty(LINKS) . for lnk file in ${LINKS} Index: etc/rc === RCS file: /cvs/src/etc/rc,v retrieving revision 1.474 diff -u -p -u -r1.474 rc --- etc/rc 29 Dec 2015 19:41:24 - 1.474 +++ etc/rc 28 Mar 2016 22:58:45 - @@ -158,6 +158,37 @@ make_keys() { ssh-keygen -A } +rebuildlibs() { + local _l _liba _libas _tmpdir + + for _liba in /usr/lib/libc.so.*.a; do + _liba=$(ls ${_liba%%.[0-9]*}*.a | sort -n | tail -1) + for _l in $_libas; do + [[ $_l == $_liba ]] && continue 2 + done + _libas="$_libas $_liba" + done + + for _liba in $_libas; do + _tmpdir=$(mktemp -dq /tmp/_librebuild.) || return + ( + set -o errexit + _lib=${_liba#/usr/lib/} + _lib=${_lib%.a} + _tmplib=$(mktemp $_lib.) + cd $_tmpdir + ar x ${_liba} + cc -shared -o $_lib $(ls *.so | sort -R) $(cat .ldadd) + [[ -s $_lib ]] && file $_lib | fgrep -q 'shared object' + cp -p /usr/lib/$_lib /usr/lib/$_tmplib + install -o root -g bin -m 0444 $_lib /usr/lib/$_lib && + rm -f /usr/lib/$_tmplib || + mv /usr/lib/$_tmplib /usr/lib/$_lib + ) + rm -rf /tmp/_librebuild.${_tmpdir#*.} + done +} + # Check filesystems, optionally by using a fsck(8) flag. # Usage: do_fsck [-flag] do_fsck() { @@ -337,6 +368,8 @@ mount -s /usr >/dev/null 2>&1 mount -s /var >/dev/null 2>&1 random_seed + +rebuildlibs # Clean up left-over files. rm -f /etc/nologin /var/spool/lock/LCK.* /var/spool/uucp/STST/* Index: lib/libc/Makefile === RCS file: /cvs/src/lib/libc/Makefile,v retrieving revision 1.38 diff -u -p -u -r1.38 Makefile --- lib/libc/Makefile 10 Nov 2015 04:14:03 - 1.38 +++ lib/libc/Makefile 28 Mar 2016 04:08:34 - @@ -6,6 +6,7 @@ .include LIB=c +LIBREBUILD=y CLEANFILES+=tags Symbols.map