Re: HEADS-UP: ASLR for 64-bit executables enabled by default on main
On Fri, Dec 10, 2021 at 06:35:47PM +0100, Marcin Wojtas wrote: > Hi Daniel > > > pt., 10 gru 2021 o 10:16 Daniel O'Connor napisał(a): > > > > > > > > > On 17 Nov 2021, at 09:00, Marcin Wojtas wrote: > > > As of b014e0f15bc7 the ASLR (Address Space Layout > > > Randomization) feature becomes enabled for the all 64-bit > > > binaries by default. > > > > Firstly, thank your for your efforts here, it is appreciated :) > > > > I am finding that the lang/sdcc port is crashing with a seg fault and the > > core dump is no help to me at all: > > [freebsd14 7:06] /usr/ports/lang/sdcc/work/sdcc-4.0.0/device/lib >sudo gdb > > ../../bin/sdcc sdcc.core > > GNU gdb (GDB) 11.1 [GDB v11.1 for FreeBSD] > > > > Reading symbols from ../../bin/sdcc... > > [New LWP 100122] > > Core was generated by `../../bin/sdcc -I../../device/include > > -I../../device/include/mcs51 -mds390 --nos'. > > Program terminated with signal SIGSEGV, Segmentation fault. > > Invalid permissions for mapped object. > > #0 0x000804e3fbc0 in setrlimit () from /lib/libc.so.7 > > (gdb) info thread > > Id Target Id Frame > > * 1LWP 1001220x000804e3fbc0 in setrlimit () from > > /lib/libc.so.7 > > (gdb) bt > > #0 0x000804e3fbc0 in setrlimit () from /lib/libc.so.7 > > Backtrace stopped: Cannot access memory at address 0x7f87fd08 > > > > If I disable ASLR (via proccontrol) then it does not crash, but I am not > > sure how I can debug it further. > > > > I've raised a bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260303 > > if you (or anyone else) has suggestions for what to try. > > > > Thanks for filing the ticket. Let's continue the conversation there. I left a comment there. The gist of it is that there are several lingering problems with the stack gap implementation, and I think we should re-disable it on main until there's some consensus on how to proceed.
Re: HEADS-UP: ASLR for 64-bit executables enabled by default on main
Hi Daniel pt., 10 gru 2021 o 10:16 Daniel O'Connor napisał(a): > > > > > On 17 Nov 2021, at 09:00, Marcin Wojtas wrote: > > As of b014e0f15bc7 the ASLR (Address Space Layout > > Randomization) feature becomes enabled for the all 64-bit > > binaries by default. > > Firstly, thank your for your efforts here, it is appreciated :) > > I am finding that the lang/sdcc port is crashing with a seg fault and the > core dump is no help to me at all: > [freebsd14 7:06] /usr/ports/lang/sdcc/work/sdcc-4.0.0/device/lib >sudo gdb > ../../bin/sdcc sdcc.core > GNU gdb (GDB) 11.1 [GDB v11.1 for FreeBSD] > > Reading symbols from ../../bin/sdcc... > [New LWP 100122] > Core was generated by `../../bin/sdcc -I../../device/include > -I../../device/include/mcs51 -mds390 --nos'. > Program terminated with signal SIGSEGV, Segmentation fault. > Invalid permissions for mapped object. > #0 0x000804e3fbc0 in setrlimit () from /lib/libc.so.7 > (gdb) info thread > Id Target Id Frame > * 1LWP 1001220x000804e3fbc0 in setrlimit () from > /lib/libc.so.7 > (gdb) bt > #0 0x000804e3fbc0 in setrlimit () from /lib/libc.so.7 > Backtrace stopped: Cannot access memory at address 0x7f87fd08 > > If I disable ASLR (via proccontrol) then it does not crash, but I am not sure > how I can debug it further. > > I've raised a bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260303 if > you (or anyone else) has suggestions for what to try. > Thanks for filing the ticket. Let's continue the conversation there. Best regards, Marcin
Re: HEADS-UP: ASLR for 64-bit executables enabled by default on main
> On 17 Nov 2021, at 09:00, Marcin Wojtas wrote: > As of b014e0f15bc7 the ASLR (Address Space Layout > Randomization) feature becomes enabled for the all 64-bit > binaries by default. Firstly, thank your for your efforts here, it is appreciated :) I am finding that the lang/sdcc port is crashing with a seg fault and the core dump is no help to me at all: [freebsd14 7:06] /usr/ports/lang/sdcc/work/sdcc-4.0.0/device/lib >sudo gdb ../../bin/sdcc sdcc.core GNU gdb (GDB) 11.1 [GDB v11.1 for FreeBSD] Reading symbols from ../../bin/sdcc... [New LWP 100122] Core was generated by `../../bin/sdcc -I../../device/include -I../../device/include/mcs51 -mds390 --nos'. Program terminated with signal SIGSEGV, Segmentation fault. Invalid permissions for mapped object. #0 0x000804e3fbc0 in setrlimit () from /lib/libc.so.7 (gdb) info thread Id Target Id Frame * 1LWP 1001220x000804e3fbc0 in setrlimit () from /lib/libc.so.7 (gdb) bt #0 0x000804e3fbc0 in setrlimit () from /lib/libc.so.7 Backtrace stopped: Cannot access memory at address 0x7f87fd08 If I disable ASLR (via proccontrol) then it does not crash, but I am not sure how I can debug it further. I've raised a bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260303 if you (or anyone else) has suggestions for what to try. Thanks. -- Daniel O'Connor "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum
Re: HEADS-UP: ASLR for 64-bit executables enabled by default on main
On Sat, 20 Nov 2021 at 20:00, Ed Maste wrote: > > On Thu, 18 Nov 2021 at 13:09, Li-Wen Hsu wrote: > > > > The mkimg ones are a bit tricky, it seems the output is changed in > > each run. We may need a way to generate reproducible results.. > > Hopefully fixed by 036af1053acd6cae68c5fb6bed30508f2e40be13. The mkimg failures are indeed fixed by the above commit - it was just a latent bug in mkimg. I've opened PR 259968 as a tracking bug for outstanding issues found as a result of enabling ASLR by default, and submitted a PR for each of the three outstanding issues.
Re: HEADS-UP: ASLR for 64-bit executables enabled by default on main
On Thu, 18 Nov 2021 at 13:09, Li-Wen Hsu wrote: > > The mkimg ones are a bit tricky, it seems the output is changed in > each run. We may need a way to generate reproducible results.. Hopefully fixed by 036af1053acd6cae68c5fb6bed30508f2e40be13.
Re: HEADS-UP: ASLR for 64-bit executables enabled by default on main
> On 18 Nov 2021, at 11:43, Marcin Wojtas wrote:
> czw., 18 lis 2021 o 19:07 Li-Wen Hsu napisał(a):
>>
>>> On Wed, Nov 17, 2021 at 6:30 AM Marcin Wojtas wrote:
>>>
>>> As of b014e0f15bc7 the ASLR (Address Space Layout
>>> Randomization) feature becomes enabled for the all 64-bit
>>> binaries by default.
>>>
>>> Address Space Layout Randomization (ASLR) is an exploit mitigation
>>> technique implemented in the majority of modern operating systems.
>>> It involves randomly positioning the base address of an executable
>>> and the position of libraries, heap, and stack, in a process's address
>>> space. Although over the years ASLR proved to not guarantee full OS
>>> security on its own, this mechanism can make exploitation more difficult
>>> (especially when combined with other methods, such as W^X).
>>>
>>> Tests on the tier 1 64-bit architectures demonstrated that the ASLR is
>>> stable and does not result in noticeable performance degradation,
>>> therefore it is considered safe to enable this mechanism by default.
>>> Moreover its effectiveness is increased for PIE (Position Independent
>>> Executable) binaries. Thanks to commit 9a227a2fd642 ("Enable PIE by
>>> default on 64-bit architectures"), building from src is not necessary
>>> to have PIE binaries and it is enough to control usage of ASLR in the
>>> OS solely by setting the appropriate sysctls. The defaults were toggled
>>> for the 64-bit PIE and non-PIE executables.
>>>
>>> As for the drawbacks, a consequence of using the ASLR is more
>>> significant VM fragmentation, hence the issues may be encountered
>>> in the systems with a limited address space in high memory consumption
>>> cases, such as buildworld. As a result, although the tests on 32-bit
>>> architectures with ASLR enabled were mostly on par with what was
>>> observed on 64-bit ones, the defaults for the former are not changed
>>> at this time. Also, for the sake of safety the feature remains disabled
>>> for 32-bit executables on 64-bit machines, too.
>>>
>>> The committed change affects the overall OS operation, so the
>>> following should be taken into consideration:
>>> * Address space fragmentation.
>>> * A changed ABI due to modified layout of address space.
>>> * More complicated debugging due to:
>>> * Non-reproducible address space layout between runs.
>>> * Some debuggers automatically disable ASLR for spawned processes,
>>>making target's environment different between debug and
>>>non-debug runs.
>>>
>>> The known issues (such as PR239873 or PR253208) have been fixed in
>>> HEAD up front, however please pay attention to the system behavior after
>>> upgrading the kernel to the newest revisions.
>>> In order to confirm/rule-out the dependency of any encountered issue
>>> on ASLR it is strongly advised to re-run the test with the feature
>>> disabled - it can be done by setting the following sysctls
>>> in the /etc/sysctl.conf file:
>>> kern.elf64.aslr.enable=0
>>> kern.elf64.aslr.pie_enable=0
>>>
>>> The change is a result of combined efforts under the auspices
>>> of the FreeBSD Foundation and the Semihalf team sponsored
>>> by Stormshield.
>>>
>>> Best regards,
>>> Marcin
>>
>> Thanks very much for working on this. FYI, there are some test cases
>> seem to be affected by this:
>>
>> https://ci.freebsd.org/job/FreeBSD-main-amd64-test/19828/testReport/
>>
>> The mkimg ones are a bit tricky, it seems the output is changed in
>> each run. We may need a way to generate reproducible results..
>>
>> I'm still checking them, but hope more people can join and fix them.
>>
>
> Thanks for bringing this up! Apart from
> sys.netpfil.common.dummynet.pf_nat other are 23 are new.
I’ve just managed to reproduce that one locally (it only happens if ipfw is
also loaded) and will dig in soon. It’s not going to be aslr related. You can
ignore that failure.
Kristof
Re: HEADS-UP: ASLR for 64-bit executables enabled by default on main
Hi,
czw., 18 lis 2021 o 19:07 Li-Wen Hsu napisał(a):
>
> On Wed, Nov 17, 2021 at 6:30 AM Marcin Wojtas wrote:
> >
> > As of b014e0f15bc7 the ASLR (Address Space Layout
> > Randomization) feature becomes enabled for the all 64-bit
> > binaries by default.
> >
> > Address Space Layout Randomization (ASLR) is an exploit mitigation
> > technique implemented in the majority of modern operating systems.
> > It involves randomly positioning the base address of an executable
> > and the position of libraries, heap, and stack, in a process's address
> > space. Although over the years ASLR proved to not guarantee full OS
> > security on its own, this mechanism can make exploitation more difficult
> > (especially when combined with other methods, such as W^X).
> >
> > Tests on the tier 1 64-bit architectures demonstrated that the ASLR is
> > stable and does not result in noticeable performance degradation,
> > therefore it is considered safe to enable this mechanism by default.
> > Moreover its effectiveness is increased for PIE (Position Independent
> > Executable) binaries. Thanks to commit 9a227a2fd642 ("Enable PIE by
> > default on 64-bit architectures"), building from src is not necessary
> > to have PIE binaries and it is enough to control usage of ASLR in the
> > OS solely by setting the appropriate sysctls. The defaults were toggled
> > for the 64-bit PIE and non-PIE executables.
> >
> > As for the drawbacks, a consequence of using the ASLR is more
> > significant VM fragmentation, hence the issues may be encountered
> > in the systems with a limited address space in high memory consumption
> > cases, such as buildworld. As a result, although the tests on 32-bit
> > architectures with ASLR enabled were mostly on par with what was
> > observed on 64-bit ones, the defaults for the former are not changed
> > at this time. Also, for the sake of safety the feature remains disabled
> > for 32-bit executables on 64-bit machines, too.
> >
> > The committed change affects the overall OS operation, so the
> > following should be taken into consideration:
> > * Address space fragmentation.
> > * A changed ABI due to modified layout of address space.
> > * More complicated debugging due to:
> > * Non-reproducible address space layout between runs.
> > * Some debuggers automatically disable ASLR for spawned processes,
> > making target's environment different between debug and
> > non-debug runs.
> >
> > The known issues (such as PR239873 or PR253208) have been fixed in
> > HEAD up front, however please pay attention to the system behavior after
> > upgrading the kernel to the newest revisions.
> > In order to confirm/rule-out the dependency of any encountered issue
> > on ASLR it is strongly advised to re-run the test with the feature
> > disabled - it can be done by setting the following sysctls
> > in the /etc/sysctl.conf file:
> > kern.elf64.aslr.enable=0
> > kern.elf64.aslr.pie_enable=0
> >
> > The change is a result of combined efforts under the auspices
> > of the FreeBSD Foundation and the Semihalf team sponsored
> > by Stormshield.
> >
> > Best regards,
> > Marcin
>
> Thanks very much for working on this. FYI, there are some test cases
> seem to be affected by this:
>
> https://ci.freebsd.org/job/FreeBSD-main-amd64-test/19828/testReport/
>
> The mkimg ones are a bit tricky, it seems the output is changed in
> each run. We may need a way to generate reproducible results..
>
> I'm still checking them, but hope more people can join and fix them.
>
Thanks for bringing this up! Apart from
sys.netpfil.common.dummynet.pf_nat other are 23 are new. I checked a
couple of next builds and the results seems to be consistent. There
are:
lib.libc.sys.setrlimit_test.setrlimit_stack
lib.libc.regex.exhaust_test.regcomp_too_big
sys.kern.coredump_phnum_test.coredump_phnum
and 20 of usr.bin.mkimg.*
We will also check and try to fix the new issues (however any help
would be appreciated), it may be also worth to create dedicated
tickets in bugzilla.
Best regards,
Marcin
Re: HEADS-UP: ASLR for 64-bit executables enabled by default on main
On Wed, Nov 17, 2021 at 6:30 AM Marcin Wojtas wrote:
>
> As of b014e0f15bc7 the ASLR (Address Space Layout
> Randomization) feature becomes enabled for the all 64-bit
> binaries by default.
>
> Address Space Layout Randomization (ASLR) is an exploit mitigation
> technique implemented in the majority of modern operating systems.
> It involves randomly positioning the base address of an executable
> and the position of libraries, heap, and stack, in a process's address
> space. Although over the years ASLR proved to not guarantee full OS
> security on its own, this mechanism can make exploitation more difficult
> (especially when combined with other methods, such as W^X).
>
> Tests on the tier 1 64-bit architectures demonstrated that the ASLR is
> stable and does not result in noticeable performance degradation,
> therefore it is considered safe to enable this mechanism by default.
> Moreover its effectiveness is increased for PIE (Position Independent
> Executable) binaries. Thanks to commit 9a227a2fd642 ("Enable PIE by
> default on 64-bit architectures"), building from src is not necessary
> to have PIE binaries and it is enough to control usage of ASLR in the
> OS solely by setting the appropriate sysctls. The defaults were toggled
> for the 64-bit PIE and non-PIE executables.
>
> As for the drawbacks, a consequence of using the ASLR is more
> significant VM fragmentation, hence the issues may be encountered
> in the systems with a limited address space in high memory consumption
> cases, such as buildworld. As a result, although the tests on 32-bit
> architectures with ASLR enabled were mostly on par with what was
> observed on 64-bit ones, the defaults for the former are not changed
> at this time. Also, for the sake of safety the feature remains disabled
> for 32-bit executables on 64-bit machines, too.
>
> The committed change affects the overall OS operation, so the
> following should be taken into consideration:
> * Address space fragmentation.
> * A changed ABI due to modified layout of address space.
> * More complicated debugging due to:
> * Non-reproducible address space layout between runs.
> * Some debuggers automatically disable ASLR for spawned processes,
> making target's environment different between debug and
> non-debug runs.
>
> The known issues (such as PR239873 or PR253208) have been fixed in
> HEAD up front, however please pay attention to the system behavior after
> upgrading the kernel to the newest revisions.
> In order to confirm/rule-out the dependency of any encountered issue
> on ASLR it is strongly advised to re-run the test with the feature
> disabled - it can be done by setting the following sysctls
> in the /etc/sysctl.conf file:
> kern.elf64.aslr.enable=0
> kern.elf64.aslr.pie_enable=0
>
> The change is a result of combined efforts under the auspices
> of the FreeBSD Foundation and the Semihalf team sponsored
> by Stormshield.
>
> Best regards,
> Marcin
Thanks very much for working on this. FYI, there are some test cases
seem to be affected by this:
https://ci.freebsd.org/job/FreeBSD-main-amd64-test/19828/testReport/
The mkimg ones are a bit tricky, it seems the output is changed in
each run. We may need a way to generate reproducible results..
I'm still checking them, but hope more people can join and fix them.
Best,
Li-Wen
HEADS-UP: ASLR for 64-bit executables enabled by default on main
As of b014e0f15bc7 the ASLR (Address Space Layout
Randomization) feature becomes enabled for the all 64-bit
binaries by default.
Address Space Layout Randomization (ASLR) is an exploit mitigation
technique implemented in the majority of modern operating systems.
It involves randomly positioning the base address of an executable
and the position of libraries, heap, and stack, in a process's address
space. Although over the years ASLR proved to not guarantee full OS
security on its own, this mechanism can make exploitation more difficult
(especially when combined with other methods, such as W^X).
Tests on the tier 1 64-bit architectures demonstrated that the ASLR is
stable and does not result in noticeable performance degradation,
therefore it is considered safe to enable this mechanism by default.
Moreover its effectiveness is increased for PIE (Position Independent
Executable) binaries. Thanks to commit 9a227a2fd642 ("Enable PIE by
default on 64-bit architectures"), building from src is not necessary
to have PIE binaries and it is enough to control usage of ASLR in the
OS solely by setting the appropriate sysctls. The defaults were toggled
for the 64-bit PIE and non-PIE executables.
As for the drawbacks, a consequence of using the ASLR is more
significant VM fragmentation, hence the issues may be encountered
in the systems with a limited address space in high memory consumption
cases, such as buildworld. As a result, although the tests on 32-bit
architectures with ASLR enabled were mostly on par with what was
observed on 64-bit ones, the defaults for the former are not changed
at this time. Also, for the sake of safety the feature remains disabled
for 32-bit executables on 64-bit machines, too.
The committed change affects the overall OS operation, so the
following should be taken into consideration:
* Address space fragmentation.
* A changed ABI due to modified layout of address space.
* More complicated debugging due to:
* Non-reproducible address space layout between runs.
* Some debuggers automatically disable ASLR for spawned processes,
making target's environment different between debug and
non-debug runs.
The known issues (such as PR239873 or PR253208) have been fixed in
HEAD up front, however please pay attention to the system behavior after
upgrading the kernel to the newest revisions.
In order to confirm/rule-out the dependency of any encountered issue
on ASLR it is strongly advised to re-run the test with the feature
disabled - it can be done by setting the following sysctls
in the /etc/sysctl.conf file:
kern.elf64.aslr.enable=0
kern.elf64.aslr.pie_enable=0
The change is a result of combined efforts under the auspices
of the FreeBSD Foundation and the Semihalf team sponsored
by Stormshield.
Best regards,
Marcin
