Re: Address Collision using i386 4G/4G Memory Split
On 18.12.18 11:32, Konstantin Belousov wrote: > On Tue, Dec 18, 2018 at 11:22:53AM +0100, Alexander Lochmann wrote: >> Some context: We are doing VM-based tracing in the FreeBSD kernel. For that, we observe parts of the kernel memory (allocations, accesses,...). Before 12.0 we simply knew that kernel addresses that we logged were unique. Moreover, when a memory access to a region of interest happened we knew that could only be kernel memory. We know have to ensure that we only record memory accesses that happen within the kernel. Our approach is to record the kernels value for the CR3 register, and record memory accesses if the CR3 registers holds the aforementioned value. >>> You must use CPL to see if the current operation mode is user or kernel. >>> If user, nothing should be done (this would avoid vm86). If kernel, you >>> need to compare current %cr3 with IdlePTD (IdlePTDP for PAE case). >>> >> Thanks for the advice! We'll include that in our toolchain. >> Do you use PLs other than 0(=kernel) and 3(=user)? > No, only 0 and 3. But be careful with vm86 (I am not sure how your VM > reports it to your instrumentation). > Ok. Thx! - Alex -- Technische Universität Dortmund Alexander LochmannPGP key: 0xBC3EF6FD Otto-Hahn-Str. 16 phone: +49.231.7556141 D-44227 Dortmund fax:+49.231.7556116 http://ess.cs.tu-dortmund.de/Staff/al signature.asc Description: OpenPGP digital signature
Re: Address Collision using i386 4G/4G Memory Split
On Tue, Dec 18, 2018 at 11:22:53AM +0100, Alexander Lochmann wrote: > > >> Some context: We are doing VM-based tracing in the FreeBSD kernel. For > >> that, we observe parts of the kernel memory (allocations, accesses,...). > >> Before 12.0 we simply knew that kernel addresses that we logged were > >> unique. Moreover, when a memory access to a region of interest happened > >> we knew that could only be kernel memory. > >> We know have to ensure that we only record memory accesses that happen > >> within the kernel. > >> Our approach is to record the kernels value for the CR3 register, and > >> record memory accesses if the CR3 registers holds the aforementioned value. > > You must use CPL to see if the current operation mode is user or kernel. > > If user, nothing should be done (this would avoid vm86). If kernel, you > > need to compare current %cr3 with IdlePTD (IdlePTDP for PAE case). > > > Thanks for the advice! We'll include that in our toolchain. > Do you use PLs other than 0(=kernel) and 3(=user)? No, only 0 and 3. But be careful with vm86 (I am not sure how your VM reports it to your instrumentation). ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Address Collision using i386 4G/4G Memory Split
>> Some context: We are doing VM-based tracing in the FreeBSD kernel. For >> that, we observe parts of the kernel memory (allocations, accesses,...). >> Before 12.0 we simply knew that kernel addresses that we logged were >> unique. Moreover, when a memory access to a region of interest happened >> we knew that could only be kernel memory. >> We know have to ensure that we only record memory accesses that happen >> within the kernel. >> Our approach is to record the kernels value for the CR3 register, and >> record memory accesses if the CR3 registers holds the aforementioned value. > You must use CPL to see if the current operation mode is user or kernel. > If user, nothing should be done (this would avoid vm86). If kernel, you > need to compare current %cr3 with IdlePTD (IdlePTDP for PAE case). > Thanks for the advice! We'll include that in our toolchain. Do you use PLs other than 0(=kernel) and 3(=user)? - Alex -- Technische Universität Dortmund Alexander LochmannPGP key: 0xBC3EF6FD Otto-Hahn-Str. 16 phone: +49.231.7556141 D-44227 Dortmund fax:+49.231.7556116 http://ess.cs.tu-dortmund.de/Staff/al signature.asc Description: OpenPGP digital signature
Re: Address Collision using i386 4G/4G Memory Split
On Tue, Dec 18, 2018 at 10:16:35AM +0100, Alexander Lochmann wrote:
> Am 18.12.18 um 06:27 schrieb Konstantin Belousov:
> > On Mon, Dec 17, 2018 at 02:51:48PM +0100, Alexander Lochmann wrote:
> >> Hi folks!
> >>
> >> According to git commit e3089a (https://reviews.freebsd.org/D1463)
> >> FreeBSD 12.0 i386 uses separate address spaces for kernel and user
> >> space. So basically two memory areas, one in each space, can have the
> >> same address.
> >> Is this possible with FreeBSD 12.0? Is this likely to happen?
> > The feature was added to HEAD during this summer, before stable/12 was
> > branched.
> Mhmkay. But how likely is it that two memory areas will get the same
> address?
It is possible.
> Does the kernel, for example, start in the high memory region and the
> user space starts in the mid region?
No, kernel now does not relocate itself, it is running with PA == VA
for text and data segment. Look at the kernel binary to see the
addresses.
> This would reduce the likelihood of two memory areas starting at the
> same virtual address.
I do not see why this would be even slightly needed.
>
> Some context: We are doing VM-based tracing in the FreeBSD kernel. For
> that, we observe parts of the kernel memory (allocations, accesses,...).
> Before 12.0 we simply knew that kernel addresses that we logged were
> unique. Moreover, when a memory access to a region of interest happened
> we knew that could only be kernel memory.
> We know have to ensure that we only record memory accesses that happen
> within the kernel.
> Our approach is to record the kernels value for the CR3 register, and
> record memory accesses if the CR3 registers holds the aforementioned value.
You must use CPL to see if the current operation mode is user or kernel.
If user, nothing should be done (this would avoid vm86). If kernel, you
need to compare current %cr3 with IdlePTD (IdlePTDP for PAE case).
There are moments where kernel is executing on the user page tables.
This happens on kernel entry/exit, and sometimes on copyout(9).
>
> >
> >>
> >> On my opinion, this is also very expensive in terms of performance.
> >> Any copy{in,out} has to flush the TLB.
> >> (http://fxr.watson.org/fxr/source/i386/i386/copyout_fast.s#L91)
> >> Why are you still using this 4G/4G approach?
> > Because it is needed for i386 to self-host, in modern world 1G KVA
> > is too small, and because it provides Meltdown mitigation.
> >
>
> --
> Technische Universität Dortmund
> Alexander LochmannPGP key: 0xBC3EF6FD
> Otto-Hahn-Str. 16 phone: +49.231.7556141
> D-44227 Dortmund fax:+49.231.7556116
> http://ess.cs.tu-dortmund.de/Staff/al
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Address Collision using i386 4G/4G Memory Split
Am 18.12.18 um 06:27 schrieb Konstantin Belousov:
> On Mon, Dec 17, 2018 at 02:51:48PM +0100, Alexander Lochmann wrote:
>> Hi folks!
>>
>> According to git commit e3089a (https://reviews.freebsd.org/D1463)
>> FreeBSD 12.0 i386 uses separate address spaces for kernel and user
>> space. So basically two memory areas, one in each space, can have the
>> same address.
>> Is this possible with FreeBSD 12.0? Is this likely to happen?
> The feature was added to HEAD during this summer, before stable/12 was
> branched.
Mhmkay. But how likely is it that two memory areas will get the same
address?
Does the kernel, for example, start in the high memory region and the
user space starts in the mid region?
This would reduce the likelihood of two memory areas starting at the
same virtual address.
Some context: We are doing VM-based tracing in the FreeBSD kernel. For
that, we observe parts of the kernel memory (allocations, accesses,...).
Before 12.0 we simply knew that kernel addresses that we logged were
unique. Moreover, when a memory access to a region of interest happened
we knew that could only be kernel memory.
We know have to ensure that we only record memory accesses that happen
within the kernel.
Our approach is to record the kernels value for the CR3 register, and
record memory accesses if the CR3 registers holds the aforementioned value.
>
>>
>> On my opinion, this is also very expensive in terms of performance.
>> Any copy{in,out} has to flush the TLB.
>> (http://fxr.watson.org/fxr/source/i386/i386/copyout_fast.s#L91)
>> Why are you still using this 4G/4G approach?
> Because it is needed for i386 to self-host, in modern world 1G KVA
> is too small, and because it provides Meltdown mitigation.
>
--
Technische Universität Dortmund
Alexander LochmannPGP key: 0xBC3EF6FD
Otto-Hahn-Str. 16 phone: +49.231.7556141
D-44227 Dortmund fax:+49.231.7556116
http://ess.cs.tu-dortmund.de/Staff/al
signature.asc
Description: OpenPGP digital signature
Re: Address Collision using i386 4G/4G Memory Split
On Tue, Dec 18, 2018 at 08:34:25AM +, Brooks Davis wrote: > On Mon, Dec 17, 2018 at 03:58:05PM -0500, Kurt Lidl wrote: > > Alexander Lochmann writes: > > > According to git commit e3089a (https://reviews.freebsd.org/D1463) > > > FreeBSD 12.0 i386 uses separate address spaces for kernel and user > > > space. So basically two memory areas, one in each space, can have the > > > same address. > > > Is this possible with FreeBSD 12.0? Is this likely to happen? > > > > If the userspace program and the kernel address happen to overlap, the > > system will deal with it. There's not anything to worry about. As to > > whether or not it's likely to happen -- I'm not sure about that. I > > expect the default stack and heap space locations for a fresh process > > have changed due to this change, but it should not matter. > > 4/4 does potentially alter the failure modes of buggy code that tries to > read directly from userspace addresses. For example, correct calls to > the sysctls fixed in r342125 may panic prior to the fix because the > addresses in question aren't mapped in kernel space. They might also > fail or behave bizarrely if the page is mapped and the value from the > kernel page is used. I believe that SMAP on amd64 is The solution to find such cases, now. And it indeed catched several real cases, e.g. pci(4), acpi_call and vbox from ports, besides the mentioned commit.. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Address Collision using i386 4G/4G Memory Split
On Mon, Dec 17, 2018 at 03:58:05PM -0500, Kurt Lidl wrote: > Alexander Lochmann writes: > > According to git commit e3089a (https://reviews.freebsd.org/D1463) > > FreeBSD 12.0 i386 uses separate address spaces for kernel and user > > space. So basically two memory areas, one in each space, can have the > > same address. > > Is this possible with FreeBSD 12.0? Is this likely to happen? > > If the userspace program and the kernel address happen to overlap, the > system will deal with it. There's not anything to worry about. As to > whether or not it's likely to happen -- I'm not sure about that. I > expect the default stack and heap space locations for a fresh process > have changed due to this change, but it should not matter. 4/4 does potentially alter the failure modes of buggy code that tries to read directly from userspace addresses. For example, correct calls to the sysctls fixed in r342125 may panic prior to the fix because the addresses in question aren't mapped in kernel space. They might also fail or behave bizarrely if the page is mapped and the value from the kernel page is used. -- Brooks signature.asc Description: PGP signature
Re: Address Collision using i386 4G/4G Memory Split
On Mon, Dec 17, 2018 at 02:51:48PM +0100, Alexander Lochmann wrote:
> Hi folks!
>
> According to git commit e3089a (https://reviews.freebsd.org/D1463)
> FreeBSD 12.0 i386 uses separate address spaces for kernel and user
> space. So basically two memory areas, one in each space, can have the
> same address.
> Is this possible with FreeBSD 12.0? Is this likely to happen?
The feature was added to HEAD during this summer, before stable/12 was
branched.
>
> On my opinion, this is also very expensive in terms of performance.
> Any copy{in,out} has to flush the TLB.
> (http://fxr.watson.org/fxr/source/i386/i386/copyout_fast.s#L91)
> Why are you still using this 4G/4G approach?
Because it is needed for i386 to self-host, in modern world 1G KVA
is too small, and because it provides Meltdown mitigation.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Address Collision using i386 4G/4G Memory Split
Alexander Lochmann writes:
According to git commit e3089a (https://reviews.freebsd.org/D1463)
FreeBSD 12.0 i386 uses separate address spaces for kernel and user
space. So basically two memory areas, one in each space, can have the
same address.
Is this possible with FreeBSD 12.0? Is this likely to happen?
If the userspace program and the kernel address happen to overlap, the
system will deal with it. There's not anything to worry about. As to
whether or not it's likely to happen -- I'm not sure about that. I
expect the default stack and heap space locations for a fresh process
have changed due to this change, but it should not matter.
On my opinion, this is also very expensive in terms of performance.
Any copy{in,out} has to flush the TLB.
(http://fxr.watson.org/fxr/source/i386/i386/copyout_fast.s#L91)
Why are you still using this 4G/4G approach?
The complete split between the user address space and kernel address
space mapping is largely due to the mitigation of the Spectre attacks,
as I understand things. To have both the kernel and userspace mapped
at the same time, can be used to extract information from the kernel
that should not be made available.
I think it falls into the "slower but safer" class of change.
Someone will, undoubtedly, correct me if I'm wrong.
-Kurt
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Address Collision using i386 4G/4G Memory Split
Hi folks!
According to git commit e3089a (https://reviews.freebsd.org/D1463)
FreeBSD 12.0 i386 uses separate address spaces for kernel and user
space. So basically two memory areas, one in each space, can have the
same address.
Is this possible with FreeBSD 12.0? Is this likely to happen?
On my opinion, this is also very expensive in terms of performance.
Any copy{in,out} has to flush the TLB.
(http://fxr.watson.org/fxr/source/i386/i386/copyout_fast.s#L91)
Why are you still using this 4G/4G approach?
Regards,
Alex
--
Technische Universität Dortmund
Alexander LochmannPGP key: 0xBC3EF6FD
Otto-Hahn-Str. 16 phone: +49.231.7556141
D-44227 Dortmund fax:+49.231.7556116
http://ess.cs.tu-dortmund.de/Staff/al
signature.asc
Description: OpenPGP digital signature
