Re: [PATCH 3/5] x86: Add -mfunction-return=

[H.J. Lu]

On Thu, Jan 11, 2018 at 2:54 PM, Jeff Law  wrote:
> On 01/07/2018 03:59 PM, H.J. Lu wrote:
>> Add -mfunction-return= option to convert function return to call and
>> return thunks.  The default is 'keep', which keeps function return
>> unmodified.  'thunk' converts function return to call and return thunk.
>> 'thunk-inline' converts function return to inlined call and return thunk.  
>> 'thunk-extern' converts function return to external call and return
>> thunk provided in a separate object file.  You can control this behavior
>> for a specific function by using the function attribute function_return.
>>
>> Function return thunk is the same as memory thunk for -mindirect-branch=
>> where the return address is at the top of the stack:
>>
>> __x86_return_thunk:
>>   call L2
>> L1:
>>   lfence
>>   jmp L1
>> L2:
>>   lea 8(%rsp), %rsp|lea 4(%esp), %esp
>>   ret
>>
>> and function return becomes
>>
>>   jmp __x86_return_thunk
>>
>> -mindirect-branch= tests are updated with -mfunction-return=keep to
>> avoid false test failures when -mfunction-return=lfence is added to
>> RUNTESTFLAGS for "make check".
>>
>> gcc/
>>
>>   * config/i386/i386-protos.h (ix86_output_function_return): New.
>>   * config/i386/i386.c (ix86_set_indirect_branch_type): Also
>>   set function_return_type.
>>   (indirect_thunk_name): Add ret_p to indicate thunk for function
>>   return.
>>   (output_indirect_thunk_function): Pass false to
>>   indirect_thunk_name.
>>   (ix86_output_indirect_branch): Likewise.
>>   (output_indirect_thunk_function): Create alias for function
>>   return thunk if regno < 0.
>>   (ix86_output_function_return): New function.
>>   (ix86_handle_fndecl_attribute): Handle function_return.
>>   (ix86_attribute_table): Add function_return.
>>   * config/i386/i386.h (machine_function): Add
>>   function_return_type.
>>   * config/i386/i386.md (simple_return_internal): Use
>>   ix86_output_function_return.
>>   (simple_return_internal_long): Likewise.
>>   * config/i386/i386.opt (mfunction-return=): New option.
>>   (indirect_branch): Mention -mfunction-return=.
>>   * doc/extend.texi: Document function_return function attribute.
>>   * doc/invoke.texi: Document -mfunction-return= option.
>>
>> gcc/testsuite/
>>
>>   * gcc.target/i386/indirect-thunk-1.c (dg-options): Add
>>   -mfunction-return=keep.
>>   * gcc.target/i386/indirect-thunk-2.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-3.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-4.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-5.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-6.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-7.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-attr-1.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-attr-2.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-attr-3.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-attr-4.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-attr-5.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-attr-6.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-attr-7.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-attr-8.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-bnd-1.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-bnd-2.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-bnd-3.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-bnd-4.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-extern-1.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-extern-2.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-extern-3.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-extern-4.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-extern-5.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-extern-6.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-extern-7.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-inline-1.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-inline-2.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-inline-3.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-inline-4.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-inline-5.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-inline-6.c: Likewise.
>>   * gcc.target/i386/indirect-thunk-inline-7.c: Likewise.
>>   * gcc.target/i386/ret-thunk-1.c: New test.
>>   * gcc.target/i386/ret-thunk-10.c: Likewise.
>>   * gcc.target/i386/ret-thunk-11.c: Likewise.
>>   * gcc.target/i386/ret-thunk-12.c: Likewise.
>>   * gcc.target/i386/ret-thunk-13.c: Likewise.
>>   * gcc.target/i386/ret-thunk-14.c: Likewise.
>>   * gcc.target/i386/ret-thunk-15.c: Likewise.
>>   * gcc.target/i386/ret-thunk-16.c: Likewise.
>>   * gcc.target/i386/ret-thunk-2.c: Likewise.
>>   * gcc.target/i386/ret-thunk-3.c: Likewise.
>>   * 

Re: [PATCH 3/5] x86: Add -mfunction-return=

[Jeff Law]

On 01/07/2018 03:59 PM, H.J. Lu wrote:
> Add -mfunction-return= option to convert function return to call and
> return thunks.  The default is 'keep', which keeps function return
> unmodified.  'thunk' converts function return to call and return thunk.
> 'thunk-inline' converts function return to inlined call and return thunk.  
> 'thunk-extern' converts function return to external call and return
> thunk provided in a separate object file.  You can control this behavior
> for a specific function by using the function attribute function_return.
> 
> Function return thunk is the same as memory thunk for -mindirect-branch=
> where the return address is at the top of the stack:
> 
> __x86_return_thunk:
>   call L2
> L1:
>   lfence
>   jmp L1
> L2:
>   lea 8(%rsp), %rsp|lea 4(%esp), %esp
>   ret
> 
> and function return becomes
> 
>   jmp __x86_return_thunk
> 
> -mindirect-branch= tests are updated with -mfunction-return=keep to
> avoid false test failures when -mfunction-return=lfence is added to
> RUNTESTFLAGS for "make check".
> 
> gcc/
> 
>   * config/i386/i386-protos.h (ix86_output_function_return): New.
>   * config/i386/i386.c (ix86_set_indirect_branch_type): Also
>   set function_return_type.
>   (indirect_thunk_name): Add ret_p to indicate thunk for function
>   return.
>   (output_indirect_thunk_function): Pass false to
>   indirect_thunk_name.
>   (ix86_output_indirect_branch): Likewise.
>   (output_indirect_thunk_function): Create alias for function
>   return thunk if regno < 0.
>   (ix86_output_function_return): New function.
>   (ix86_handle_fndecl_attribute): Handle function_return.
>   (ix86_attribute_table): Add function_return.
>   * config/i386/i386.h (machine_function): Add
>   function_return_type.
>   * config/i386/i386.md (simple_return_internal): Use
>   ix86_output_function_return.
>   (simple_return_internal_long): Likewise.
>   * config/i386/i386.opt (mfunction-return=): New option.
>   (indirect_branch): Mention -mfunction-return=.
>   * doc/extend.texi: Document function_return function attribute.
>   * doc/invoke.texi: Document -mfunction-return= option.
> 
> gcc/testsuite/
> 
>   * gcc.target/i386/indirect-thunk-1.c (dg-options): Add
>   -mfunction-return=keep.
>   * gcc.target/i386/indirect-thunk-2.c: Likewise.
>   * gcc.target/i386/indirect-thunk-3.c: Likewise.
>   * gcc.target/i386/indirect-thunk-4.c: Likewise.
>   * gcc.target/i386/indirect-thunk-5.c: Likewise.
>   * gcc.target/i386/indirect-thunk-6.c: Likewise.
>   * gcc.target/i386/indirect-thunk-7.c: Likewise.
>   * gcc.target/i386/indirect-thunk-attr-1.c: Likewise.
>   * gcc.target/i386/indirect-thunk-attr-2.c: Likewise.
>   * gcc.target/i386/indirect-thunk-attr-3.c: Likewise.
>   * gcc.target/i386/indirect-thunk-attr-4.c: Likewise.
>   * gcc.target/i386/indirect-thunk-attr-5.c: Likewise.
>   * gcc.target/i386/indirect-thunk-attr-6.c: Likewise.
>   * gcc.target/i386/indirect-thunk-attr-7.c: Likewise.
>   * gcc.target/i386/indirect-thunk-attr-8.c: Likewise.
>   * gcc.target/i386/indirect-thunk-bnd-1.c: Likewise.
>   * gcc.target/i386/indirect-thunk-bnd-2.c: Likewise.
>   * gcc.target/i386/indirect-thunk-bnd-3.c: Likewise.
>   * gcc.target/i386/indirect-thunk-bnd-4.c: Likewise.
>   * gcc.target/i386/indirect-thunk-extern-1.c: Likewise.
>   * gcc.target/i386/indirect-thunk-extern-2.c: Likewise.
>   * gcc.target/i386/indirect-thunk-extern-3.c: Likewise.
>   * gcc.target/i386/indirect-thunk-extern-4.c: Likewise.
>   * gcc.target/i386/indirect-thunk-extern-5.c: Likewise.
>   * gcc.target/i386/indirect-thunk-extern-6.c: Likewise.
>   * gcc.target/i386/indirect-thunk-extern-7.c: Likewise.
>   * gcc.target/i386/indirect-thunk-inline-1.c: Likewise.
>   * gcc.target/i386/indirect-thunk-inline-2.c: Likewise.
>   * gcc.target/i386/indirect-thunk-inline-3.c: Likewise.
>   * gcc.target/i386/indirect-thunk-inline-4.c: Likewise.
>   * gcc.target/i386/indirect-thunk-inline-5.c: Likewise.
>   * gcc.target/i386/indirect-thunk-inline-6.c: Likewise.
>   * gcc.target/i386/indirect-thunk-inline-7.c: Likewise.
>   * gcc.target/i386/ret-thunk-1.c: New test.
>   * gcc.target/i386/ret-thunk-10.c: Likewise.
>   * gcc.target/i386/ret-thunk-11.c: Likewise.
>   * gcc.target/i386/ret-thunk-12.c: Likewise.
>   * gcc.target/i386/ret-thunk-13.c: Likewise.
>   * gcc.target/i386/ret-thunk-14.c: Likewise.
>   * gcc.target/i386/ret-thunk-15.c: Likewise.
>   * gcc.target/i386/ret-thunk-16.c: Likewise.
>   * gcc.target/i386/ret-thunk-2.c: Likewise.
>   * gcc.target/i386/ret-thunk-3.c: Likewise.
>   * gcc.target/i386/ret-thunk-4.c: Likewise.
>   * gcc.target/i386/ret-thunk-5.c: Likewise.
>   * gcc.target/i386/ret-thunk-6.c: Likewise.
>   * 

Re: [PATCH 3/5] x86: Add -mfunction-return=

[David Woodhouse]

On Mon, 2018-01-08 at 03:59 -0800, H.J. Lu wrote:
> On Mon, Jan 8, 2018 at 1:56 AM, Martin Liška  wrote:
> > 
> > On 01/07/2018 11:59 PM, H.J. Lu wrote:
> > > 
> > > Function return thunk is the same as memory thunk for -mindirect-branch=
> > > where the return address is at the top of the stack:
> > > 
> > > __x86_return_thunk:
> > >   call L2
> > > L1:
> > >   lfence
> > >   jmp L1
> > > L2:
> > >   lea 8(%rsp), %rsp|lea 4(%esp), %esp
> > >   ret
> > > 
> > > and function return becomes
> > > 
> > >   jmp __x86_return_thunk
> > Hello.
> > 
> > Can you please explain more usage of the option? Is to prevent a speculative
> > execution of 'ret' instruction (which is an indirect call), as described in 
> > [1]?
> > The paper mentions that return stack predictors are commonly implemented in 
> > some form.
> > Looks that current version of Linux patches does not use the option.
> > 
> This option is requested by Linux kernel people.  It may be used in
> the future.

Right. Essentially the new set of vulnerability are all about
speculative execution. Instructions which *don't* get committed, and
it's supposed to be like they never happen, actually *do* have side-
effects and can leak information.

This is *particularly* problematic for Intel CPUs where the CPU
architects said "ah, screw it, let's not do memory permission checks in
advance; as long as we make sure it's done before we *commit* an
instruction it'll be fine". With the result that you can now basically
read *all* of kernel memory, and hence all of physical memory, directly
from userspace on Intel CPUs. Oops :)

The fix for *that* one is to actually remove the kernel pages from the
page tables while running userspace, instead of just setting the
permissions to prevent access. Hence the whole Kernel Page Table
Isolation thing.

The next interesting attack is the so-called "variant 2" where the
attacker pollutes the branch predictors so that in *kernel* mode the
CPU *speculatively* runs... well, whatever the attacker wants. This is
one that affects lots of vendors, not just Intel. We mitigate this by
eliminating *all* the indirect branches in the kernel, to make it
immune to such an attack.

This is all very well, but *some* CPUs also pull in predictions from
the generic branch target predictor when the return stack buffer has
underflowed (e.g. if there was a call stack of more than X depth).
Hence, in some cases we may yet end up needing this -mfunction-return=
thunk too. As you (Martin) note, we don't use it *yet*. The full set of
mitigations for the various attacks are still being put together, and
the optimal choice for each CPU family does end up being different.

smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 3/5] x86: Add -mfunction-return=

[H.J. Lu]

On Mon, Jan 8, 2018 at 1:56 AM, Martin Liška  wrote:
> On 01/07/2018 11:59 PM, H.J. Lu wrote:
>> Function return thunk is the same as memory thunk for -mindirect-branch=
>> where the return address is at the top of the stack:
>>
>> __x86_return_thunk:
>>   call L2
>> L1:
>>   lfence
>>   jmp L1
>> L2:
>>   lea 8(%rsp), %rsp|lea 4(%esp), %esp
>>   ret
>>
>> and function return becomes
>>
>>   jmp __x86_return_thunk
>
> Hello.
>
> Can you please explain more usage of the option? Is to prevent a speculative
> execution of 'ret' instruction (which is an indirect call), as described in 
> [1]?
> The paper mentions that return stack predictors are commonly implemented in 
> some form.
> Looks that current version of Linux patches does not use the option.
>

This option is requested by Linux kernel people.  It may be used in
the future.

-- 
H.J.

Re: [PATCH 3/5] x86: Add -mfunction-return=

[Martin Liška]

On 01/07/2018 11:59 PM, H.J. Lu wrote:
> Function return thunk is the same as memory thunk for -mindirect-branch=
> where the return address is at the top of the stack:
> 
> __x86_return_thunk:
>   call L2
> L1:
>   lfence
>   jmp L1
> L2:
>   lea 8(%rsp), %rsp|lea 4(%esp), %esp
>   ret
> 
> and function return becomes
> 
>   jmp __x86_return_thunk

Hello.

Can you please explain more usage of the option? Is to prevent a speculative
execution of 'ret' instruction (which is an indirect call), as described in [1]?
The paper mentions that return stack predictors are commonly implemented in 
some form.
Looks that current version of Linux patches does not use the option.

Thanks,
Martin

[1] https://support.google.com/faqs/answer/7625886