[tip:x86/pti] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

2018-01-14 Thread tip-bot for Tom Lendacky 
Commit-ID:  28d437d550e1e39f805d99f9f8ac399c778827b7
Gitweb: https://git.kernel.org/tip/28d437d550e1e39f805d99f9f8ac399c778827b7
Author: Tom Lendacky 
AuthorDate: Sat, 13 Jan 2018 17:27:30 -0600
Committer:  Thomas Gleixner 
CommitDate: Mon, 15 Jan 2018 00:32:55 +0100

x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

The PAUSE instruction is currently used in the retpoline and RSB filling
macros as a speculation trap.  The use of PAUSE was originally suggested
because it showed a very, very small difference in the amount of
cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
the PAUSE instruction is not a serializing instruction, so the pause/jmp
loop will use excess power as it is speculated over waiting for return
to mispredict to the correct target.

The RSB filling macro is applicable to AMD, and, if software is unable to
verify that LFENCE is serializing on AMD (possible when running under a
hypervisor), the generic retpoline support will be used and, so, is also
applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
LFENCE instruction to the speculation trap for AMD.

The same sequence has been adopted by GCC for the GCC generated retpolines.

Signed-off-by: Tom Lendacky 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Borislav Petkov 
Acked-by: David Woodhouse 
Acked-by: Arjan van de Ven 
Cc: Rik van Riel 
Cc: Andi Kleen 
Cc: Paul Turner 
Cc: Peter Zijlstra 
Cc: Tim Chen 
Cc: Jiri Kosina 
Cc: Dave Hansen 
Cc: Andy Lutomirski 
Cc: Josh Poimboeuf 
Cc: Dan Williams 
Cc: Linus Torvalds 
Cc: Greg Kroah-Hartman 
Cc: Kees Cook 
Link: 
https://lkml.kernel.org/r/20180113232730.31060.36287.st...@tlendack-t1.amdoffice.net

---
 arch/x86/include/asm/nospec-branch.h | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/nospec-branch.h 
b/arch/x86/include/asm/nospec-branch.h
index 402a11c..7b45d84 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -11,7 +11,7 @@
  * Fill the CPU return stack buffer.
  *
  * Each entry in the RSB, if used for a speculative 'ret', contains an
- * infinite 'pause; jmp' loop to capture speculative execution.
+ * infinite 'pause; lfence; jmp' loop to capture speculative execution.
  *
  * This is required in various cases for retpoline and IBRS-based
  * mitigations for the Spectre variant 2 vulnerability. Sometimes to
@@ -38,11 +38,13 @@
call772f;   \
 773:   /* speculation trap */  \
pause;  \
+   lfence; \
jmp 773b;   \
 772:   \
call774f;   \
 775:   /* speculation trap */  \
pause;  \
+   lfence; \
jmp 775b;   \
 774:   \
dec reg;\
@@ -73,6 +75,7 @@
call.Ldo_rop_\@
 .Lspec_trap_\@:
pause
+   lfence
jmp .Lspec_trap_\@
 .Ldo_rop_\@:
mov \reg, (%_ASM_SP)
@@ -165,6 +168,7 @@
"   .align 16\n"\
"901:   call   903f;\n" \
"902:   pause;\n"   \
+   "   lfence;\n"  \
"   jmp902b;\n" \
"   .align 16\n"\
"903:   addl   $4, %%esp;\n"\

[tip:x86/pti] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

2018-01-14 Thread tip-bot for Tom Lendacky 
Commit-ID:  28d437d550e1e39f805d99f9f8ac399c778827b7
Gitweb: https://git.kernel.org/tip/28d437d550e1e39f805d99f9f8ac399c778827b7
Author: Tom Lendacky 
AuthorDate: Sat, 13 Jan 2018 17:27:30 -0600
Committer:  Thomas Gleixner 
CommitDate: Mon, 15 Jan 2018 00:32:55 +0100

x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

The PAUSE instruction is currently used in the retpoline and RSB filling
macros as a speculation trap.  The use of PAUSE was originally suggested
because it showed a very, very small difference in the amount of
cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
the PAUSE instruction is not a serializing instruction, so the pause/jmp
loop will use excess power as it is speculated over waiting for return
to mispredict to the correct target.

The RSB filling macro is applicable to AMD, and, if software is unable to
verify that LFENCE is serializing on AMD (possible when running under a
hypervisor), the generic retpoline support will be used and, so, is also
applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
LFENCE instruction to the speculation trap for AMD.

The same sequence has been adopted by GCC for the GCC generated retpolines.

Signed-off-by: Tom Lendacky 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Borislav Petkov 
Acked-by: David Woodhouse 
Acked-by: Arjan van de Ven 
Cc: Rik van Riel 
Cc: Andi Kleen 
Cc: Paul Turner 
Cc: Peter Zijlstra 
Cc: Tim Chen 
Cc: Jiri Kosina 
Cc: Dave Hansen 
Cc: Andy Lutomirski 
Cc: Josh Poimboeuf 
Cc: Dan Williams 
Cc: Linus Torvalds 
Cc: Greg Kroah-Hartman 
Cc: Kees Cook 
Link: 
https://lkml.kernel.org/r/20180113232730.31060.36287.st...@tlendack-t1.amdoffice.net

---
 arch/x86/include/asm/nospec-branch.h | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/nospec-branch.h 
b/arch/x86/include/asm/nospec-branch.h
index 402a11c..7b45d84 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -11,7 +11,7 @@
  * Fill the CPU return stack buffer.
  *
  * Each entry in the RSB, if used for a speculative 'ret', contains an
- * infinite 'pause; jmp' loop to capture speculative execution.
+ * infinite 'pause; lfence; jmp' loop to capture speculative execution.
  *
  * This is required in various cases for retpoline and IBRS-based
  * mitigations for the Spectre variant 2 vulnerability. Sometimes to
@@ -38,11 +38,13 @@
call772f;   \
 773:   /* speculation trap */  \
pause;  \
+   lfence; \
jmp 773b;   \
 772:   \
call774f;   \
 775:   /* speculation trap */  \
pause;  \
+   lfence; \
jmp 775b;   \
 774:   \
dec reg;\
@@ -73,6 +75,7 @@
call.Ldo_rop_\@
 .Lspec_trap_\@:
pause
+   lfence
jmp .Lspec_trap_\@
 .Ldo_rop_\@:
mov \reg, (%_ASM_SP)
@@ -165,6 +168,7 @@
"   .align 16\n"\
"901:   call   903f;\n" \
"902:   pause;\n"   \
+   "   lfence;\n"  \
"   jmp902b;\n" \
"   .align 16\n"\
"903:   addl   $4, %%esp;\n"\

[tip:x86/pti] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

2018-01-14 Thread tip-bot for Tom Lendacky 
Commit-ID:  607d93a2658a0075047e87e47fde5bcc626a1918
Gitweb: https://git.kernel.org/tip/607d93a2658a0075047e87e47fde5bcc626a1918
Author: Tom Lendacky 
AuthorDate: Sat, 13 Jan 2018 17:27:30 -0600
Committer:  Thomas Gleixner 
CommitDate: Sun, 14 Jan 2018 18:22:15 +0100

x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

The PAUSE instruction is currently used in the retpoline and RSB filling
macros as a speculation trap.  The use of PAUSE was originally suggested
because it showed a very, very small difference in the amount of
cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
the PAUSE instruction is not a serializing instruction, so the pause/jmp
loop will use excess power as it is speculated over waiting for return
to mispredict to the correct target.

The RSB filling macro is applicable to AMD, and, if software is unable to
verify that LFENCE is serializing on AMD (possible when running under a
hypervisor), the generic retpoline support will be used and, so, is also
applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
LFENCE instruction to the speculation trap for AMD.

The same sequence has been adopted by GCC for the GCC generated retpolines.

Signed-off-by: Tom Lendacky 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Borislav Petkov 
Acked-by: David Woodhouse 
Acked-by: Arjan van de Ven 
Cc: Rik van Riel 
Cc: Andi Kleen 
Cc: Paul Turner 
Cc: Peter Zijlstra 
Cc: Tim Chen 
Cc: Jiri Kosina 
Cc: Dave Hansen 
Cc: Andy Lutomirski 
Cc: Josh Poimboeuf 
Cc: Dan Williams 
Cc: Linus Torvalds 
Cc: Greg Kroah-Hartman 
Cc: Kees Cook 
Link: 
https://lkml.kernel.org/r/20180113232730.31060.36287.st...@tlendack-t1.amdoffice.net

---
 arch/x86/include/asm/nospec-branch.h | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/nospec-branch.h 
b/arch/x86/include/asm/nospec-branch.h
index 402a11c..7b45d84 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -11,7 +11,7 @@
  * Fill the CPU return stack buffer.
  *
  * Each entry in the RSB, if used for a speculative 'ret', contains an
- * infinite 'pause; jmp' loop to capture speculative execution.
+ * infinite 'pause; lfence; jmp' loop to capture speculative execution.
  *
  * This is required in various cases for retpoline and IBRS-based
  * mitigations for the Spectre variant 2 vulnerability. Sometimes to
@@ -38,11 +38,13 @@
call772f;   \
 773:   /* speculation trap */  \
pause;  \
+   lfence; \
jmp 773b;   \
 772:   \
call774f;   \
 775:   /* speculation trap */  \
pause;  \
+   lfence; \
jmp 775b;   \
 774:   \
dec reg;\
@@ -73,6 +75,7 @@
call.Ldo_rop_\@
 .Lspec_trap_\@:
pause
+   lfence
jmp .Lspec_trap_\@
 .Ldo_rop_\@:
mov \reg, (%_ASM_SP)
@@ -165,6 +168,7 @@
"   .align 16\n"\
"901:   call   903f;\n" \
"902:   pause;\n"   \
+   "   lfence;\n"  \
"   jmp902b;\n" \
"   .align 16\n"\
"903:   addl   $4, %%esp;\n"\

[tip:x86/pti] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

2018-01-14 Thread tip-bot for Tom Lendacky 
Commit-ID:  607d93a2658a0075047e87e47fde5bcc626a1918
Gitweb: https://git.kernel.org/tip/607d93a2658a0075047e87e47fde5bcc626a1918
Author: Tom Lendacky 
AuthorDate: Sat, 13 Jan 2018 17:27:30 -0600
Committer:  Thomas Gleixner 
CommitDate: Sun, 14 Jan 2018 18:22:15 +0100

x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

The PAUSE instruction is currently used in the retpoline and RSB filling
macros as a speculation trap.  The use of PAUSE was originally suggested
because it showed a very, very small difference in the amount of
cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
the PAUSE instruction is not a serializing instruction, so the pause/jmp
loop will use excess power as it is speculated over waiting for return
to mispredict to the correct target.

The RSB filling macro is applicable to AMD, and, if software is unable to
verify that LFENCE is serializing on AMD (possible when running under a
hypervisor), the generic retpoline support will be used and, so, is also
applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
LFENCE instruction to the speculation trap for AMD.

The same sequence has been adopted by GCC for the GCC generated retpolines.

Signed-off-by: Tom Lendacky 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Borislav Petkov 
Acked-by: David Woodhouse 
Acked-by: Arjan van de Ven 
Cc: Rik van Riel 
Cc: Andi Kleen 
Cc: Paul Turner 
Cc: Peter Zijlstra 
Cc: Tim Chen 
Cc: Jiri Kosina 
Cc: Dave Hansen 
Cc: Andy Lutomirski 
Cc: Josh Poimboeuf 
Cc: Dan Williams 
Cc: Linus Torvalds 
Cc: Greg Kroah-Hartman 
Cc: Kees Cook 
Link: 
https://lkml.kernel.org/r/20180113232730.31060.36287.st...@tlendack-t1.amdoffice.net

---
 arch/x86/include/asm/nospec-branch.h | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/nospec-branch.h 
b/arch/x86/include/asm/nospec-branch.h
index 402a11c..7b45d84 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -11,7 +11,7 @@
  * Fill the CPU return stack buffer.
  *
  * Each entry in the RSB, if used for a speculative 'ret', contains an
- * infinite 'pause; jmp' loop to capture speculative execution.
+ * infinite 'pause; lfence; jmp' loop to capture speculative execution.
  *
  * This is required in various cases for retpoline and IBRS-based
  * mitigations for the Spectre variant 2 vulnerability. Sometimes to
@@ -38,11 +38,13 @@
call772f;   \
 773:   /* speculation trap */  \
pause;  \
+   lfence; \
jmp 773b;   \
 772:   \
call774f;   \
 775:   /* speculation trap */  \
pause;  \
+   lfence; \
jmp 775b;   \
 774:   \
dec reg;\
@@ -73,6 +75,7 @@
call.Ldo_rop_\@
 .Lspec_trap_\@:
pause
+   lfence
jmp .Lspec_trap_\@
 .Ldo_rop_\@:
mov \reg, (%_ASM_SP)
@@ -165,6 +168,7 @@
"   .align 16\n"\
"901:   call   903f;\n" \
"902:   pause;\n"   \
+   "   lfence;\n"  \
"   jmp902b;\n" \
"   .align 16\n"\
"903:   addl   $4, %%esp;\n"\