Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 17:42 +0100, Andrea Arcangeli wrote:
> On Fri, Jan 05, 2018 at 04:37:30PM +, David Woodhouse wrote:
> > You are completely ignoring pre-Skylake here.
> > 
> > On pre-Skylake, retpoline is perfectly sufficient and it's a *lot*
> > faster than the IBRS option which is almost prohibitively slow.
> > 
> > We didn't do it just for fun. And it's working fine; it isn't *that*
> > complex.
> 
> How do you enable IBRS when the CPU switches to SMM?

SMM is fine, as Arjan said. It's only for stuff like EFI runtime calls,
and then only if you're really paranoid.

> Do you already have this 2-way code emission from gcc and patching
> with a 3-way alternatives at boot between ibrs and 2 reptoline version
> emitted by gcc and alternatives between ibrs and ibpb where SPEC_CTRL
> is missing on some CPU but IBPB_SUPPORT is available?

This was implemented in Intel's patch sets that they've been sending
out. I don't really know why we've suddenly gone back to the drawing
board and turned things around to put retpoline first in the series,
etc.

I'm also mildly concerned that all the variant 1 patches have just
disappeared.

> Or are you talking about having done this on a non upstream Xen build
> only without the 2-way code emission for gcc?

Xen has it too, but no. I was talking about Linux.

smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 17:42 +0100, Andrea Arcangeli wrote:
> On Fri, Jan 05, 2018 at 04:37:30PM +, David Woodhouse wrote:
> > You are completely ignoring pre-Skylake here.
> > 
> > On pre-Skylake, retpoline is perfectly sufficient and it's a *lot*
> > faster than the IBRS option which is almost prohibitively slow.
> > 
> > We didn't do it just for fun. And it's working fine; it isn't *that*
> > complex.
> 
> How do you enable IBRS when the CPU switches to SMM?

SMM is fine, as Arjan said. It's only for stuff like EFI runtime calls,
and then only if you're really paranoid.

> Do you already have this 2-way code emission from gcc and patching
> with a 3-way alternatives at boot between ibrs and 2 reptoline version
> emitted by gcc and alternatives between ibrs and ibpb where SPEC_CTRL
> is missing on some CPU but IBPB_SUPPORT is available?

This was implemented in Intel's patch sets that they've been sending
out. I don't really know why we've suddenly gone back to the drawing
board and turned things around to put retpoline first in the series,
etc.

I'm also mildly concerned that all the variant 1 patches have just
disappeared.

> Or are you talking about having done this on a non upstream Xen build
> only without the 2-way code emission for gcc?

Xen has it too, but no. I was talking about Linux.

smime.p7s
Description: S/MIME cryptographic signature

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> On Fri, Jan 05, 2018 at 04:37:30PM +, David Woodhouse wrote:
> > You are completely ignoring pre-Skylake here.
> >
> > On pre-Skylake, retpoline is perfectly sufficient and it's a *lot*
> > faster than the IBRS option which is almost prohibitively slow.
> >
> > We didn't do it just for fun. And it's working fine; it isn't *that*
> > complex.
> 
> How do you enable IBRS when the CPU switches to SMM?

you do not need to. SMM takes care of itself.

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> On Fri, Jan 05, 2018 at 04:37:30PM +, David Woodhouse wrote:
> > You are completely ignoring pre-Skylake here.
> >
> > On pre-Skylake, retpoline is perfectly sufficient and it's a *lot*
> > faster than the IBRS option which is almost prohibitively slow.
> >
> > We didn't do it just for fun. And it's working fine; it isn't *that*
> > complex.
> 
> How do you enable IBRS when the CPU switches to SMM?

you do not need to. SMM takes care of itself.

Re: [PATCH 0/7] IBRS patch series

[Andrea Arcangeli]

On Fri, Jan 05, 2018 at 04:37:30PM +, David Woodhouse wrote:
> You are completely ignoring pre-Skylake here.
> 
> On pre-Skylake, retpoline is perfectly sufficient and it's a *lot*
> faster than the IBRS option which is almost prohibitively slow.
> 
> We didn't do it just for fun. And it's working fine; it isn't *that*
> complex.

How do you enable IBRS when the CPU switches to SMM?

Do you already have this 2-way code emission from gcc and patching
with a 3-way alternatives at boot between ibrs and 2 reptoline version
emitted by gcc and alternatives between ibrs and ibpb where SPEC_CTRL
is missing on some CPU but IBPB_SUPPORT is available?

Or are you talking about having done this on a non upstream Xen build
only without the 2-way code emission for gcc?

Re: [PATCH 0/7] IBRS patch series

[Andrea Arcangeli]

On Fri, Jan 05, 2018 at 04:37:30PM +, David Woodhouse wrote:
> You are completely ignoring pre-Skylake here.
> 
> On pre-Skylake, retpoline is perfectly sufficient and it's a *lot*
> faster than the IBRS option which is almost prohibitively slow.
> 
> We didn't do it just for fun. And it's working fine; it isn't *that*
> complex.

How do you enable IBRS when the CPU switches to SMM?

Do you already have this 2-way code emission from gcc and patching
with a 3-way alternatives at boot between ibrs and 2 reptoline version
emitted by gcc and alternatives between ibrs and ibpb where SPEC_CTRL
is missing on some CPU but IBPB_SUPPORT is available?

Or are you talking about having done this on a non upstream Xen build
only without the 2-way code emission for gcc?

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 17:05 +0100, Andrea Arcangeli wrote:
> On Fri, Jan 05, 2018 at 03:38:24PM +, David Woodhouse wrote:
> > 
> > We had IBRS first, and especially on Broadwell and earlier, its
> > performance really is painful.
> > 
> > Then came retpoline, purely as an optimisation. A very *important*
> > performance improvement, but an optimisation nonetheless.
> > 
> > When looking at optimisations, it is rare for us to say "oh, well it
> > opens up only a *small* theoretical security hole, but it's faster so
> > that's OK".
> I couldn't express any better than the above, the way I look at
> repotlines.
> 
> Now seeing how this thing escalates with 2-way gcc code emission with
> amd version that will not have ibrs at all to call it around the bios
> etc... and IBRS skylake is still needed as a 3-way alternative and no
> code exists to even patch it all at boot like that, and then qemu has
> to be compiled with reptoline and 2-way alternative there too, to
> achieve ibrs 2 ibpb 1, it's not looking like the way to go to me.
> 
> Not in the short term at least, and for the long term "reptoline" is
> guaranteed a totally useless effort.
> 
> reptoline is like highmem kmap() etc... eventually nobody will ever
> use reptoline. reptoline is more interesting for downstream in fact if
> something where at least we won't have to deal with that forever where
> there's more control on the toolchain used, and after a certain number
> of years that code expires.
> 
> I can imagine we'll unfortunately have to deal with reptoline at some
> point, but starting with reptoline at least looks backwards,
> especially if it's the long term you're planning for.

You are completely ignoring pre-Skylake here.

On pre-Skylake, retpoline is perfectly sufficient and it's a *lot*
faster than the IBRS option which is almost prohibitively slow.

We didn't do it just for fun. And it's working fine; it isn't *that*
complex.


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 17:05 +0100, Andrea Arcangeli wrote:
> On Fri, Jan 05, 2018 at 03:38:24PM +, David Woodhouse wrote:
> > 
> > We had IBRS first, and especially on Broadwell and earlier, its
> > performance really is painful.
> > 
> > Then came retpoline, purely as an optimisation. A very *important*
> > performance improvement, but an optimisation nonetheless.
> > 
> > When looking at optimisations, it is rare for us to say "oh, well it
> > opens up only a *small* theoretical security hole, but it's faster so
> > that's OK".
> I couldn't express any better than the above, the way I look at
> repotlines.
> 
> Now seeing how this thing escalates with 2-way gcc code emission with
> amd version that will not have ibrs at all to call it around the bios
> etc... and IBRS skylake is still needed as a 3-way alternative and no
> code exists to even patch it all at boot like that, and then qemu has
> to be compiled with reptoline and 2-way alternative there too, to
> achieve ibrs 2 ibpb 1, it's not looking like the way to go to me.
> 
> Not in the short term at least, and for the long term "reptoline" is
> guaranteed a totally useless effort.
> 
> reptoline is like highmem kmap() etc... eventually nobody will ever
> use reptoline. reptoline is more interesting for downstream in fact if
> something where at least we won't have to deal with that forever where
> there's more control on the toolchain used, and after a certain number
> of years that code expires.
> 
> I can imagine we'll unfortunately have to deal with reptoline at some
> point, but starting with reptoline at least looks backwards,
> especially if it's the long term you're planning for.

You are completely ignoring pre-Skylake here.

On pre-Skylake, retpoline is perfectly sufficient and it's a *lot*
faster than the IBRS option which is almost prohibitively slow.

We didn't do it just for fun. And it's working fine; it isn't *that*
complex.


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[Andrea Arcangeli]

On Fri, Jan 05, 2018 at 03:38:24PM +, David Woodhouse wrote:
> We had IBRS first, and especially on Broadwell and earlier, its
> performance really is painful.
> 
> Then came retpoline, purely as an optimisation. A very *important*
> performance improvement, but an optimisation nonetheless.
>
> When looking at optimisations, it is rare for us to say "oh, well it
> opens up only a *small* theoretical security hole, but it's faster so
> that's OK".

I couldn't express any better than the above, the way I look at
repotlines.

Now seeing how this thing escalates with 2-way gcc code emission with
amd version that will not have ibrs at all to call it around the bios
etc... and IBRS skylake is still needed as a 3-way alternative and no
code exists to even patch it all at boot like that, and then qemu has
to be compiled with reptoline and 2-way alternative there too, to
achieve ibrs 2 ibpb 1, it's not looking like the way to go to me.

Not in the short term at least, and for the long term "reptoline" is
guaranteed a totally useless effort.

reptoline is like highmem kmap() etc... eventually nobody will ever
use reptoline. reptoline is more interesting for downstream in fact if
something where at least we won't have to deal with that forever where
there's more control on the toolchain used, and after a certain number
of years that code expires.

I can imagine we'll unfortunately have to deal with reptoline at some
point, but starting with reptoline at least looks backwards,
especially if it's the long term you're planning for.

Thanks,
Andrea

Re: [PATCH 0/7] IBRS patch series

[Andrea Arcangeli]

On Fri, Jan 05, 2018 at 03:38:24PM +, David Woodhouse wrote:
> We had IBRS first, and especially on Broadwell and earlier, its
> performance really is painful.
> 
> Then came retpoline, purely as an optimisation. A very *important*
> performance improvement, but an optimisation nonetheless.
>
> When looking at optimisations, it is rare for us to say "oh, well it
> opens up only a *small* theoretical security hole, but it's faster so
> that's OK".

I couldn't express any better than the above, the way I look at
repotlines.

Now seeing how this thing escalates with 2-way gcc code emission with
amd version that will not have ibrs at all to call it around the bios
etc... and IBRS skylake is still needed as a 3-way alternative and no
code exists to even patch it all at boot like that, and then qemu has
to be compiled with reptoline and 2-way alternative there too, to
achieve ibrs 2 ibpb 1, it's not looking like the way to go to me.

Not in the short term at least, and for the long term "reptoline" is
guaranteed a totally useless effort.

reptoline is like highmem kmap() etc... eventually nobody will ever
use reptoline. reptoline is more interesting for downstream in fact if
something where at least we won't have to deal with that forever where
there's more control on the toolchain used, and after a certain number
of years that code expires.

I can imagine we'll unfortunately have to deal with reptoline at some
point, but starting with reptoline at least looks backwards,
especially if it's the long term you're planning for.

Thanks,
Andrea

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 14:42 +, Van De Ven, Arjan wrote:

> This is why I said I would like to see retpoline be the default, with
> IBRS an opt-in for the paranoid. I guess David will turn that on ;-)

I can live with that.

It really depends how you look at it.

We had IBRS first, and especially on Broadwell and earlier, its
performance really is painful.

Then came retpoline, purely as an optimisation. A very *important*
performance improvement, but an optimisation nonetheless.

When looking at optimisations, it is rare for us to say "oh, well it
opens up only a *small* theoretical security hole, but it's faster so
that's OK".

Retpoline is sufficient on pre-Skylake. On Skylake... it's complicated,
and that's why my default position has always been that we should
prefer IBRS there. Especially as it isn't *even* as much of a
performance win there.

But you're right, the holes that it opens up when compared to IBRS are
minimal, and it *is* still a performance win (just less so than on
earlier CPUs). So if your recommendation is that we stick with
retpoline until IBRS_ATT comes along, I can live with that. People
always have the option to build without retpoline, and then they get to
use IBRS in the kernel if they want.


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 14:42 +, Van De Ven, Arjan wrote:

> This is why I said I would like to see retpoline be the default, with
> IBRS an opt-in for the paranoid. I guess David will turn that on ;-)

I can live with that.

It really depends how you look at it.

We had IBRS first, and especially on Broadwell and earlier, its
performance really is painful.

Then came retpoline, purely as an optimisation. A very *important*
performance improvement, but an optimisation nonetheless.

When looking at optimisations, it is rare for us to say "oh, well it
opens up only a *small* theoretical security hole, but it's faster so
that's OK".

Retpoline is sufficient on pre-Skylake. On Skylake... it's complicated,
and that's why my default position has always been that we should
prefer IBRS there. Especially as it isn't *even* as much of a
performance win there.

But you're right, the holes that it opens up when compared to IBRS are
minimal, and it *is* still a performance win (just less so than on
earlier CPUs). So if your recommendation is that we stick with
retpoline until IBRS_ATT comes along, I can live with that. People
always have the option to build without retpoline, and then they get to
use IBRS in the kernel if they want.


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[Andrea Arcangeli]

On Fri, Jan 05, 2018 at 02:52:33PM +, Van De Ven, Arjan wrote:
> I'm sorry but your whole statement reeks a little bit of "perfect is the 
> enemy of good"

My point is exactly that this sentences could apply to spectre
variant#2 in the first place..

If we start moving in any direction, either we reach "perfection" or
moving towards perfection and still being in "perfect is the enemy of
good" territory doesn't sound great to me.

The reptoline status with 3 way " IBRS skylake mode" or " 2way
reptoline code emission from gcc for older CPUs or CPUs without
SPEC_CTRL" plus IBRS forced still to be used around all
firmware/bios/SMM (how do you do IBRS on those CPUS without IBRS, that
requires yet another IBPB alternative combined with AMD reptoline
case) is far from a "less is more" or KISS.

This whole reptoline work is total waste for future silicon so I like
to keep it simple and obviously mathematically safe at the same
time... and also easy to activate for full math safe qemu ibrs 2 mode
without having to do a 2-way reptoline gcc code emission and dynamic
patching on qemu startup.

Re: [PATCH 0/7] IBRS patch series

[Andrea Arcangeli]

On Fri, Jan 05, 2018 at 02:52:33PM +, Van De Ven, Arjan wrote:
> I'm sorry but your whole statement reeks a little bit of "perfect is the 
> enemy of good"

My point is exactly that this sentences could apply to spectre
variant#2 in the first place..

If we start moving in any direction, either we reach "perfection" or
moving towards perfection and still being in "perfect is the enemy of
good" territory doesn't sound great to me.

The reptoline status with 3 way " IBRS skylake mode" or " 2way
reptoline code emission from gcc for older CPUs or CPUs without
SPEC_CTRL" plus IBRS forced still to be used around all
firmware/bios/SMM (how do you do IBRS on those CPUS without IBRS, that
requires yet another IBPB alternative combined with AMD reptoline
case) is far from a "less is more" or KISS.

This whole reptoline work is total waste for future silicon so I like
to keep it simple and obviously mathematically safe at the same
time... and also easy to activate for full math safe qemu ibrs 2 mode
without having to do a 2-way reptoline gcc code emission and dynamic
patching on qemu startup.

Re: [PATCH 0/7] IBRS patch series

[Thomas Gleixner]

On Fri, 5 Jan 2018, Andrea Arcangeli wrote:
> On Thu, Jan 04, 2018 at 09:22:34PM +, Van De Ven, Arjan wrote:
> > personally I am comfortable with retpoline on Skylake, but I would
> >like to have IBRS as an opt in for the paranoid.
> 
> I think this whole variant#2 issue has to be fixed mathematically or
> not at all, the reason is that it's already "near zero risk" to begin
> with.
> 
> If we leave any theoretical issues open we can as well stick with the
> "near zero risk" wording on AMD website and not fix anything.
> 
> Doing a huge amount of work with reptoline and then you find SMM is
> called reproducibly somehow and a new PoC could exist for it, not fun.
> 
> Whatever solution should be total and guaranteed or there's huge
> amount of resources wasted.
> 
> Again spectre variant#2 this is "near zero risk" to begin with, on all
> CPUs because of the complexity to mount an attack on any given
> kernel/CPU combination out there. I'm mostly thinking about the "guest
> mode"/"host kernel mod" isolation here but this should apply in
> general to all kernel protection for spectre variant#2.
> 
> If we can do a 3 way alternative IBRS on skylake, repotline + IBRS
> around all BIOS, graphics firmware and pci firmware + kernel asm
> patched with a 2-way alternative between amd reptoline and intel
> reptoline emitted 2-way by same gcc build, and still guarantee this
> solution is as mathematically safe as a pure IBRS (which is obviously
> mathematically safe), ok. Otherwise it'd prefer to stick to pure IBRS
> and skip the reptoline complexity and fallouts here and there around
> bios asm firmware SMM etc.. not to tell the need to rebuild qemu with
> reptoline with 2-way amd/intel alternatives self modifying code in
> userland during qemu startup to achieve ibrs 1 ibpb 1 + prctl IBRS on
> qemu with repotline.

I agree. The amount of options we get plus the desire to do prctl/cgroup
and whatever based enable/disable makes me nervous as hell. Especially the
per task/process/cgroup runtime magic is bound to dig yet another hole of
speculation attack vector into the existing mess.

Thanks,

tglx

Re: [PATCH 0/7] IBRS patch series

[Thomas Gleixner]

On Fri, 5 Jan 2018, Andrea Arcangeli wrote:
> On Thu, Jan 04, 2018 at 09:22:34PM +, Van De Ven, Arjan wrote:
> > personally I am comfortable with retpoline on Skylake, but I would
> >like to have IBRS as an opt in for the paranoid.
> 
> I think this whole variant#2 issue has to be fixed mathematically or
> not at all, the reason is that it's already "near zero risk" to begin
> with.
> 
> If we leave any theoretical issues open we can as well stick with the
> "near zero risk" wording on AMD website and not fix anything.
> 
> Doing a huge amount of work with reptoline and then you find SMM is
> called reproducibly somehow and a new PoC could exist for it, not fun.
> 
> Whatever solution should be total and guaranteed or there's huge
> amount of resources wasted.
> 
> Again spectre variant#2 this is "near zero risk" to begin with, on all
> CPUs because of the complexity to mount an attack on any given
> kernel/CPU combination out there. I'm mostly thinking about the "guest
> mode"/"host kernel mod" isolation here but this should apply in
> general to all kernel protection for spectre variant#2.
> 
> If we can do a 3 way alternative IBRS on skylake, repotline + IBRS
> around all BIOS, graphics firmware and pci firmware + kernel asm
> patched with a 2-way alternative between amd reptoline and intel
> reptoline emitted 2-way by same gcc build, and still guarantee this
> solution is as mathematically safe as a pure IBRS (which is obviously
> mathematically safe), ok. Otherwise it'd prefer to stick to pure IBRS
> and skip the reptoline complexity and fallouts here and there around
> bios asm firmware SMM etc.. not to tell the need to rebuild qemu with
> reptoline with 2-way amd/intel alternatives self modifying code in
> userland during qemu startup to achieve ibrs 1 ibpb 1 + prctl IBRS on
> qemu with repotline.

I agree. The amount of options we get plus the desire to do prctl/cgroup
and whatever based enable/disable makes me nervous as hell. Especially the
per task/process/cgroup runtime magic is bound to dig yet another hole of
speculation attack vector into the existing mess.

Thanks,

tglx

Re: [PATCH 0/7] IBRS patch series

[Yves-Alexis Perez]

On Fri, 2018-01-05 at 15:26 +0100, Paolo Bonzini wrote:
> Those from November seem way too early to include IBRS/IBPB.  Maybe the
> two from December 3rd, but I wouldn't be 100% sure.

So, for my CPU with updated microcode:

processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model   : 61
model name  : Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz
stepping: 4
microcode   : 0x28

cpuid returns:

   0x0007 0x00: eax=0x ebx=0x021c27ab ecx=0x
edx=0x0c00

So bit 26/27 are set, which as I understand means IBRS is supported (but I
would appreciate any pointer to relevant documentation on this).
> 
> So it would be even nicer to know how those microcode updates were tested.

At least I didn't test IBRS/IBPB here. I could do it provided I'm pointed to a
tree with all the things to test.
> 
> (And by the way, the LFENCE change is for variant 1 aka CVE-2017-5753).

Ok, good to know. Is the kernel support part for LFENCE in the same thread (I
have to admit I'm a bit lost).

Regards,
-- 
Yves-Alexis

signature.asc
Description: This is a digitally signed message part

Re: [PATCH 0/7] IBRS patch series

[Yves-Alexis Perez]

On Fri, 2018-01-05 at 15:26 +0100, Paolo Bonzini wrote:
> Those from November seem way too early to include IBRS/IBPB.  Maybe the
> two from December 3rd, but I wouldn't be 100% sure.

So, for my CPU with updated microcode:

processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model   : 61
model name  : Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz
stepping: 4
microcode   : 0x28

cpuid returns:

   0x0007 0x00: eax=0x ebx=0x021c27ab ecx=0x
edx=0x0c00

So bit 26/27 are set, which as I understand means IBRS is supported (but I
would appreciate any pointer to relevant documentation on this).
> 
> So it would be even nicer to know how those microcode updates were tested.

At least I didn't test IBRS/IBPB here. I could do it provided I'm pointed to a
tree with all the things to test.
> 
> (And by the way, the LFENCE change is for variant 1 aka CVE-2017-5753).

Ok, good to know. Is the kernel support part for LFENCE in the same thread (I
have to admit I'm a bit lost).

Regards,
-- 
Yves-Alexis

signature.asc
Description: This is a digitally signed message part

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> 
> Doing a huge amount of work with reptoline and then you find SMM is
> called reproducibly somehow and a new PoC could exist for it, not fun.

retpoline we want for broadwell and earlier anyway..

I'm sorry but your whole statement reeks a little bit of "perfect is the enemy 
of good"

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> 
> Doing a huge amount of work with reptoline and then you find SMM is
> called reproducibly somehow and a new PoC could exist for it, not fun.

retpoline we want for broadwell and earlier anyway..

I'm sorry but your whole statement reeks a little bit of "perfect is the enemy 
of good"

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> -Original Message-
> From: Paolo Bonzini [mailto:paolo.bonz...@gmail.com] On Behalf Of Paolo
> Bonzini
> Sent: Friday, January 05, 2018 3:32 AM
> To: Van De Ven, Arjan ; Linus Torvalds
> ; David Woodhouse 
> Cc: Tim Chen ; Thomas Gleixner
> ; Andy Lutomirski ; Greg KH
> ; Hansen, Dave ;
> Andrea Arcangeli ; Andi Kleen ;
> Linux Kernel Mail

> > personally I am comfortable with retpoline on Skylake, but I would
> > like to have IBRS as an opt in for the paranoid.
> >
> 
> IBRS actually seems to perform more than decently on Skylake with the
> microcode updates we're shipping in RHEL---much more so than Broadwell
> or Haswell.  Can you confirm that this is expected?

Retpoline outperforms IBRS even on skylake family (including the client 
kabylake etc)
by some real margin, but IBRS is certainly not the end of the world either... 
more a case of 'we can do even faster'. The difference also gets bigger in 
retpoline's favor for older generations cpus.

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> -Original Message-
> From: Paolo Bonzini [mailto:paolo.bonz...@gmail.com] On Behalf Of Paolo
> Bonzini
> Sent: Friday, January 05, 2018 3:32 AM
> To: Van De Ven, Arjan ; Linus Torvalds
> ; David Woodhouse 
> Cc: Tim Chen ; Thomas Gleixner
> ; Andy Lutomirski ; Greg KH
> ; Hansen, Dave ;
> Andrea Arcangeli ; Andi Kleen ;
> Linux Kernel Mail

> > personally I am comfortable with retpoline on Skylake, but I would
> > like to have IBRS as an opt in for the paranoid.
> >
> 
> IBRS actually seems to perform more than decently on Skylake with the
> microcode updates we're shipping in RHEL---much more so than Broadwell
> or Haswell.  Can you confirm that this is expected?

Retpoline outperforms IBRS even on skylake family (including the client 
kabylake etc)
by some real margin, but IBRS is certainly not the end of the world either... 
more a case of 'we can do even faster'. The difference also gets bigger in 
retpoline's favor for older generations cpus.

Re: [PATCH 0/7] IBRS patch series

[Andrea Arcangeli]

On Thu, Jan 04, 2018 at 09:22:34PM +, Van De Ven, Arjan wrote:
> personally I am comfortable with retpoline on Skylake, but I would
>like to have IBRS as an opt in for the paranoid.

I think this whole variant#2 issue has to be fixed mathematically or
not at all, the reason is that it's already "near zero risk" to begin
with.

If we leave any theoretical issues open we can as well stick with the
"near zero risk" wording on AMD website and not fix anything.

Doing a huge amount of work with reptoline and then you find SMM is
called reproducibly somehow and a new PoC could exist for it, not fun.

Whatever solution should be total and guaranteed or there's huge
amount of resources wasted.

Again spectre variant#2 this is "near zero risk" to begin with, on all
CPUs because of the complexity to mount an attack on any given
kernel/CPU combination out there. I'm mostly thinking about the "guest
mode"/"host kernel mod" isolation here but this should apply in
general to all kernel protection for spectre variant#2.

If we can do a 3 way alternative IBRS on skylake, repotline + IBRS
around all BIOS, graphics firmware and pci firmware + kernel asm
patched with a 2-way alternative between amd reptoline and intel
reptoline emitted 2-way by same gcc build, and still guarantee this
solution is as mathematically safe as a pure IBRS (which is obviously
mathematically safe), ok. Otherwise it'd prefer to stick to pure IBRS
and skip the reptoline complexity and fallouts here and there around
bios asm firmware SMM etc.. not to tell the need to rebuild qemu with
reptoline with 2-way amd/intel alternatives self modifying code in
userland during qemu startup to achieve ibrs 1 ibpb 1 + prctl IBRS on
qemu with repotline.

Thanks,
Andrea

Re: [PATCH 0/7] IBRS patch series

[Andrea Arcangeli]

On Thu, Jan 04, 2018 at 09:22:34PM +, Van De Ven, Arjan wrote:
> personally I am comfortable with retpoline on Skylake, but I would
>like to have IBRS as an opt in for the paranoid.

I think this whole variant#2 issue has to be fixed mathematically or
not at all, the reason is that it's already "near zero risk" to begin
with.

If we leave any theoretical issues open we can as well stick with the
"near zero risk" wording on AMD website and not fix anything.

Doing a huge amount of work with reptoline and then you find SMM is
called reproducibly somehow and a new PoC could exist for it, not fun.

Whatever solution should be total and guaranteed or there's huge
amount of resources wasted.

Again spectre variant#2 this is "near zero risk" to begin with, on all
CPUs because of the complexity to mount an attack on any given
kernel/CPU combination out there. I'm mostly thinking about the "guest
mode"/"host kernel mod" isolation here but this should apply in
general to all kernel protection for spectre variant#2.

If we can do a 3 way alternative IBRS on skylake, repotline + IBRS
around all BIOS, graphics firmware and pci firmware + kernel asm
patched with a 2-way alternative between amd reptoline and intel
reptoline emitted 2-way by same gcc build, and still guarantee this
solution is as mathematically safe as a pure IBRS (which is obviously
mathematically safe), ok. Otherwise it'd prefer to stick to pure IBRS
and skip the reptoline complexity and fallouts here and there around
bios asm firmware SMM etc.. not to tell the need to rebuild qemu with
reptoline with 2-way amd/intel alternatives self modifying code in
userland during qemu startup to achieve ibrs 1 ibpb 1 + prctl IBRS on
qemu with repotline.

Thanks,
Andrea

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> > So long as the underlying binary satisfies the precondition that it
> > will not underflow its own RSB.
> >
> > Then we if we subsequently guarantee never to _reduce_ the number of
> > entries in its RSB at any point remote to its own execution, then the
> > precondition is preserved and underflow will not occur.
> 
> The problem is that underflow can occur not only on a retpoline, but
> also on *any* bare ret.

that is not true though

yes there's NMIs and interrupts but those Paul managed.

the cases where underflow can happen are not infinite or unbound, but a 
specific set of causes as Paul already mentioned.

Now there is complexity in proving that all that is there, and if you're that 
paranoid, IBRS is certainly an option and the performance, while not nearly as 
good as retpoline, is not horrid either on Skylake. 

This is why I said I would like to see retpoline be the default, with IBRS an 
opt-in for the paranoid. I guess David will turn that on ;-)

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> > So long as the underlying binary satisfies the precondition that it
> > will not underflow its own RSB.
> >
> > Then we if we subsequently guarantee never to _reduce_ the number of
> > entries in its RSB at any point remote to its own execution, then the
> > precondition is preserved and underflow will not occur.
> 
> The problem is that underflow can occur not only on a retpoline, but
> also on *any* bare ret.

that is not true though

yes there's NMIs and interrupts but those Paul managed.

the cases where underflow can happen are not infinite or unbound, but a 
specific set of causes as Paul already mentioned.

Now there is complexity in proving that all that is there, and if you're that 
paranoid, IBRS is certainly an option and the performance, while not nearly as 
good as retpoline, is not horrid either on Skylake. 

This is why I said I would like to see retpoline be the default, with IBRS an 
opt-in for the paranoid. I guess David will turn that on ;-)

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 03:52 -0800, Paul Turner wrote:
> 
> These are also mitigatable; the retpoline sequence itself will never
> result in an RSB underflow.

Unless an event occurs which clears the RSB between the CALL and the
RET of the retpoline.

> So long as the underlying binary satisfies the precondition that it
> will not underflow its own RSB.
> 
> Then we if we subsequently guarantee never to _reduce_ the number of
> entries in its RSB at any point remote to its own execution, then the
> precondition is preserved and underflow will not occur.

The problem is that underflow can occur not only on a retpoline, but
also on *any* bare ret.

Unless we want to do something evil like turning them all into a
sequence of 'call $+1; sub $8, %rsp; ret' and narrowing the race window
for that 'external event' to be negligible.

On the whole, since IBRS doesn't perform as badly on Skylake+ as it
does on earlier CPUs, it makes more sense just to use IBRS on Skylake+.

Unless we *only* have retpoline, of course, in which case we use that.


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 03:52 -0800, Paul Turner wrote:
> 
> These are also mitigatable; the retpoline sequence itself will never
> result in an RSB underflow.

Unless an event occurs which clears the RSB between the CALL and the
RET of the retpoline.

> So long as the underlying binary satisfies the precondition that it
> will not underflow its own RSB.
> 
> Then we if we subsequently guarantee never to _reduce_ the number of
> entries in its RSB at any point remote to its own execution, then the
> precondition is preserved and underflow will not occur.

The problem is that underflow can occur not only on a retpoline, but
also on *any* bare ret.

Unless we want to do something evil like turning them all into a
sequence of 'call $+1; sub $8, %rsp; ret' and narrowing the race window
for that 'external event' to be negligible.

On the whole, since IBRS doesn't perform as badly on Skylake+ as it
does on earlier CPUs, it makes more sense just to use IBRS on Skylake+.

Unless we *only* have retpoline, of course, in which case we use that.


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[Paolo Bonzini]

On 05/01/2018 15:01, Greg KH wrote:
>> Obviously it lacks a *lot* of processors (especially pre-Haswell).
> 
> I'm running Arch, but it would be nice to know where those microcode
> updates came from, given that they aren't on the "official" Intel page
> yet :)

Those from November seem way too early to include IBRS/IBPB.  Maybe the
two from December 3rd, but I wouldn't be 100% sure.

So it would be even nicer to know how those microcode updates were tested.

(And by the way, the LFENCE change is for variant 1 aka CVE-2017-5753).

Paolo

Re: [PATCH 0/7] IBRS patch series

[Paolo Bonzini]

On 05/01/2018 15:01, Greg KH wrote:
>> Obviously it lacks a *lot* of processors (especially pre-Haswell).
> 
> I'm running Arch, but it would be nice to know where those microcode
> updates came from, given that they aren't on the "official" Intel page
> yet :)

Those from November seem way too early to include IBRS/IBPB.  Maybe the
two from December 3rd, but I wouldn't be 100% sure.

So it would be even nicer to know how those microcode updates were tested.

(And by the way, the LFENCE change is for variant 1 aka CVE-2017-5753).

Paolo

Re: [PATCH 0/7] IBRS patch series

[Greg KH]

On Fri, Jan 05, 2018 at 02:47:45PM +0100, Yves-Alexis Perez wrote:
> On Fri, 2018-01-05 at 14:28 +0100, Greg KH wrote:
> > > iucode-tool -L -tr n10ur16w.iso |grep 2017-11
> > >001/020: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size
> > > 18432
> > 
> > That's been out for a while now:
> > https://downloadcenter.intel.com/download/27337/Linux-Processor-Micr
> > ocode-Data-File
> > 
> > so I really don't think this is an "update" yet.
> 
> Actually for my processor (0x000306d4 / i5-5200U) this release (20171117) only
> contains:
> 
>   050/001: sig 0x000306d4, pf_mask 0xc0, 2017-01-27, rev 0x0025, size 17408
> 
> I find it weird that the Intel release from 2017-11-17 doesn't contain a
> microcode file supposedly released on the same day, but it seems to be the
> case.

Doh, same here, sorry about that, should have checked the microcode
image file myself.

> > I saw a bios update for my Dell laptop dated December 28, but it didn't
> > seem to change anything with the cpuids or microcode status from before.
> 
> Not sure which distribution you use nowadays but Henrique just pushed a
> microcode update to Debian with some collected microcodes:
> 
> https://tracker.debian.org/news/899110
> 
> Obviously it lacks a *lot* of processors (especially pre-Haswell).

I'm running Arch, but it would be nice to know where those microcode
updates came from, given that they aren't on the "official" Intel page
yet :)

> > A hard answer to what "soon" means here would be nice to get, as I'm
> > seeing a lot of misunderstanding floating around right now :(
> 
> Agreed, but I'm not sure who can provide the information.

That's why I'm asking here, hopefully Arjan might have an idea.

thanks,

greg k-h

Re: [PATCH 0/7] IBRS patch series

[Greg KH]

On Fri, Jan 05, 2018 at 02:47:45PM +0100, Yves-Alexis Perez wrote:
> On Fri, 2018-01-05 at 14:28 +0100, Greg KH wrote:
> > > iucode-tool -L -tr n10ur16w.iso |grep 2017-11
> > >001/020: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size
> > > 18432
> > 
> > That's been out for a while now:
> > https://downloadcenter.intel.com/download/27337/Linux-Processor-Micr
> > ocode-Data-File
> > 
> > so I really don't think this is an "update" yet.
> 
> Actually for my processor (0x000306d4 / i5-5200U) this release (20171117) only
> contains:
> 
>   050/001: sig 0x000306d4, pf_mask 0xc0, 2017-01-27, rev 0x0025, size 17408
> 
> I find it weird that the Intel release from 2017-11-17 doesn't contain a
> microcode file supposedly released on the same day, but it seems to be the
> case.

Doh, same here, sorry about that, should have checked the microcode
image file myself.

> > I saw a bios update for my Dell laptop dated December 28, but it didn't
> > seem to change anything with the cpuids or microcode status from before.
> 
> Not sure which distribution you use nowadays but Henrique just pushed a
> microcode update to Debian with some collected microcodes:
> 
> https://tracker.debian.org/news/899110
> 
> Obviously it lacks a *lot* of processors (especially pre-Haswell).

I'm running Arch, but it would be nice to know where those microcode
updates came from, given that they aren't on the "official" Intel page
yet :)

> > A hard answer to what "soon" means here would be nice to get, as I'm
> > seeing a lot of misunderstanding floating around right now :(
> 
> Agreed, but I'm not sure who can provide the information.

That's why I'm asking here, hopefully Arjan might have an idea.

thanks,

greg k-h

Re: [PATCH 0/7] IBRS patch series

[Yves-Alexis Perez]

On Fri, 2018-01-05 at 14:28 +0100, Greg KH wrote:
> > iucode-tool -L -tr n10ur16w.iso |grep 2017-11
> >001/020: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size
> > 18432
> 
> That's been out for a while now:
> https://downloadcenter.intel.com/download/27337/Linux-Processor-Micr
> ocode-Data-File
> 
> so I really don't think this is an "update" yet.

Actually for my processor (0x000306d4 / i5-5200U) this release (20171117) only
contains:

  050/001: sig 0x000306d4, pf_mask 0xc0, 2017-01-27, rev 0x0025, size 17408

I find it weird that the Intel release from 2017-11-17 doesn't contain a
microcode file supposedly released on the same day, but it seems to be the
case.

> I saw a bios update for my Dell laptop dated December 28, but it didn't
> seem to change anything with the cpuids or microcode status from before.

Not sure which distribution you use nowadays but Henrique just pushed a
microcode update to Debian with some collected microcodes:

https://tracker.debian.org/news/899110

Obviously it lacks a *lot* of processors (especially pre-Haswell).
> 
> A hard answer to what "soon" means here would be nice to get, as I'm
> seeing a lot of misunderstanding floating around right now :(

Agreed, but I'm not sure who can provide the information.

Regards,
-- 
Yves-Alexis

signature.asc
Description: This is a digitally signed message part

Re: [PATCH 0/7] IBRS patch series

[Yves-Alexis Perez]

On Fri, 2018-01-05 at 14:28 +0100, Greg KH wrote:
> > iucode-tool -L -tr n10ur16w.iso |grep 2017-11
> >001/020: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size
> > 18432
> 
> That's been out for a while now:
> https://downloadcenter.intel.com/download/27337/Linux-Processor-Micr
> ocode-Data-File
> 
> so I really don't think this is an "update" yet.

Actually for my processor (0x000306d4 / i5-5200U) this release (20171117) only
contains:

  050/001: sig 0x000306d4, pf_mask 0xc0, 2017-01-27, rev 0x0025, size 17408

I find it weird that the Intel release from 2017-11-17 doesn't contain a
microcode file supposedly released on the same day, but it seems to be the
case.

> I saw a bios update for my Dell laptop dated December 28, but it didn't
> seem to change anything with the cpuids or microcode status from before.

Not sure which distribution you use nowadays but Henrique just pushed a
microcode update to Debian with some collected microcodes:

https://tracker.debian.org/news/899110

Obviously it lacks a *lot* of processors (especially pre-Haswell).
> 
> A hard answer to what "soon" means here would be nice to get, as I'm
> seeing a lot of misunderstanding floating around right now :(

Agreed, but I'm not sure who can provide the information.

Regards,
-- 
Yves-Alexis

signature.asc
Description: This is a digitally signed message part

Re: [PATCH 0/7] IBRS patch series

[Greg KH]

On Thu, Jan 04, 2018 at 10:01:52PM +0100, Yves-Alexis Perez wrote:
> On Thu, 2018-01-04 at 11:10 -0800, Tim Chen wrote:
> > > Are there plans to make the corresponding microcode support available?
> > > 
> > 
> > The microcode patches should be released soon.
> 
> In the meantime, Lenovo has started issuing BIOS/UEFI updates which include
> microcode updates for this.
> 
> See for example for the ThinkPad X250
> 
> https://download.lenovo.com/pccbbs/mobiles/n10ur16w.txt
> https://download.lenovo.com/pccbbs/mobiles/n10ur16w.iso
> 
> iucode-tool -L -tr n10ur16w.iso |grep 2017-11
>   001/020: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size 18432

That's been out for a while now:

https://downloadcenter.intel.com/download/27337/Linux-Processor-Microcode-Data-File

so I really don't think this is an "update" yet.

I saw a bios update for my Dell laptop dated December 28, but it didn't
seem to change anything with the cpuids or microcode status from before.

A hard answer to what "soon" means here would be nice to get, as I'm
seeing a lot of misunderstanding floating around right now :(

thanks,

greg k-h

Re: [PATCH 0/7] IBRS patch series

[Greg KH]

On Thu, Jan 04, 2018 at 10:01:52PM +0100, Yves-Alexis Perez wrote:
> On Thu, 2018-01-04 at 11:10 -0800, Tim Chen wrote:
> > > Are there plans to make the corresponding microcode support available?
> > > 
> > 
> > The microcode patches should be released soon.
> 
> In the meantime, Lenovo has started issuing BIOS/UEFI updates which include
> microcode updates for this.
> 
> See for example for the ThinkPad X250
> 
> https://download.lenovo.com/pccbbs/mobiles/n10ur16w.txt
> https://download.lenovo.com/pccbbs/mobiles/n10ur16w.iso
> 
> iucode-tool -L -tr n10ur16w.iso |grep 2017-11
>   001/020: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size 18432

That's been out for a while now:

https://downloadcenter.intel.com/download/27337/Linux-Processor-Microcode-Data-File

so I really don't think this is an "update" yet.

I saw a bios update for my Dell laptop dated December 28, but it didn't
seem to change anything with the cpuids or microcode status from before.

A hard answer to what "soon" means here would be nice to get, as I'm
seeing a lot of misunderstanding floating around right now :(

thanks,

greg k-h

Re: [PATCH 0/7] IBRS patch series

[Paul Turner]

On Fri, Jan 5, 2018 at 3:32 AM, Paolo Bonzini  wrote:
> On 04/01/2018 22:22, Van De Ven, Arjan wrote:
>> this is about a level of paranoia you are comfortable with.
>>
>> Retpoline on Skylake raises the bar for the issue enormously, but
>> there are a set of corner cases that exist and that are not trivial
>> to prove you dealt with them.
>>
>> personally I am comfortable with retpoline on Skylake, but I would
>> like to have IBRS as an opt in for the paranoid.
>>
>
> IBRS actually seems to perform more than decently on Skylake with the
> microcode updates we're shipping in RHEL---much more so than Broadwell
> or Haswell.  Can you confirm that this is expected?
>

The cost of IBRS performance varies with processor generation.
Skylake incurs the least overhead.  It is expected that future
generations will be better still.

There are two distinct costs: The overhead to an indirect branch, and
the transition cost for enabling/disabling the feature as we schedule
into (and out of) protected code.

A naked indirect call (with IBRS enabled) on Skylake and a retpolined
call have approximately the same cost.
(I have not compared this cost for pre-Skylake uarchs.)

The main difference is that a retpolined binary does not incur the
transition costs, nor does it interact with sibling execution while
protected code is running.

A trade-off to consider when choosing between them is that it does on
the other hand carry a higher implementation (versus run-time) cost.
As Arjan references above, this is also a sliding scale.  The bar is
significantly raised by compiling with retpoline alone, and the
vulnerability is wholly eliminated if the preconditions are also
satisfied.
(And obviously, if you do not have the sources to the target you are
trying to protect, IBRS allows you to run it in a protected fashion --
while it cannot easily be retpolined.)

Re: [PATCH 0/7] IBRS patch series

[Paul Turner]

On Fri, Jan 5, 2018 at 3:32 AM, Paolo Bonzini  wrote:
> On 04/01/2018 22:22, Van De Ven, Arjan wrote:
>> this is about a level of paranoia you are comfortable with.
>>
>> Retpoline on Skylake raises the bar for the issue enormously, but
>> there are a set of corner cases that exist and that are not trivial
>> to prove you dealt with them.
>>
>> personally I am comfortable with retpoline on Skylake, but I would
>> like to have IBRS as an opt in for the paranoid.
>>
>
> IBRS actually seems to perform more than decently on Skylake with the
> microcode updates we're shipping in RHEL---much more so than Broadwell
> or Haswell.  Can you confirm that this is expected?
>

The cost of IBRS performance varies with processor generation.
Skylake incurs the least overhead.  It is expected that future
generations will be better still.

There are two distinct costs: The overhead to an indirect branch, and
the transition cost for enabling/disabling the feature as we schedule
into (and out of) protected code.

A naked indirect call (with IBRS enabled) on Skylake and a retpolined
call have approximately the same cost.
(I have not compared this cost for pre-Skylake uarchs.)

The main difference is that a retpolined binary does not incur the
transition costs, nor does it interact with sibling execution while
protected code is running.

A trade-off to consider when choosing between them is that it does on
the other hand carry a higher implementation (versus run-time) cost.
As Arjan references above, this is also a sliding scale.  The bar is
significantly raised by compiling with retpoline alone, and the
vulnerability is wholly eliminated if the preconditions are also
satisfied.
(And obviously, if you do not have the sources to the target you are
trying to protect, IBRS allows you to run it in a protected fashion --
while it cannot easily be retpolined.)

Re: [PATCH 0/7] IBRS patch series

[Paul Turner]

On Thu, Jan 4, 2018 at 11:33 AM, Linus Torvalds
 wrote:
> On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse  wrote:
>>
>> On Skylake the target for a 'ret' instruction may also come from the
>> BTB. So if you ever let the RSB (which remembers where the 'call's came
>> from get empty, you end up vulnerable.
>
> That sounds like it could cause mispredicts, but it doesn't sound 
> _exploitable_.
>
> Sure, interrupts in between the call instruction and the 'ret' could
> overflow the return stack. And we could migrate to another CPU. And so
> apparently SMM clears the return stack too.
>
> ... but again, none of them sound even remotely _exploitable_.

These are also mitigatable; the retpoline sequence itself will never
result in an RSB underflow.

So long as the underlying binary satisfies the precondition that it
will not underflow its own RSB.

Then we if we subsequently guarantee never to _reduce_ the number of
entries in its RSB at any point remote to its own execution, then the
precondition is preserved and underflow will not occur.
This does not preclude perturbing the RSB, only that it's contents
contain at least as many (non-exploitable) entries as when execution
was interrupted.
We do not need to actually track how many entries it's currently
using, because returning control to it with a full RSB, guarantees
that number of entries cannot have decreased.

>
> Remember: it's not mispredicts that leak information. It's *exploits"
> that use forced very specific  mispredicts to leak information.
>
> There's a big difference there. And I think patch authors should keep
> that difference in mind.
>
> For example, flushing the BTB at kernel entry doesn't mean that later
> in-kernel indirect branches don't get predicted, and doesn't even mean
> that they don't get mis-predicted. It only means that an exploit can't
> pre-populate those things and use them for exploits.

Or worse, that a co-executing SMT sibling is not continually refilling them.

>
>Linus

Re: [PATCH 0/7] IBRS patch series

[Paul Turner]

On Thu, Jan 4, 2018 at 11:33 AM, Linus Torvalds
 wrote:
> On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse  wrote:
>>
>> On Skylake the target for a 'ret' instruction may also come from the
>> BTB. So if you ever let the RSB (which remembers where the 'call's came
>> from get empty, you end up vulnerable.
>
> That sounds like it could cause mispredicts, but it doesn't sound 
> _exploitable_.
>
> Sure, interrupts in between the call instruction and the 'ret' could
> overflow the return stack. And we could migrate to another CPU. And so
> apparently SMM clears the return stack too.
>
> ... but again, none of them sound even remotely _exploitable_.

These are also mitigatable; the retpoline sequence itself will never
result in an RSB underflow.

So long as the underlying binary satisfies the precondition that it
will not underflow its own RSB.

Then we if we subsequently guarantee never to _reduce_ the number of
entries in its RSB at any point remote to its own execution, then the
precondition is preserved and underflow will not occur.
This does not preclude perturbing the RSB, only that it's contents
contain at least as many (non-exploitable) entries as when execution
was interrupted.
We do not need to actually track how many entries it's currently
using, because returning control to it with a full RSB, guarantees
that number of entries cannot have decreased.

>
> Remember: it's not mispredicts that leak information. It's *exploits"
> that use forced very specific  mispredicts to leak information.
>
> There's a big difference there. And I think patch authors should keep
> that difference in mind.
>
> For example, flushing the BTB at kernel entry doesn't mean that later
> in-kernel indirect branches don't get predicted, and doesn't even mean
> that they don't get mis-predicted. It only means that an exploit can't
> pre-populate those things and use them for exploits.

Or worse, that a co-executing SMT sibling is not continually refilling them.

>
>Linus

Re: [PATCH 0/7] IBRS patch series

[Paolo Bonzini]

On 04/01/2018 22:22, Van De Ven, Arjan wrote:
> this is about a level of paranoia you are comfortable with.
> 
> Retpoline on Skylake raises the bar for the issue enormously, but
> there are a set of corner cases that exist and that are not trivial
> to prove you dealt with them.
> 
> personally I am comfortable with retpoline on Skylake, but I would
> like to have IBRS as an opt in for the paranoid.
> 

IBRS actually seems to perform more than decently on Skylake with the
microcode updates we're shipping in RHEL---much more so than Broadwell
or Haswell.  Can you confirm that this is expected?

Paolo

Re: [PATCH 0/7] IBRS patch series

[Paolo Bonzini]

On 04/01/2018 22:22, Van De Ven, Arjan wrote:
> this is about a level of paranoia you are comfortable with.
> 
> Retpoline on Skylake raises the bar for the issue enormously, but
> there are a set of corner cases that exist and that are not trivial
> to prove you dealt with them.
> 
> personally I am comfortable with retpoline on Skylake, but I would
> like to have IBRS as an opt in for the paranoid.
> 

IBRS actually seems to perform more than decently on Skylake with the
microcode updates we're shipping in RHEL---much more so than Broadwell
or Haswell.  Can you confirm that this is expected?

Paolo

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 06:25 +0100, Florian Weimer wrote:
> 
> Retpoline also looks incompatible with CET, so future Intel CPUs will
> eventually need a different approach anyway.

CPUs with CET will have the indirect branch problem fixed, so the
retpoline ALTERNATIVE will be used which is a bare 'jmp *%reg'.

That's why we switched the ABI for the retpoline from having the target
address on the stack, to passing it in in a register.


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Fri, 2018-01-05 at 06:25 +0100, Florian Weimer wrote:
> 
> Retpoline also looks incompatible with CET, so future Intel CPUs will
> eventually need a different approach anyway.

CPUs with CET will have the indirect branch problem fixed, so the
retpoline ALTERNATIVE will be used which is a bare 'jmp *%reg'.

That's why we switched the ABI for the retpoline from having the target
address on the stack, to passing it in in a register.


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[Florian Weimer]

* Linus Torvalds:

> On Thu, Jan 4, 2018 at 9:56 AM, Tim Chen  wrote:
>>
>> Speculation on Skylake and later requires these patches ("dynamic IBRS")
>> be used instead of retpoline[1].
>
> Can somebody explain this part?
>
> I was assuming that retpoline would work around this issue on all uarchs.
>
> This seems to say "retpoline does nothing on Skylake+"

Retpoline also looks incompatible with CET, so future Intel CPUs will
eventually need a different approach anyway.

Re: [PATCH 0/7] IBRS patch series

[Florian Weimer]

* Linus Torvalds:

> On Thu, Jan 4, 2018 at 9:56 AM, Tim Chen  wrote:
>>
>> Speculation on Skylake and later requires these patches ("dynamic IBRS")
>> be used instead of retpoline[1].
>
> Can somebody explain this part?
>
> I was assuming that retpoline would work around this issue on all uarchs.
>
> This seems to say "retpoline does nothing on Skylake+"

Retpoline also looks incompatible with CET, so future Intel CPUs will
eventually need a different approach anyway.

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse 
> wrote:
> >
> > On Skylake the target for a 'ret' instruction may also come from the
> > BTB. So if you ever let the RSB (which remembers where the 'call's came
> > from get empty, you end up vulnerable.
> 
> That sounds like it could cause mispredicts, but it doesn't sound 
> _exploitable_.
> 
> Sure, interrupts in between the call instruction and the 'ret' could
> overflow the return stack. And we could migrate to another CPU. And so
> apparently SMM clears the return stack too.
> 
> ... but again, none of them sound even remotely _exploitable_.
> 

this is about a level of paranoia you are comfortable with.

Retpoline on Skylake raises the bar for the issue enormously, but there are a 
set of corner cases that exist and that are not trivial to prove you dealt with 
them.

personally I am comfortable with retpoline on Skylake, but I would like to have 
IBRS as an opt in for the paranoid.

RE: [PATCH 0/7] IBRS patch series

[Van De Ven, Arjan]

> On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse 
> wrote:
> >
> > On Skylake the target for a 'ret' instruction may also come from the
> > BTB. So if you ever let the RSB (which remembers where the 'call's came
> > from get empty, you end up vulnerable.
> 
> That sounds like it could cause mispredicts, but it doesn't sound 
> _exploitable_.
> 
> Sure, interrupts in between the call instruction and the 'ret' could
> overflow the return stack. And we could migrate to another CPU. And so
> apparently SMM clears the return stack too.
> 
> ... but again, none of them sound even remotely _exploitable_.
> 

this is about a level of paranoia you are comfortable with.

Retpoline on Skylake raises the bar for the issue enormously, but there are a 
set of corner cases that exist and that are not trivial to prove you dealt with 
them.

personally I am comfortable with retpoline on Skylake, but I would like to have 
IBRS as an opt in for the paranoid.

Re: [PATCH 0/7] IBRS patch series

[Yves-Alexis Perez]

On Thu, 2018-01-04 at 11:10 -0800, Tim Chen wrote:
> > Are there plans to make the corresponding microcode support available?
> > 
> 
> The microcode patches should be released soon.

In the meantime, Lenovo has started issuing BIOS/UEFI updates which include
microcode updates for this.

See for example for the ThinkPad X250

https://download.lenovo.com/pccbbs/mobiles/n10ur16w.txt
https://download.lenovo.com/pccbbs/mobiles/n10ur16w.iso

iucode-tool -L -tr n10ur16w.iso |grep 2017-11
  001/020: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size 18432
  001/022: sig 0x00040651, pf_mask 0x72, 2017-11-20, rev 0x0021, size 22528
  001/025: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size 18432
  001/027: sig 0x00040651, pf_mask 0x72, 2017-11-20, rev 0x0021, size 22528
  001/030: sig 0x000306c3, pf_mask 0x32, 2017-11-20, rev 0x0023, size 23552
  001/032: sig 0x00040651, pf_mask 0x72, 2017-11-20, rev 0x0021, size 22528

Regards,
-- 
Yves-Alexis

signature.asc
Description: This is a digitally signed message part

Re: [PATCH 0/7] IBRS patch series

[Yves-Alexis Perez]

On Thu, 2018-01-04 at 11:10 -0800, Tim Chen wrote:
> > Are there plans to make the corresponding microcode support available?
> > 
> 
> The microcode patches should be released soon.

In the meantime, Lenovo has started issuing BIOS/UEFI updates which include
microcode updates for this.

See for example for the ThinkPad X250

https://download.lenovo.com/pccbbs/mobiles/n10ur16w.txt
https://download.lenovo.com/pccbbs/mobiles/n10ur16w.iso

iucode-tool -L -tr n10ur16w.iso |grep 2017-11
  001/020: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size 18432
  001/022: sig 0x00040651, pf_mask 0x72, 2017-11-20, rev 0x0021, size 22528
  001/025: sig 0x000306d4, pf_mask 0xc0, 2017-11-17, rev 0x0028, size 18432
  001/027: sig 0x00040651, pf_mask 0x72, 2017-11-20, rev 0x0021, size 22528
  001/030: sig 0x000306c3, pf_mask 0x32, 2017-11-20, rev 0x0023, size 23552
  001/032: sig 0x00040651, pf_mask 0x72, 2017-11-20, rev 0x0021, size 22528

Regards,
-- 
Yves-Alexis

signature.asc
Description: This is a digitally signed message part

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Thu, 2018-01-04 at 19:40 +, Andrew Cooper wrote:
> 
> Also remember that sibling threads share a BTB, so you can't rely on
> isolated straight-line codepath on the current cpu for safety. (e.g. by
> issuing an IBPB on every entry to supervisor mode).

That is just one of a whole litany of reasons the RSB gets flushed,
*many* of which were individually sufficient for us to throw our toys
out of the pram and say "OK, this doesn't work on Skylake".


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Thu, 2018-01-04 at 19:40 +, Andrew Cooper wrote:
> 
> Also remember that sibling threads share a BTB, so you can't rely on
> isolated straight-line codepath on the current cpu for safety. (e.g. by
> issuing an IBPB on every entry to supervisor mode).

That is just one of a whole litany of reasons the RSB gets flushed,
*many* of which were individually sufficient for us to throw our toys
out of the pram and say "OK, this doesn't work on Skylake".


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[Andrew Cooper]

On 04/01/18 19:33, Linus Torvalds wrote:
> On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse  wrote:
>> On Skylake the target for a 'ret' instruction may also come from the
>> BTB. So if you ever let the RSB (which remembers where the 'call's came
>> from get empty, you end up vulnerable.
> That sounds like it could cause mispredicts, but it doesn't sound 
> _exploitable_.
>
> Sure, interrupts in between the call instruction and the 'ret' could
> overflow the return stack. And we could migrate to another CPU. And so
> apparently SMM clears the return stack too.
>
> ... but again, none of them sound even remotely _exploitable_.
>
> Remember: it's not mispredicts that leak information. It's *exploits"
> that use forced very specific  mispredicts to leak information.
>
> There's a big difference there. And I think patch authors should keep
> that difference in mind.
>
> For example, flushing the BTB at kernel entry doesn't mean that later
> in-kernel indirect branches don't get predicted, and doesn't even mean
> that they don't get mis-predicted. It only means that an exploit can't
> pre-populate those things and use them for exploits.

Retpoline as a mitigation strategy swaps indirect branches for returns,
to avoid using predictions which come from the BTB, as they can be
poisoned by an attacker.

The problem with Skylake+ is that an RSB underflow falls back to using a
BTB prediction, which allows the attacker to take control of speculation.

Also remember that sibling threads share a BTB, so you can't rely on
isolated straight-line codepath on the current cpu for safety. (e.g. by
issuing an IBPB on every entry to supervisor mode).

~Andrew

Re: [PATCH 0/7] IBRS patch series

[Andrew Cooper]

On 04/01/18 19:33, Linus Torvalds wrote:
> On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse  wrote:
>> On Skylake the target for a 'ret' instruction may also come from the
>> BTB. So if you ever let the RSB (which remembers where the 'call's came
>> from get empty, you end up vulnerable.
> That sounds like it could cause mispredicts, but it doesn't sound 
> _exploitable_.
>
> Sure, interrupts in between the call instruction and the 'ret' could
> overflow the return stack. And we could migrate to another CPU. And so
> apparently SMM clears the return stack too.
>
> ... but again, none of them sound even remotely _exploitable_.
>
> Remember: it's not mispredicts that leak information. It's *exploits"
> that use forced very specific  mispredicts to leak information.
>
> There's a big difference there. And I think patch authors should keep
> that difference in mind.
>
> For example, flushing the BTB at kernel entry doesn't mean that later
> in-kernel indirect branches don't get predicted, and doesn't even mean
> that they don't get mis-predicted. It only means that an exploit can't
> pre-populate those things and use them for exploits.

Retpoline as a mitigation strategy swaps indirect branches for returns,
to avoid using predictions which come from the BTB, as they can be
poisoned by an attacker.

The problem with Skylake+ is that an RSB underflow falls back to using a
BTB prediction, which allows the attacker to take control of speculation.

Also remember that sibling threads share a BTB, so you can't rely on
isolated straight-line codepath on the current cpu for safety. (e.g. by
issuing an IBPB on every entry to supervisor mode).

~Andrew

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Thu, 2018-01-04 at 11:33 -0800, Linus Torvalds wrote:
> On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse  wrote:
> >
> > On Skylake the target for a 'ret' instruction may also come from the
> > BTB. So if you ever let the RSB (which remembers where the 'call's came
> > from get empty, you end up vulnerable.
> 
> That sounds like it could cause mispredicts, but it doesn't sound 
> _exploitable_.
> 
> Sure, interrupts in between the call instruction and the 'ret' could
> overflow the return stack. And we could migrate to another CPU. And so
> apparently SMM clears the return stack too.
> 
> ... but again, none of them sound even remotely _exploitable_.

The concern is that the attacker could poison the BTB for a 'ret'
insteruction, as in the general case of the SP2 (conditional branch
misprediction) attack, so that it predicts a branch to an address of
the attacker's choice.

Now *most* of the time, one might expect the target for that 'ret' to
come from the RSB. But if there is a way to force the RSB to empty, or
the attacker is just happy to keep trying, and wait for things like SMI
to make it work every now and then, then it *might* be exploitable.

It's quite possible that a proof exists that all the above is *so*
hypothetical and unlikely, that we might as well use retpoline on
Skylake too. So far, nobody's proved it sufficiently; that's all.
So we're erring on the side of caution there.

smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Thu, 2018-01-04 at 11:33 -0800, Linus Torvalds wrote:
> On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse  wrote:
> >
> > On Skylake the target for a 'ret' instruction may also come from the
> > BTB. So if you ever let the RSB (which remembers where the 'call's came
> > from get empty, you end up vulnerable.
> 
> That sounds like it could cause mispredicts, but it doesn't sound 
> _exploitable_.
> 
> Sure, interrupts in between the call instruction and the 'ret' could
> overflow the return stack. And we could migrate to another CPU. And so
> apparently SMM clears the return stack too.
> 
> ... but again, none of them sound even remotely _exploitable_.

The concern is that the attacker could poison the BTB for a 'ret'
insteruction, as in the general case of the SP2 (conditional branch
misprediction) attack, so that it predicts a branch to an address of
the attacker's choice.

Now *most* of the time, one might expect the target for that 'ret' to
come from the RSB. But if there is a way to force the RSB to empty, or
the attacker is just happy to keep trying, and wait for things like SMI
to make it work every now and then, then it *might* be exploitable.

It's quite possible that a proof exists that all the above is *so*
hypothetical and unlikely, that we might as well use retpoline on
Skylake too. So far, nobody's proved it sufficiently; that's all.
So we're erring on the side of caution there.

smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[Linus Torvalds]

On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse  wrote:
>
> On Skylake the target for a 'ret' instruction may also come from the
> BTB. So if you ever let the RSB (which remembers where the 'call's came
> from get empty, you end up vulnerable.

That sounds like it could cause mispredicts, but it doesn't sound _exploitable_.

Sure, interrupts in between the call instruction and the 'ret' could
overflow the return stack. And we could migrate to another CPU. And so
apparently SMM clears the return stack too.

... but again, none of them sound even remotely _exploitable_.

Remember: it's not mispredicts that leak information. It's *exploits"
that use forced very specific  mispredicts to leak information.

There's a big difference there. And I think patch authors should keep
that difference in mind.

For example, flushing the BTB at kernel entry doesn't mean that later
in-kernel indirect branches don't get predicted, and doesn't even mean
that they don't get mis-predicted. It only means that an exploit can't
pre-populate those things and use them for exploits.

   Linus

Re: [PATCH 0/7] IBRS patch series

[Linus Torvalds]

On Thu, Jan 4, 2018 at 11:19 AM, David Woodhouse  wrote:
>
> On Skylake the target for a 'ret' instruction may also come from the
> BTB. So if you ever let the RSB (which remembers where the 'call's came
> from get empty, you end up vulnerable.

That sounds like it could cause mispredicts, but it doesn't sound _exploitable_.

Sure, interrupts in between the call instruction and the 'ret' could
overflow the return stack. And we could migrate to another CPU. And so
apparently SMM clears the return stack too.

... but again, none of them sound even remotely _exploitable_.

Remember: it's not mispredicts that leak information. It's *exploits"
that use forced very specific  mispredicts to leak information.

There's a big difference there. And I think patch authors should keep
that difference in mind.

For example, flushing the BTB at kernel entry doesn't mean that later
in-kernel indirect branches don't get predicted, and doesn't even mean
that they don't get mis-predicted. It only means that an exploit can't
pre-populate those things and use them for exploits.

   Linus

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Thu, 2018-01-04 at 11:00 -0800, Linus Torvalds wrote:
> On Thu, Jan 4, 2018 at 9:56 AM, Tim Chen 
> wrote:
> > 
> > 
> > Speculation on Skylake and later requires these patches ("dynamic
> > IBRS")
> > be used instead of retpoline[1].
> Can somebody explain this part?
> 
> I was assuming that retpoline would work around this issue on all
> uarchs.
> 
> This seems to say "retpoline does nothing on Skylake+"

On Skylake the target for a 'ret' instruction may also come from the
BTB. So if you ever let the RSB (which remembers where the 'call's came
from get empty, you end up vulnerable.

Other than the obvious call stack of more than 16 calls in depth,
there's also a big list of other things which can empty the RSB,
including an SMI.

Which basically makes retpoline on Skylake+ *very* hard to use
reliably. The plan is to use IBRS there and not retpoline.

smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[David Woodhouse]

On Thu, 2018-01-04 at 11:00 -0800, Linus Torvalds wrote:
> On Thu, Jan 4, 2018 at 9:56 AM, Tim Chen 
> wrote:
> > 
> > 
> > Speculation on Skylake and later requires these patches ("dynamic
> > IBRS")
> > be used instead of retpoline[1].
> Can somebody explain this part?
> 
> I was assuming that retpoline would work around this issue on all
> uarchs.
> 
> This seems to say "retpoline does nothing on Skylake+"

On Skylake the target for a 'ret' instruction may also come from the
BTB. So if you ever let the RSB (which remembers where the 'call's came
from get empty, you end up vulnerable.

Other than the obvious call stack of more than 16 calls in depth,
there's also a big list of other things which can empty the RSB,
including an SMI.

Which basically makes retpoline on Skylake+ *very* hard to use
reliably. The plan is to use IBRS there and not retpoline.

smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 0/7] IBRS patch series

[Tim Chen]

On 01/04/2018 11:05 AM, Justin Forbes wrote:
> On Thu, Jan 4, 2018 at 11:56 AM, Tim Chen  wrote:
>> This patch series enables the basic detection and usage of x86 indirect
>> branch speculation feature.  It enables the indirect branch restricted
>> speculation (IBRS) on kernel entry and disables it on exit.
>> It enumerates the indirect branch prediction barrier (IBPB).
>>
>> The x86 IBRS feature requires corresponding microcode support.
>> It mitigates the variant 2 vulnerability described in
>> https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html
>>
> 
> Are there plans to make the corresponding microcode support available?
> 

The microcode patches should be released soon.

Tim

Re: [PATCH 0/7] IBRS patch series

[Tim Chen]

On 01/04/2018 11:05 AM, Justin Forbes wrote:
> On Thu, Jan 4, 2018 at 11:56 AM, Tim Chen  wrote:
>> This patch series enables the basic detection and usage of x86 indirect
>> branch speculation feature.  It enables the indirect branch restricted
>> speculation (IBRS) on kernel entry and disables it on exit.
>> It enumerates the indirect branch prediction barrier (IBPB).
>>
>> The x86 IBRS feature requires corresponding microcode support.
>> It mitigates the variant 2 vulnerability described in
>> https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html
>>
> 
> Are there plans to make the corresponding microcode support available?
> 

The microcode patches should be released soon.

Tim

Re: [PATCH 0/7] IBRS patch series

[Justin Forbes]

On Thu, Jan 4, 2018 at 11:56 AM, Tim Chen  wrote:
> This patch series enables the basic detection and usage of x86 indirect
> branch speculation feature.  It enables the indirect branch restricted
> speculation (IBRS) on kernel entry and disables it on exit.
> It enumerates the indirect branch prediction barrier (IBPB).
>
> The x86 IBRS feature requires corresponding microcode support.
> It mitigates the variant 2 vulnerability described in
> https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html
>

Are there plans to make the corresponding microcode support available?

Re: [PATCH 0/7] IBRS patch series

[Justin Forbes]

On Thu, Jan 4, 2018 at 11:56 AM, Tim Chen  wrote:
> This patch series enables the basic detection and usage of x86 indirect
> branch speculation feature.  It enables the indirect branch restricted
> speculation (IBRS) on kernel entry and disables it on exit.
> It enumerates the indirect branch prediction barrier (IBPB).
>
> The x86 IBRS feature requires corresponding microcode support.
> It mitigates the variant 2 vulnerability described in
> https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html
>

Are there plans to make the corresponding microcode support available?

Re: [PATCH 0/7] IBRS patch series

[Linus Torvalds]

On Thu, Jan 4, 2018 at 9:56 AM, Tim Chen  wrote:
>
> Speculation on Skylake and later requires these patches ("dynamic IBRS")
> be used instead of retpoline[1].

Can somebody explain this part?

I was assuming that retpoline would work around this issue on all uarchs.

This seems to say "retpoline does nothing on Skylake+"

  Linus

Re: [PATCH 0/7] IBRS patch series

[Linus Torvalds]

On Thu, Jan 4, 2018 at 9:56 AM, Tim Chen  wrote:
>
> Speculation on Skylake and later requires these patches ("dynamic IBRS")
> be used instead of retpoline[1].

Can somebody explain this part?

I was assuming that retpoline would work around this issue on all uarchs.

This seems to say "retpoline does nothing on Skylake+"

  Linus

[PATCH 0/7] IBRS patch series

[Tim Chen]

This patch series enables the basic detection and usage of x86 indirect
branch speculation feature.  It enables the indirect branch restricted
speculation (IBRS) on kernel entry and disables it on exit.
It enumerates the indirect branch prediction barrier (IBPB).

The x86 IBRS feature requires corresponding microcode support.
It mitigates the variant 2 vulnerability described in
https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html

If IBRS is set, near returns and near indirect jumps/calls will not
allow their predicted target address to be controlled by code that
executed in a less privileged prediction mode before the IBRS mode was
last written with a value of 1 or on another logical processor so long
as all RSB entries from the previous less privileged prediction mode
are overwritten.

Setting of IBPB ensures that earlier code's behavior does not control later
indirect branch predictions.  It is used when context switching to new
untrusted address space. Unlike IBRS, IBPB is a command MSR
and does not retain its state.

Speculation on Skylake and later requires these patches ("dynamic IBRS")
be used instead of retpoline[1].  If you are very paranoid or you run on
a CPU where IBRS=1 is cheaper, you may also want to run in "IBRS always"
mode.

See: 
https://docs.google.com/document/d/e/2PACX-1vSMrwkaoSUBAFc6Fjd19F18c1O9pudkfAY-7lGYGOTN8mc9ul-J6pWadcAaBJZcVA7W_3jlLKRtKRbd/pub

More detailed description of IBRS is described in the first patch.

It is applied on top of the page table isolation changes.

A run time and boot time control of the IBRS feature is provided

There are 2 ways to control IBRS

1. At boot time
noibrs kernel boot parameter will disable IBRS usage

Otherwise if the above parameters are not specified, the system
will enable ibrs and ibpb usage if the cpu supports it.

2. At run time
echo 0 > /sys/kernel/debug/ibrs_enabled will turn off IBRS
echo 1 > /sys/kernel/debug/ibrs_enabled will turn on IBRS in kernel
echo 2 > /sys/kernel/debug/ibrs_enabled will turn on IBRS in both userspace 
and kernel (IBRS always)

[1] https://lkml.org/lkml/2018/1/4/174

Tim Chen (7):
  x86/feature: Detect the x86 feature to control Speculation
  x86/enter: MACROS to set/clear IBRS
  x86/enter: Use IBRS on syscall and interrupts
  x86/idle: Disable IBRS entering idle and enable it on wakeup
  x86: Use IBRS for firmware update path
  x86/spec_ctrl: Add sysctl knobs to enable/disable SPEC_CTRL feature
  x86/microcode: Recheck IBRS features on microcode reload

 Documentation/admin-guide/kernel-parameters.txt |   4 +
 arch/x86/entry/entry_64.S   |  24 +++
 arch/x86/entry/entry_64_compat.S|   9 +
 arch/x86/include/asm/apm.h  |   6 +
 arch/x86/include/asm/cpufeatures.h  |   1 +
 arch/x86/include/asm/efi.h  |  16 +-
 arch/x86/include/asm/msr-index.h|   7 +
 arch/x86/include/asm/mwait.h|  19 ++
 arch/x86/include/asm/spec_ctrl.h| 253 
 arch/x86/kernel/cpu/Makefile|   1 +
 arch/x86/kernel/cpu/microcode/core.c|   6 +
 arch/x86/kernel/cpu/scattered.c |  11 ++
 arch/x86/kernel/cpu/spec_ctrl.c | 124 
 arch/x86/kernel/process.c   |   9 +-
 14 files changed, 486 insertions(+), 4 deletions(-)
 create mode 100644 arch/x86/include/asm/spec_ctrl.h
 create mode 100644 arch/x86/kernel/cpu/spec_ctrl.c

-- 
2.9.4

[PATCH 0/7] IBRS patch series

[Tim Chen]

This patch series enables the basic detection and usage of x86 indirect
branch speculation feature.  It enables the indirect branch restricted
speculation (IBRS) on kernel entry and disables it on exit.
It enumerates the indirect branch prediction barrier (IBPB).

The x86 IBRS feature requires corresponding microcode support.
It mitigates the variant 2 vulnerability described in
https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html

If IBRS is set, near returns and near indirect jumps/calls will not
allow their predicted target address to be controlled by code that
executed in a less privileged prediction mode before the IBRS mode was
last written with a value of 1 or on another logical processor so long
as all RSB entries from the previous less privileged prediction mode
are overwritten.

Setting of IBPB ensures that earlier code's behavior does not control later
indirect branch predictions.  It is used when context switching to new
untrusted address space. Unlike IBRS, IBPB is a command MSR
and does not retain its state.

Speculation on Skylake and later requires these patches ("dynamic IBRS")
be used instead of retpoline[1].  If you are very paranoid or you run on
a CPU where IBRS=1 is cheaper, you may also want to run in "IBRS always"
mode.

See: 
https://docs.google.com/document/d/e/2PACX-1vSMrwkaoSUBAFc6Fjd19F18c1O9pudkfAY-7lGYGOTN8mc9ul-J6pWadcAaBJZcVA7W_3jlLKRtKRbd/pub

More detailed description of IBRS is described in the first patch.

It is applied on top of the page table isolation changes.

A run time and boot time control of the IBRS feature is provided

There are 2 ways to control IBRS

1. At boot time
noibrs kernel boot parameter will disable IBRS usage

Otherwise if the above parameters are not specified, the system
will enable ibrs and ibpb usage if the cpu supports it.

2. At run time
echo 0 > /sys/kernel/debug/ibrs_enabled will turn off IBRS
echo 1 > /sys/kernel/debug/ibrs_enabled will turn on IBRS in kernel
echo 2 > /sys/kernel/debug/ibrs_enabled will turn on IBRS in both userspace 
and kernel (IBRS always)

[1] https://lkml.org/lkml/2018/1/4/174

Tim Chen (7):
  x86/feature: Detect the x86 feature to control Speculation
  x86/enter: MACROS to set/clear IBRS
  x86/enter: Use IBRS on syscall and interrupts
  x86/idle: Disable IBRS entering idle and enable it on wakeup
  x86: Use IBRS for firmware update path
  x86/spec_ctrl: Add sysctl knobs to enable/disable SPEC_CTRL feature
  x86/microcode: Recheck IBRS features on microcode reload

 Documentation/admin-guide/kernel-parameters.txt |   4 +
 arch/x86/entry/entry_64.S   |  24 +++
 arch/x86/entry/entry_64_compat.S|   9 +
 arch/x86/include/asm/apm.h  |   6 +
 arch/x86/include/asm/cpufeatures.h  |   1 +
 arch/x86/include/asm/efi.h  |  16 +-
 arch/x86/include/asm/msr-index.h|   7 +
 arch/x86/include/asm/mwait.h|  19 ++
 arch/x86/include/asm/spec_ctrl.h| 253 
 arch/x86/kernel/cpu/Makefile|   1 +
 arch/x86/kernel/cpu/microcode/core.c|   6 +
 arch/x86/kernel/cpu/scattered.c |  11 ++
 arch/x86/kernel/cpu/spec_ctrl.c | 124 
 arch/x86/kernel/process.c   |   9 +-
 14 files changed, 486 insertions(+), 4 deletions(-)
 create mode 100644 arch/x86/include/asm/spec_ctrl.h
 create mode 100644 arch/x86/kernel/cpu/spec_ctrl.c

-- 
2.9.4