Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Masahiro Yamada]

2018-02-08 9:57 GMT+09:00 Kees Cook :
> On Thu, Feb 8, 2018 at 10:44 AM, Masahiro Yamada
>  wrote:
>> 2018-02-08 2:55 GMT+09:00 Linus Torvalds :
>>> What I would really want - and this is entirely unrelated to this
>>> particular case - is to have those damn compiler option tests as part
>>> of the config phase in general. We now have about a million of these
>>> crazy things, where we have config options that simply depend on which
>>> compiler we have, and we have no sane way to show them at
>>> configuration time.
>>>
>>> Though Andrew's tree I got yet another ugly hack
>>> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
>>> by turning it into a special magic Kconfig entry in the main Makefile.
>>> See commit 44c6dc940b19 ("Makefile: introduce
>>> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
>>> and honestly, I'm still thinking of just reverting it, because it's
>>> _so_ ugly and _so_ wrong.
>>>
>>> What we need is an extension to the Kconfig language itself so that we can 
>>> do
>>>
>>>config CC_HAS_RETPOLINE
>>> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>>>
>>> or something. And then we can make sane _conditional_ dependencies at
>>> Kconfig time, and our makefiles would be much cleaner too when you
>>> could just do
>>>
>>>  cflags-$(USE_RETPOLINE) += -mfunction-return=thunk 
>>> -mindirect-branch-table
>>>
>>> because the validity of the C compiler flag has been tested when 
>>> configuring.
>>>
>>> And then we could add that warning at configure time (or just disable
>>> the option there thanks to "depends on CC_HAS_xyz" logic).
>>>
>>> All our compiler option handling right now is just nasty nasty nasty crud.
>>>
>>> Adding more people in the hopes that somebody gets motivated.. I've
>>> talked about this before, so far we haven't made any progress.
>>
>>
>> Sorry for slow progress.
>>
>> I agreed this before, and still motivated.
>> (because I also motivated to remove kbuild cache.
>> This turned out not so clever as I first thought)
>>
>> I was trying to do this, but in this development cycle
>> I spent most of my time to flush out lots of piled up Kconfig patches.
>> Sorry.
>>
>> Unless somebody is working, I will.

I sent the first draft.

For people interested, I pushed the series in the following too.

git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
kconfig-opt-shell-rfc



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Masahiro Yamada]

2018-02-08 9:57 GMT+09:00 Kees Cook :
> On Thu, Feb 8, 2018 at 10:44 AM, Masahiro Yamada
>  wrote:
>> 2018-02-08 2:55 GMT+09:00 Linus Torvalds :
>>> What I would really want - and this is entirely unrelated to this
>>> particular case - is to have those damn compiler option tests as part
>>> of the config phase in general. We now have about a million of these
>>> crazy things, where we have config options that simply depend on which
>>> compiler we have, and we have no sane way to show them at
>>> configuration time.
>>>
>>> Though Andrew's tree I got yet another ugly hack
>>> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
>>> by turning it into a special magic Kconfig entry in the main Makefile.
>>> See commit 44c6dc940b19 ("Makefile: introduce
>>> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
>>> and honestly, I'm still thinking of just reverting it, because it's
>>> _so_ ugly and _so_ wrong.
>>>
>>> What we need is an extension to the Kconfig language itself so that we can 
>>> do
>>>
>>>config CC_HAS_RETPOLINE
>>> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>>>
>>> or something. And then we can make sane _conditional_ dependencies at
>>> Kconfig time, and our makefiles would be much cleaner too when you
>>> could just do
>>>
>>>  cflags-$(USE_RETPOLINE) += -mfunction-return=thunk 
>>> -mindirect-branch-table
>>>
>>> because the validity of the C compiler flag has been tested when 
>>> configuring.
>>>
>>> And then we could add that warning at configure time (or just disable
>>> the option there thanks to "depends on CC_HAS_xyz" logic).
>>>
>>> All our compiler option handling right now is just nasty nasty nasty crud.
>>>
>>> Adding more people in the hopes that somebody gets motivated.. I've
>>> talked about this before, so far we haven't made any progress.
>>
>>
>> Sorry for slow progress.
>>
>> I agreed this before, and still motivated.
>> (because I also motivated to remove kbuild cache.
>> This turned out not so clever as I first thought)
>>
>> I was trying to do this, but in this development cycle
>> I spent most of my time to flush out lots of piled up Kconfig patches.
>> Sorry.
>>
>> Unless somebody is working, I will.

I sent the first draft.

For people interested, I pushed the series in the following too.

git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
kconfig-opt-shell-rfc



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Kees Cook]

On Thu, Feb 8, 2018 at 10:44 AM, Masahiro Yamada
 wrote:
> 2018-02-08 2:55 GMT+09:00 Linus Torvalds :
>> What I would really want - and this is entirely unrelated to this
>> particular case - is to have those damn compiler option tests as part
>> of the config phase in general. We now have about a million of these
>> crazy things, where we have config options that simply depend on which
>> compiler we have, and we have no sane way to show them at
>> configuration time.
>>
>> Though Andrew's tree I got yet another ugly hack
>> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
>> by turning it into a special magic Kconfig entry in the main Makefile.
>> See commit 44c6dc940b19 ("Makefile: introduce
>> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
>> and honestly, I'm still thinking of just reverting it, because it's
>> _so_ ugly and _so_ wrong.
>>
>> What we need is an extension to the Kconfig language itself so that we can do
>>
>>config CC_HAS_RETPOLINE
>> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>>
>> or something. And then we can make sane _conditional_ dependencies at
>> Kconfig time, and our makefiles would be much cleaner too when you
>> could just do
>>
>>  cflags-$(USE_RETPOLINE) += -mfunction-return=thunk 
>> -mindirect-branch-table
>>
>> because the validity of the C compiler flag has been tested when configuring.
>>
>> And then we could add that warning at configure time (or just disable
>> the option there thanks to "depends on CC_HAS_xyz" logic).
>>
>> All our compiler option handling right now is just nasty nasty nasty crud.
>>
>> Adding more people in the hopes that somebody gets motivated.. I've
>> talked about this before, so far we haven't made any progress.
>
>
> Sorry for slow progress.
>
> I agreed this before, and still motivated.
> (because I also motivated to remove kbuild cache.
> This turned out not so clever as I first thought)
>
> I was trying to do this, but in this development cycle
> I spent most of my time to flush out lots of piled up Kconfig patches.
> Sorry.
>
> Unless somebody is working, I will.

FWIW, I did try to do this before I went with the STACKPROTECTOR_AUTO
solution, and it defeated me at every turn. Between the circular
dependency of the Makefile setting up KBUILD flags and Kconfig wanting
to know about the compiler, I got stuck. And it also seemed like the
cache was a problem too, as I couldn't find a way to re-evaluate the
script-controlled CONFIG items once it got cached.

And ultimately, a lot of the operational logic ended up sticking
around in the Makefile anyway (to provide fallback decisions, warns,
and errors). To correctly deal with the complex corner-cases for
stack-protector, I end up with three pieces:

the user config:

- do you want auto, weak, strong, or off?

the compiler tests:

- which of weak, strong, or off are supported?
- does the flag _actually produce working code_ for this
architecture/compiler version/etc?

and the build behavior:

- for auto, choose best available flag and warn if none
- for user-selected weak or strong, stop if unavailable/non-functional
- for off (or auto-none) use the "off" flag, if it is supported (to
force-disable it on ssp-by-default distro compilers)

And all of this needs to bypass the Kconfig cache, since if it gets
cached, it won't get re-evaluated when a selection is changed. If we
had a working "option shell $(CC) ..." then I could do the "compiler
tests" part in Kconfig, and I could probably do the bulk of the "build
behavior" logic in Kconfig too, though any intermediate configs would
need to avoid getting cached too.

The ssp handling has always been extremely complex due to all its gory
details, but I've tried _really_ hard to improve it, keep it
documented, and in one place (e.g. the compiler tests used to be
stuffed in per-arch Makefile).

-Kees

-- 
Kees Cook
Pixel Security

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Kees Cook]

On Thu, Feb 8, 2018 at 10:44 AM, Masahiro Yamada
 wrote:
> 2018-02-08 2:55 GMT+09:00 Linus Torvalds :
>> What I would really want - and this is entirely unrelated to this
>> particular case - is to have those damn compiler option tests as part
>> of the config phase in general. We now have about a million of these
>> crazy things, where we have config options that simply depend on which
>> compiler we have, and we have no sane way to show them at
>> configuration time.
>>
>> Though Andrew's tree I got yet another ugly hack
>> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
>> by turning it into a special magic Kconfig entry in the main Makefile.
>> See commit 44c6dc940b19 ("Makefile: introduce
>> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
>> and honestly, I'm still thinking of just reverting it, because it's
>> _so_ ugly and _so_ wrong.
>>
>> What we need is an extension to the Kconfig language itself so that we can do
>>
>>config CC_HAS_RETPOLINE
>> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>>
>> or something. And then we can make sane _conditional_ dependencies at
>> Kconfig time, and our makefiles would be much cleaner too when you
>> could just do
>>
>>  cflags-$(USE_RETPOLINE) += -mfunction-return=thunk 
>> -mindirect-branch-table
>>
>> because the validity of the C compiler flag has been tested when configuring.
>>
>> And then we could add that warning at configure time (or just disable
>> the option there thanks to "depends on CC_HAS_xyz" logic).
>>
>> All our compiler option handling right now is just nasty nasty nasty crud.
>>
>> Adding more people in the hopes that somebody gets motivated.. I've
>> talked about this before, so far we haven't made any progress.
>
>
> Sorry for slow progress.
>
> I agreed this before, and still motivated.
> (because I also motivated to remove kbuild cache.
> This turned out not so clever as I first thought)
>
> I was trying to do this, but in this development cycle
> I spent most of my time to flush out lots of piled up Kconfig patches.
> Sorry.
>
> Unless somebody is working, I will.

FWIW, I did try to do this before I went with the STACKPROTECTOR_AUTO
solution, and it defeated me at every turn. Between the circular
dependency of the Makefile setting up KBUILD flags and Kconfig wanting
to know about the compiler, I got stuck. And it also seemed like the
cache was a problem too, as I couldn't find a way to re-evaluate the
script-controlled CONFIG items once it got cached.

And ultimately, a lot of the operational logic ended up sticking
around in the Makefile anyway (to provide fallback decisions, warns,
and errors). To correctly deal with the complex corner-cases for
stack-protector, I end up with three pieces:

the user config:

- do you want auto, weak, strong, or off?

the compiler tests:

- which of weak, strong, or off are supported?
- does the flag _actually produce working code_ for this
architecture/compiler version/etc?

and the build behavior:

- for auto, choose best available flag and warn if none
- for user-selected weak or strong, stop if unavailable/non-functional
- for off (or auto-none) use the "off" flag, if it is supported (to
force-disable it on ssp-by-default distro compilers)

And all of this needs to bypass the Kconfig cache, since if it gets
cached, it won't get re-evaluated when a selection is changed. If we
had a working "option shell $(CC) ..." then I could do the "compiler
tests" part in Kconfig, and I could probably do the bulk of the "build
behavior" logic in Kconfig too, though any intermediate configs would
need to avoid getting cached too.

The ssp handling has always been extremely complex due to all its gory
details, but I've tried _really_ hard to improve it, keep it
documented, and in one place (e.g. the compiler tests used to be
stuffed in per-arch Makefile).

-Kees

-- 
Kees Cook
Pixel Security

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Linus Torvalds]

On Wed, Feb 7, 2018 at 3:44 PM, Masahiro Yamada
 wrote:
>
> I agreed this before, and still motivated.
> (because I also motivated to remove kbuild cache.

I actually wish I still had my old "run shell script" thing. I had
some very preliminary patches that actually worked for simple things,
and you could do something like

  config SOME_NAME
bool
option shell "true"

and it would act basically like "option env", except it didn't do
"getenv()", it did "system()" and checked the return value. So the
above would make SOME_NAME have the value 'y', because when you
executed "true" it was successful. I have this dim memory of allowing
it to set strings too (filling in the default value with the stdout
output from the shell execution).

So I had some experimental patch like that, and it kind of worked, but
I never finished it.

But the reason I never completed it was that for the compiler option
case, it really wanted more than a shell command, it needed to get the
whole $(CC) etc from the make environment.

I don't remember the exact syntax I used, but I think it was based on
that "option env" syntax, just replacing "env" with "shell". But
searching my mail archives I can't find anything, so I may never have
sent anything out. And so the patch is long gone.

Maybe I'll get frustrated enough and try to recreate it. I don't think
the patch was that big (but as mentioned, it really wasn't in a form
where it was _useful_ yet).

 Linus

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Linus Torvalds]

On Wed, Feb 7, 2018 at 3:44 PM, Masahiro Yamada
 wrote:
>
> I agreed this before, and still motivated.
> (because I also motivated to remove kbuild cache.

I actually wish I still had my old "run shell script" thing. I had
some very preliminary patches that actually worked for simple things,
and you could do something like

  config SOME_NAME
bool
option shell "true"

and it would act basically like "option env", except it didn't do
"getenv()", it did "system()" and checked the return value. So the
above would make SOME_NAME have the value 'y', because when you
executed "true" it was successful. I have this dim memory of allowing
it to set strings too (filling in the default value with the stdout
output from the shell execution).

So I had some experimental patch like that, and it kind of worked, but
I never finished it.

But the reason I never completed it was that for the compiler option
case, it really wanted more than a shell command, it needed to get the
whole $(CC) etc from the make environment.

I don't remember the exact syntax I used, but I think it was based on
that "option env" syntax, just replacing "env" with "shell". But
searching my mail archives I can't find anything, so I may never have
sent anything out. And so the patch is long gone.

Maybe I'll get frustrated enough and try to recreate it. I don't think
the patch was that big (but as mentioned, it really wasn't in a form
where it was _useful_ yet).

 Linus

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Masahiro Yamada]

2018-02-08 2:55 GMT+09:00 Linus Torvalds :
> On Wed, Feb 7, 2018 at 4:17 AM, Martin Schwidefsky
>  wrote:
>>> That isn't, though. Linus asked us to drop the $(warning) part.
>>>
>>> ... and then spent a week building with a non-retpoline compiler and
>>> not noticing, so he might have changed his mind ;)
>>
>> I found the warning to have some value, it helps for the case where my
>> fingers are faster than my brain and I type "make" instead of "smake"
>> which uses the alternative compiler with the required support.
>>
>> @Linus: do you want a warning or prefer not to have one ?
>
> Honestly, I think I'd be much happier with the warning as part of the
> "make config" phase.
>
> What really annoyed me was that it showed up at every build.
>
> What I would really want - and this is entirely unrelated to this
> particular case - is to have those damn compiler option tests as part
> of the config phase in general. We now have about a million of these
> crazy things, where we have config options that simply depend on which
> compiler we have, and we have no sane way to show them at
> configuration time.
>
> Though Andrew's tree I got yet another ugly hack
> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
> by turning it into a special magic Kconfig entry in the main Makefile.
> See commit 44c6dc940b19 ("Makefile: introduce
> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
> and honestly, I'm still thinking of just reverting it, because it's
> _so_ ugly and _so_ wrong.
>
> What we need is an extension to the Kconfig language itself so that we can do
>
>config CC_HAS_RETPOLINE
> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>
> or something. And then we can make sane _conditional_ dependencies at
> Kconfig time, and our makefiles would be much cleaner too when you
> could just do
>
>  cflags-$(USE_RETPOLINE) += -mfunction-return=thunk 
> -mindirect-branch-table
>
> because the validity of the C compiler flag has been tested when configuring.
>
> And then we could add that warning at configure time (or just disable
> the option there thanks to "depends on CC_HAS_xyz" logic).
>
> All our compiler option handling right now is just nasty nasty nasty crud.
>
> Adding more people in the hopes that somebody gets motivated.. I've
> talked about this before, so far we haven't made any progress.


Sorry for slow progress.

I agreed this before, and still motivated.
(because I also motivated to remove kbuild cache.
This turned out not so clever as I first thought)

I was trying to do this, but in this development cycle
I spent most of my time to flush out lots of piled up Kconfig patches.
Sorry.

Unless somebody is working, I will.



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Masahiro Yamada]

2018-02-08 2:55 GMT+09:00 Linus Torvalds :
> On Wed, Feb 7, 2018 at 4:17 AM, Martin Schwidefsky
>  wrote:
>>> That isn't, though. Linus asked us to drop the $(warning) part.
>>>
>>> ... and then spent a week building with a non-retpoline compiler and
>>> not noticing, so he might have changed his mind ;)
>>
>> I found the warning to have some value, it helps for the case where my
>> fingers are faster than my brain and I type "make" instead of "smake"
>> which uses the alternative compiler with the required support.
>>
>> @Linus: do you want a warning or prefer not to have one ?
>
> Honestly, I think I'd be much happier with the warning as part of the
> "make config" phase.
>
> What really annoyed me was that it showed up at every build.
>
> What I would really want - and this is entirely unrelated to this
> particular case - is to have those damn compiler option tests as part
> of the config phase in general. We now have about a million of these
> crazy things, where we have config options that simply depend on which
> compiler we have, and we have no sane way to show them at
> configuration time.
>
> Though Andrew's tree I got yet another ugly hack
> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
> by turning it into a special magic Kconfig entry in the main Makefile.
> See commit 44c6dc940b19 ("Makefile: introduce
> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
> and honestly, I'm still thinking of just reverting it, because it's
> _so_ ugly and _so_ wrong.
>
> What we need is an extension to the Kconfig language itself so that we can do
>
>config CC_HAS_RETPOLINE
> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>
> or something. And then we can make sane _conditional_ dependencies at
> Kconfig time, and our makefiles would be much cleaner too when you
> could just do
>
>  cflags-$(USE_RETPOLINE) += -mfunction-return=thunk 
> -mindirect-branch-table
>
> because the validity of the C compiler flag has been tested when configuring.
>
> And then we could add that warning at configure time (or just disable
> the option there thanks to "depends on CC_HAS_xyz" logic).
>
> All our compiler option handling right now is just nasty nasty nasty crud.
>
> Adding more people in the hopes that somebody gets motivated.. I've
> talked about this before, so far we haven't made any progress.


Sorry for slow progress.

I agreed this before, and still motivated.
(because I also motivated to remove kbuild cache.
This turned out not so clever as I first thought)

I was trying to do this, but in this development cycle
I spent most of my time to flush out lots of piled up Kconfig patches.
Sorry.

Unless somebody is working, I will.



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Linus Torvalds]

On Wed, Feb 7, 2018 at 4:17 AM, Martin Schwidefsky
 wrote:
>> That isn't, though. Linus asked us to drop the $(warning) part.
>>
>> ... and then spent a week building with a non-retpoline compiler and
>> not noticing, so he might have changed his mind ;)
>
> I found the warning to have some value, it helps for the case where my
> fingers are faster than my brain and I type "make" instead of "smake"
> which uses the alternative compiler with the required support.
>
> @Linus: do you want a warning or prefer not to have one ?

Honestly, I think I'd be much happier with the warning as part of the
"make config" phase.

What really annoyed me was that it showed up at every build.

What I would really want - and this is entirely unrelated to this
particular case - is to have those damn compiler option tests as part
of the config phase in general. We now have about a million of these
crazy things, where we have config options that simply depend on which
compiler we have, and we have no sane way to show them at
configuration time.

Though Andrew's tree I got yet another ugly hack
(CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
by turning it into a special magic Kconfig entry in the main Makefile.
See commit 44c6dc940b19 ("Makefile: introduce
CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
and honestly, I'm still thinking of just reverting it, because it's
_so_ ugly and _so_ wrong.

What we need is an extension to the Kconfig language itself so that we can do

   config CC_HAS_RETPOLINE
cc_option "-mindirect-branch=thunk -mindirect-branch-table"

or something. And then we can make sane _conditional_ dependencies at
Kconfig time, and our makefiles would be much cleaner too when you
could just do

 cflags-$(USE_RETPOLINE) += -mfunction-return=thunk -mindirect-branch-table

because the validity of the C compiler flag has been tested when configuring.

And then we could add that warning at configure time (or just disable
the option there thanks to "depends on CC_HAS_xyz" logic).

All our compiler option handling right now is just nasty nasty nasty crud.

Adding more people in the hopes that somebody gets motivated.. I've
talked about this before, so far we haven't made any progress.

  Linus

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Linus Torvalds]

On Wed, Feb 7, 2018 at 4:17 AM, Martin Schwidefsky
 wrote:
>> That isn't, though. Linus asked us to drop the $(warning) part.
>>
>> ... and then spent a week building with a non-retpoline compiler and
>> not noticing, so he might have changed his mind ;)
>
> I found the warning to have some value, it helps for the case where my
> fingers are faster than my brain and I type "make" instead of "smake"
> which uses the alternative compiler with the required support.
>
> @Linus: do you want a warning or prefer not to have one ?

Honestly, I think I'd be much happier with the warning as part of the
"make config" phase.

What really annoyed me was that it showed up at every build.

What I would really want - and this is entirely unrelated to this
particular case - is to have those damn compiler option tests as part
of the config phase in general. We now have about a million of these
crazy things, where we have config options that simply depend on which
compiler we have, and we have no sane way to show them at
configuration time.

Though Andrew's tree I got yet another ugly hack
(CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
by turning it into a special magic Kconfig entry in the main Makefile.
See commit 44c6dc940b19 ("Makefile: introduce
CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
and honestly, I'm still thinking of just reverting it, because it's
_so_ ugly and _so_ wrong.

What we need is an extension to the Kconfig language itself so that we can do

   config CC_HAS_RETPOLINE
cc_option "-mindirect-branch=thunk -mindirect-branch-table"

or something. And then we can make sane _conditional_ dependencies at
Kconfig time, and our makefiles would be much cleaner too when you
could just do

 cflags-$(USE_RETPOLINE) += -mfunction-return=thunk -mindirect-branch-table

because the validity of the C compiler flag has been tested when configuring.

And then we could add that warning at configure time (or just disable
the option there thanks to "depends on CC_HAS_xyz" logic).

All our compiler option handling right now is just nasty nasty nasty crud.

Adding more people in the hopes that somebody gets motivated.. I've
talked about this before, so far we haven't made any progress.

  Linus

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[David Woodhouse]

On Wed, 2018-02-07 at 13:17 +0100, Martin Schwidefsky wrote:
> On Wed, 07 Feb 2018 12:07:55 +
> David Woodhouse  wrote:
> 
> > 
> > On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> > > 
> > > This is really unfortunate naming of kernel option.
> > > 
> > > spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> > > suspect you are turning the bug _workaround_ off.  
> > That's consistent with what we have on x86.
> > 
> > > 
> > > > 
> > > > +ifdef CONFIG_EXPOLINE
> > > > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) 
> > > > -mindirect-branch=thunk),y)
> > > > +CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > > > +CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > > > +CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > > > +export CC_FLAGS_EXPOLINE
> > > > +cflags-y += $(CC_FLAGS_EXPOLINE)
> > > > +  else
> > > > +$(warning "Your gcc lacks the -mindirect-branch= option")
> > > > +  endif
> > > > +endif  
> > That isn't, though. Linus asked us to drop the $(warning) part.
> > 
> > ... and then spent a week building with a non-retpoline compiler and
> > not noticing, so he might have changed his mind ;)
>
> I found the warning to have some value, it helps for the case where my
> fingers are faster than my brain and I type "make" instead of "smake"
> which uses the alternative compiler with the required support.
> 
> @Linus: do you want a warning or prefer not to have one ?

FWIW I agreed to drop it when the plan in my head was "we'll just turn
on IBRS instead if the compiler doesn't do full retpoline support".

Now that Linus has expressed a disinclination to take IBRS support in
that form, I might be more inclined to defend the $(warning) too.

smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[David Woodhouse]

On Wed, 2018-02-07 at 13:17 +0100, Martin Schwidefsky wrote:
> On Wed, 07 Feb 2018 12:07:55 +
> David Woodhouse  wrote:
> 
> > 
> > On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> > > 
> > > This is really unfortunate naming of kernel option.
> > > 
> > > spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> > > suspect you are turning the bug _workaround_ off.  
> > That's consistent with what we have on x86.
> > 
> > > 
> > > > 
> > > > +ifdef CONFIG_EXPOLINE
> > > > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) 
> > > > -mindirect-branch=thunk),y)
> > > > +CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > > > +CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > > > +CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > > > +export CC_FLAGS_EXPOLINE
> > > > +cflags-y += $(CC_FLAGS_EXPOLINE)
> > > > +  else
> > > > +$(warning "Your gcc lacks the -mindirect-branch= option")
> > > > +  endif
> > > > +endif  
> > That isn't, though. Linus asked us to drop the $(warning) part.
> > 
> > ... and then spent a week building with a non-retpoline compiler and
> > not noticing, so he might have changed his mind ;)
>
> I found the warning to have some value, it helps for the case where my
> fingers are faster than my brain and I type "make" instead of "smake"
> which uses the alternative compiler with the required support.
> 
> @Linus: do you want a warning or prefer not to have one ?

FWIW I agreed to drop it when the plan in my head was "we'll just turn
on IBRS instead if the compiler doesn't do full retpoline support".

Now that Linus has expressed a disinclination to take IBRS support in
that form, I might be more inclined to defend the $(warning) too.

smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Martin Schwidefsky]

On Wed, 07 Feb 2018 12:07:55 +
David Woodhouse  wrote:

> On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> > This is really unfortunate naming of kernel option.
> > 
> > spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> > suspect you are turning the bug _workaround_ off.  
> 
> That's consistent with what we have on x86.
> 
> > > +ifdef CONFIG_EXPOLINE
> > > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> > > +CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > > +CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > > +CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > > +export CC_FLAGS_EXPOLINE
> > > +cflags-y += $(CC_FLAGS_EXPOLINE)
> > > +  else
> > > +$(warning "Your gcc lacks the -mindirect-branch= option")
> > > +  endif
> > > +endif  
> 
> That isn't, though. Linus asked us to drop the $(warning) part.
> 
> ... and then spent a week building with a non-retpoline compiler and
> not noticing, so he might have changed his mind ;)

I found the warning to have some value, it helps for the case where my
fingers are faster than my brain and I type "make" instead of "smake"
which uses the alternative compiler with the required support.

@Linus: do you want a warning or prefer not to have one ?

-- 
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Martin Schwidefsky]

On Wed, 07 Feb 2018 12:07:55 +
David Woodhouse  wrote:

> On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> > This is really unfortunate naming of kernel option.
> > 
> > spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> > suspect you are turning the bug _workaround_ off.  
> 
> That's consistent with what we have on x86.
> 
> > > +ifdef CONFIG_EXPOLINE
> > > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> > > +CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > > +CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > > +CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > > +export CC_FLAGS_EXPOLINE
> > > +cflags-y += $(CC_FLAGS_EXPOLINE)
> > > +  else
> > > +$(warning "Your gcc lacks the -mindirect-branch= option")
> > > +  endif
> > > +endif  
> 
> That isn't, though. Linus asked us to drop the $(warning) part.
> 
> ... and then spent a week building with a non-retpoline compiler and
> not noticing, so he might have changed his mind ;)

I found the warning to have some value, it helps for the case where my
fingers are faster than my brain and I type "make" instead of "smake"
which uses the alternative compiler with the required support.

@Linus: do you want a warning or prefer not to have one ?

-- 
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[David Woodhouse]

On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> This is really unfortunate naming of kernel option.
> 
> spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> suspect you are turning the bug _workaround_ off.

That's consistent with what we have on x86.

> > +ifdef CONFIG_EXPOLINE
> > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> > +CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > +CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > +CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > +export CC_FLAGS_EXPOLINE
> > +cflags-y += $(CC_FLAGS_EXPOLINE)
> > +  else
> > +$(warning "Your gcc lacks the -mindirect-branch= option")
> > +  endif
> > +endif

That isn't, though. Linus asked us to drop the $(warning) part.

... and then spent a week building with a non-retpoline compiler and
not noticing, so he might have changed his mind ;)


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[David Woodhouse]

On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> This is really unfortunate naming of kernel option.
> 
> spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> suspect you are turning the bug _workaround_ off.

That's consistent with what we have on x86.

> > +ifdef CONFIG_EXPOLINE
> > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> > +CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > +CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > +CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > +export CC_FLAGS_EXPOLINE
> > +cflags-y += $(CC_FLAGS_EXPOLINE)
> > +  else
> > +$(warning "Your gcc lacks the -mindirect-branch= option")
> > +  endif
> > +endif

That isn't, though. Linus asked us to drop the $(warning) part.

... and then spent a week building with a non-retpoline compiler and
not noticing, so he might have changed his mind ;)


smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Martin Schwidefsky]

On Wed, 7 Feb 2018 11:07:26 +0100
Pavel Machek  wrote:

> On Wed 2018-02-07 08:00:11, Martin Schwidefsky wrote:
> > Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
> > -mfunction_return= compiler options to create a kernel fortified against
> > the specte v2 attack.
> > 
> > With CONFIG_EXPOLINE=y all indirect branches will be issued with an
> > execute type instruction. For z10 or newer the EXRL instruction will
> > be used, for older machines the EX instruction. The typical indirect
> > call
> > 
> > basr%r14,%r1
> > 
> > is replaced with a PC relative call to a new thunk
> > 
> > brasl   %r14,__s390x_indirect_jump_r1
> > 
> > The thunk contains the EXRL/EX instruction to the indirect branch
> > 
> > __s390x_indirect_jump_r1:
> > exrl0,0f
> > j   .
> > 0:  br  %r1
> > 
> > The detour via the execute type instruction has a performance impact.
> > To get rid of the detour the new kernel parameter "nospectre_v2" and
> > "spectre_v2=[on,off,auto]" can be used. If the parameter is specified
> > the kernel and module code will be patched at runtime.  
> 
> This is really unfortunate naming of kernel option.
> 
> spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> suspect you are turning the bug _workaround_ off.
 
Well, that is the 1:1 copy of the x86 option. Do you want to change that
one as well?

-- 
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Martin Schwidefsky]

On Wed, 7 Feb 2018 11:07:26 +0100
Pavel Machek  wrote:

> On Wed 2018-02-07 08:00:11, Martin Schwidefsky wrote:
> > Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
> > -mfunction_return= compiler options to create a kernel fortified against
> > the specte v2 attack.
> > 
> > With CONFIG_EXPOLINE=y all indirect branches will be issued with an
> > execute type instruction. For z10 or newer the EXRL instruction will
> > be used, for older machines the EX instruction. The typical indirect
> > call
> > 
> > basr%r14,%r1
> > 
> > is replaced with a PC relative call to a new thunk
> > 
> > brasl   %r14,__s390x_indirect_jump_r1
> > 
> > The thunk contains the EXRL/EX instruction to the indirect branch
> > 
> > __s390x_indirect_jump_r1:
> > exrl0,0f
> > j   .
> > 0:  br  %r1
> > 
> > The detour via the execute type instruction has a performance impact.
> > To get rid of the detour the new kernel parameter "nospectre_v2" and
> > "spectre_v2=[on,off,auto]" can be used. If the parameter is specified
> > the kernel and module code will be patched at runtime.  
> 
> This is really unfortunate naming of kernel option.
> 
> spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> suspect you are turning the bug _workaround_ off.
 
Well, that is the 1:1 copy of the x86 option. Do you want to change that
one as well?

-- 
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Pavel Machek]

On Wed 2018-02-07 08:00:11, Martin Schwidefsky wrote:
> Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
> -mfunction_return= compiler options to create a kernel fortified against
> the specte v2 attack.
> 
> With CONFIG_EXPOLINE=y all indirect branches will be issued with an
> execute type instruction. For z10 or newer the EXRL instruction will
> be used, for older machines the EX instruction. The typical indirect
> call
> 
>   basr%r14,%r1
> 
> is replaced with a PC relative call to a new thunk
> 
>   brasl   %r14,__s390x_indirect_jump_r1
> 
> The thunk contains the EXRL/EX instruction to the indirect branch
> 
> __s390x_indirect_jump_r1:
>   exrl0,0f
>   j   .
> 0:br  %r1
> 
> The detour via the execute type instruction has a performance impact.
> To get rid of the detour the new kernel parameter "nospectre_v2" and
> "spectre_v2=[on,off,auto]" can be used. If the parameter is specified
> the kernel and module code will be patched at runtime.

This is really unfortunate naming of kernel option.

spectre_v2=off sounds like we are turning the "bug" off, but i somehow
suspect you are turning the bug _workaround_ off.

Pavel

> Signed-off-by: Martin Schwidefsky 
> ---
>  arch/s390/Kconfig |  28 +
>  arch/s390/Makefile|  12 
>  arch/s390/include/asm/lowcore.h   |   6 +-
>  arch/s390/include/asm/nospec-branch.h |  18 ++
>  arch/s390/kernel/Makefile |   4 ++
>  arch/s390/kernel/entry.S  | 113 
> ++
>  arch/s390/kernel/module.c |  62 ---
>  arch/s390/kernel/nospec-branch.c  | 100 ++
>  arch/s390/kernel/setup.c  |   4 ++
>  arch/s390/kernel/smp.c|   1 +
>  arch/s390/kernel/vmlinux.lds.S|  14 +
>  drivers/s390/char/Makefile|   2 +
>  12 files changed, 329 insertions(+), 35 deletions(-)
>  create mode 100644 arch/s390/include/asm/nospec-branch.h
>  create mode 100644 arch/s390/kernel/nospec-branch.c
> 
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index d514e25..d4a65bf 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -557,6 +557,34 @@ config KERNEL_NOBP
>  
> If unsure, say N.
>  
> +config EXPOLINE
> + def_bool n
> + prompt "Avoid speculative indirect branches in the kernel"
> + help
> +   Compile the kernel with the expoline compiler options to guard
> +   against kernel-to-user data leaks by avoiding speculative indirect
> +   branches.
> +   Requires a compiler with -mindirect-branch=thunk support for full
> +   protection. The kernel may run slower.
> +
> +   If unsure, say N.
> +
> +choice
> + prompt "Expoline default"
> + depends on EXPOLINE
> + default EXPOLINE_FULL
> +
> +config EXPOLINE_OFF
> + bool "spectre_v2=off"
> +
> +config EXPOLINE_MEDIUM
> + bool "spectre_v2=auto"
> +
> +config EXPOLINE_FULL
> + bool "spectre_v2=on"
> +
> +endchoice
> +
>  endmenu
>  
>  menu "Memory setup"
> diff --git a/arch/s390/Makefile b/arch/s390/Makefile
> index fd691c4..2f925ef 100644
> --- a/arch/s390/Makefile
> +++ b/arch/s390/Makefile
> @@ -78,6 +78,18 @@ ifeq ($(call cc-option-yn,-mwarn-dynamicstack),y)
>  cflags-$(CONFIG_WARN_DYNAMIC_STACK) += -mwarn-dynamicstack
>  endif
>  
> +ifdef CONFIG_EXPOLINE
> +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> +CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> +CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> +CC_FLAGS_EXPOLINE += -mindirect-branch-table
> +export CC_FLAGS_EXPOLINE
> +cflags-y += $(CC_FLAGS_EXPOLINE)
> +  else
> +$(warning "Your gcc lacks the -mindirect-branch= option")
> +  endif
> +endif
> +
>  ifdef CONFIG_FUNCTION_TRACER
>  # make use of hotpatch feature if the compiler supports it
>  cc_hotpatch  := -mhotpatch=0,3
> diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcore.h
> index c63986a..5bc 100644
> --- a/arch/s390/include/asm/lowcore.h
> +++ b/arch/s390/include/asm/lowcore.h
> @@ -136,7 +136,11 @@ struct lowcore {
>   __u64   vdso_per_cpu_data;  /* 0x03b8 */
>   __u64   machine_flags;  /* 0x03c0 */
>   __u64   gmap;   /* 0x03c8 */
> - __u8pad_0x03d0[0x0e00-0x03d0];  /* 0x03d0 */
> + __u8pad_0x03d0[0x0400-0x03d0];  /* 0x03d0 */
> +
> + /* br %r1 trampoline */
> + __u16   br_r1_trampoline;   /* 0x0400 */
> + __u8pad_0x0402[0x0e00-0x0402];  /* 0x0402 */
>  
>   /*
>* 0xe00 contains the address of the IPL Parameter Information
> diff --git a/arch/s390/include/asm/nospec-branch.h 
> b/arch/s390/include/asm/nospec-branch.h
> new file mode 100644
> index 000..7df48e5
> --- /dev/null
> 

Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

[Pavel Machek]

On Wed 2018-02-07 08:00:11, Martin Schwidefsky wrote:
> Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
> -mfunction_return= compiler options to create a kernel fortified against
> the specte v2 attack.
> 
> With CONFIG_EXPOLINE=y all indirect branches will be issued with an
> execute type instruction. For z10 or newer the EXRL instruction will
> be used, for older machines the EX instruction. The typical indirect
> call
> 
>   basr%r14,%r1
> 
> is replaced with a PC relative call to a new thunk
> 
>   brasl   %r14,__s390x_indirect_jump_r1
> 
> The thunk contains the EXRL/EX instruction to the indirect branch
> 
> __s390x_indirect_jump_r1:
>   exrl0,0f
>   j   .
> 0:br  %r1
> 
> The detour via the execute type instruction has a performance impact.
> To get rid of the detour the new kernel parameter "nospectre_v2" and
> "spectre_v2=[on,off,auto]" can be used. If the parameter is specified
> the kernel and module code will be patched at runtime.

This is really unfortunate naming of kernel option.

spectre_v2=off sounds like we are turning the "bug" off, but i somehow
suspect you are turning the bug _workaround_ off.

Pavel

> Signed-off-by: Martin Schwidefsky 
> ---
>  arch/s390/Kconfig |  28 +
>  arch/s390/Makefile|  12 
>  arch/s390/include/asm/lowcore.h   |   6 +-
>  arch/s390/include/asm/nospec-branch.h |  18 ++
>  arch/s390/kernel/Makefile |   4 ++
>  arch/s390/kernel/entry.S  | 113 
> ++
>  arch/s390/kernel/module.c |  62 ---
>  arch/s390/kernel/nospec-branch.c  | 100 ++
>  arch/s390/kernel/setup.c  |   4 ++
>  arch/s390/kernel/smp.c|   1 +
>  arch/s390/kernel/vmlinux.lds.S|  14 +
>  drivers/s390/char/Makefile|   2 +
>  12 files changed, 329 insertions(+), 35 deletions(-)
>  create mode 100644 arch/s390/include/asm/nospec-branch.h
>  create mode 100644 arch/s390/kernel/nospec-branch.c
> 
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index d514e25..d4a65bf 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -557,6 +557,34 @@ config KERNEL_NOBP
>  
> If unsure, say N.
>  
> +config EXPOLINE
> + def_bool n
> + prompt "Avoid speculative indirect branches in the kernel"
> + help
> +   Compile the kernel with the expoline compiler options to guard
> +   against kernel-to-user data leaks by avoiding speculative indirect
> +   branches.
> +   Requires a compiler with -mindirect-branch=thunk support for full
> +   protection. The kernel may run slower.
> +
> +   If unsure, say N.
> +
> +choice
> + prompt "Expoline default"
> + depends on EXPOLINE
> + default EXPOLINE_FULL
> +
> +config EXPOLINE_OFF
> + bool "spectre_v2=off"
> +
> +config EXPOLINE_MEDIUM
> + bool "spectre_v2=auto"
> +
> +config EXPOLINE_FULL
> + bool "spectre_v2=on"
> +
> +endchoice
> +
>  endmenu
>  
>  menu "Memory setup"
> diff --git a/arch/s390/Makefile b/arch/s390/Makefile
> index fd691c4..2f925ef 100644
> --- a/arch/s390/Makefile
> +++ b/arch/s390/Makefile
> @@ -78,6 +78,18 @@ ifeq ($(call cc-option-yn,-mwarn-dynamicstack),y)
>  cflags-$(CONFIG_WARN_DYNAMIC_STACK) += -mwarn-dynamicstack
>  endif
>  
> +ifdef CONFIG_EXPOLINE
> +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> +CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> +CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> +CC_FLAGS_EXPOLINE += -mindirect-branch-table
> +export CC_FLAGS_EXPOLINE
> +cflags-y += $(CC_FLAGS_EXPOLINE)
> +  else
> +$(warning "Your gcc lacks the -mindirect-branch= option")
> +  endif
> +endif
> +
>  ifdef CONFIG_FUNCTION_TRACER
>  # make use of hotpatch feature if the compiler supports it
>  cc_hotpatch  := -mhotpatch=0,3
> diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcore.h
> index c63986a..5bc 100644
> --- a/arch/s390/include/asm/lowcore.h
> +++ b/arch/s390/include/asm/lowcore.h
> @@ -136,7 +136,11 @@ struct lowcore {
>   __u64   vdso_per_cpu_data;  /* 0x03b8 */
>   __u64   machine_flags;  /* 0x03c0 */
>   __u64   gmap;   /* 0x03c8 */
> - __u8pad_0x03d0[0x0e00-0x03d0];  /* 0x03d0 */
> + __u8pad_0x03d0[0x0400-0x03d0];  /* 0x03d0 */
> +
> + /* br %r1 trampoline */
> + __u16   br_r1_trampoline;   /* 0x0400 */
> + __u8pad_0x0402[0x0e00-0x0402];  /* 0x0402 */
>  
>   /*
>* 0xe00 contains the address of the IPL Parameter Information
> diff --git a/arch/s390/include/asm/nospec-branch.h 
> b/arch/s390/include/asm/nospec-branch.h
> new file mode 100644
> index 000..7df48e5
> --- /dev/null
> +++ 

[PATCH 6/6] s390: introduce execute-trampolines for branches

[Martin Schwidefsky]

Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
-mfunction_return= compiler options to create a kernel fortified against
the specte v2 attack.

With CONFIG_EXPOLINE=y all indirect branches will be issued with an
execute type instruction. For z10 or newer the EXRL instruction will
be used, for older machines the EX instruction. The typical indirect
call

basr%r14,%r1

is replaced with a PC relative call to a new thunk

brasl   %r14,__s390x_indirect_jump_r1

The thunk contains the EXRL/EX instruction to the indirect branch

__s390x_indirect_jump_r1:
exrl0,0f
j   .
0:  br  %r1

The detour via the execute type instruction has a performance impact.
To get rid of the detour the new kernel parameter "nospectre_v2" and
"spectre_v2=[on,off,auto]" can be used. If the parameter is specified
the kernel and module code will be patched at runtime.

Signed-off-by: Martin Schwidefsky 
---
 arch/s390/Kconfig |  28 +
 arch/s390/Makefile|  12 
 arch/s390/include/asm/lowcore.h   |   6 +-
 arch/s390/include/asm/nospec-branch.h |  18 ++
 arch/s390/kernel/Makefile |   4 ++
 arch/s390/kernel/entry.S  | 113 ++
 arch/s390/kernel/module.c |  62 ---
 arch/s390/kernel/nospec-branch.c  | 100 ++
 arch/s390/kernel/setup.c  |   4 ++
 arch/s390/kernel/smp.c|   1 +
 arch/s390/kernel/vmlinux.lds.S|  14 +
 drivers/s390/char/Makefile|   2 +
 12 files changed, 329 insertions(+), 35 deletions(-)
 create mode 100644 arch/s390/include/asm/nospec-branch.h
 create mode 100644 arch/s390/kernel/nospec-branch.c

diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index d514e25..d4a65bf 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -557,6 +557,34 @@ config KERNEL_NOBP
 
  If unsure, say N.
 
+config EXPOLINE
+   def_bool n
+   prompt "Avoid speculative indirect branches in the kernel"
+   help
+ Compile the kernel with the expoline compiler options to guard
+ against kernel-to-user data leaks by avoiding speculative indirect
+ branches.
+ Requires a compiler with -mindirect-branch=thunk support for full
+ protection. The kernel may run slower.
+
+ If unsure, say N.
+
+choice
+   prompt "Expoline default"
+   depends on EXPOLINE
+   default EXPOLINE_FULL
+
+config EXPOLINE_OFF
+   bool "spectre_v2=off"
+
+config EXPOLINE_MEDIUM
+   bool "spectre_v2=auto"
+
+config EXPOLINE_FULL
+   bool "spectre_v2=on"
+
+endchoice
+
 endmenu
 
 menu "Memory setup"
diff --git a/arch/s390/Makefile b/arch/s390/Makefile
index fd691c4..2f925ef 100644
--- a/arch/s390/Makefile
+++ b/arch/s390/Makefile
@@ -78,6 +78,18 @@ ifeq ($(call cc-option-yn,-mwarn-dynamicstack),y)
 cflags-$(CONFIG_WARN_DYNAMIC_STACK) += -mwarn-dynamicstack
 endif
 
+ifdef CONFIG_EXPOLINE
+  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
+CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
+CC_FLAGS_EXPOLINE += -mfunction-return=thunk
+CC_FLAGS_EXPOLINE += -mindirect-branch-table
+export CC_FLAGS_EXPOLINE
+cflags-y += $(CC_FLAGS_EXPOLINE)
+  else
+$(warning "Your gcc lacks the -mindirect-branch= option")
+  endif
+endif
+
 ifdef CONFIG_FUNCTION_TRACER
 # make use of hotpatch feature if the compiler supports it
 cc_hotpatch:= -mhotpatch=0,3
diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcore.h
index c63986a..5bc 100644
--- a/arch/s390/include/asm/lowcore.h
+++ b/arch/s390/include/asm/lowcore.h
@@ -136,7 +136,11 @@ struct lowcore {
__u64   vdso_per_cpu_data;  /* 0x03b8 */
__u64   machine_flags;  /* 0x03c0 */
__u64   gmap;   /* 0x03c8 */
-   __u8pad_0x03d0[0x0e00-0x03d0];  /* 0x03d0 */
+   __u8pad_0x03d0[0x0400-0x03d0];  /* 0x03d0 */
+
+   /* br %r1 trampoline */
+   __u16   br_r1_trampoline;   /* 0x0400 */
+   __u8pad_0x0402[0x0e00-0x0402];  /* 0x0402 */
 
/*
 * 0xe00 contains the address of the IPL Parameter Information
diff --git a/arch/s390/include/asm/nospec-branch.h 
b/arch/s390/include/asm/nospec-branch.h
new file mode 100644
index 000..7df48e5
--- /dev/null
+++ b/arch/s390/include/asm/nospec-branch.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_EXPOLINE_H
+#define _ASM_S390_EXPOLINE_H
+
+#ifndef __ASSEMBLY__
+
+#include 
+
+extern int nospec_call_disable;
+extern int nospec_return_disable;
+
+void nospec_init_branches(void);
+void nospec_call_revert(s32 *start, s32 *end);
+void nospec_return_revert(s32 *start, s32 *end);
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_S390_EXPOLINE_H */
diff --git a/arch/s390/kernel/Makefile 

[PATCH 6/6] s390: introduce execute-trampolines for branches

[Martin Schwidefsky]

Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
-mfunction_return= compiler options to create a kernel fortified against
the specte v2 attack.

With CONFIG_EXPOLINE=y all indirect branches will be issued with an
execute type instruction. For z10 or newer the EXRL instruction will
be used, for older machines the EX instruction. The typical indirect
call

basr%r14,%r1

is replaced with a PC relative call to a new thunk

brasl   %r14,__s390x_indirect_jump_r1

The thunk contains the EXRL/EX instruction to the indirect branch

__s390x_indirect_jump_r1:
exrl0,0f
j   .
0:  br  %r1

The detour via the execute type instruction has a performance impact.
To get rid of the detour the new kernel parameter "nospectre_v2" and
"spectre_v2=[on,off,auto]" can be used. If the parameter is specified
the kernel and module code will be patched at runtime.

Signed-off-by: Martin Schwidefsky 
---
 arch/s390/Kconfig |  28 +
 arch/s390/Makefile|  12 
 arch/s390/include/asm/lowcore.h   |   6 +-
 arch/s390/include/asm/nospec-branch.h |  18 ++
 arch/s390/kernel/Makefile |   4 ++
 arch/s390/kernel/entry.S  | 113 ++
 arch/s390/kernel/module.c |  62 ---
 arch/s390/kernel/nospec-branch.c  | 100 ++
 arch/s390/kernel/setup.c  |   4 ++
 arch/s390/kernel/smp.c|   1 +
 arch/s390/kernel/vmlinux.lds.S|  14 +
 drivers/s390/char/Makefile|   2 +
 12 files changed, 329 insertions(+), 35 deletions(-)
 create mode 100644 arch/s390/include/asm/nospec-branch.h
 create mode 100644 arch/s390/kernel/nospec-branch.c

diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index d514e25..d4a65bf 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -557,6 +557,34 @@ config KERNEL_NOBP
 
  If unsure, say N.
 
+config EXPOLINE
+   def_bool n
+   prompt "Avoid speculative indirect branches in the kernel"
+   help
+ Compile the kernel with the expoline compiler options to guard
+ against kernel-to-user data leaks by avoiding speculative indirect
+ branches.
+ Requires a compiler with -mindirect-branch=thunk support for full
+ protection. The kernel may run slower.
+
+ If unsure, say N.
+
+choice
+   prompt "Expoline default"
+   depends on EXPOLINE
+   default EXPOLINE_FULL
+
+config EXPOLINE_OFF
+   bool "spectre_v2=off"
+
+config EXPOLINE_MEDIUM
+   bool "spectre_v2=auto"
+
+config EXPOLINE_FULL
+   bool "spectre_v2=on"
+
+endchoice
+
 endmenu
 
 menu "Memory setup"
diff --git a/arch/s390/Makefile b/arch/s390/Makefile
index fd691c4..2f925ef 100644
--- a/arch/s390/Makefile
+++ b/arch/s390/Makefile
@@ -78,6 +78,18 @@ ifeq ($(call cc-option-yn,-mwarn-dynamicstack),y)
 cflags-$(CONFIG_WARN_DYNAMIC_STACK) += -mwarn-dynamicstack
 endif
 
+ifdef CONFIG_EXPOLINE
+  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
+CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
+CC_FLAGS_EXPOLINE += -mfunction-return=thunk
+CC_FLAGS_EXPOLINE += -mindirect-branch-table
+export CC_FLAGS_EXPOLINE
+cflags-y += $(CC_FLAGS_EXPOLINE)
+  else
+$(warning "Your gcc lacks the -mindirect-branch= option")
+  endif
+endif
+
 ifdef CONFIG_FUNCTION_TRACER
 # make use of hotpatch feature if the compiler supports it
 cc_hotpatch:= -mhotpatch=0,3
diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcore.h
index c63986a..5bc 100644
--- a/arch/s390/include/asm/lowcore.h
+++ b/arch/s390/include/asm/lowcore.h
@@ -136,7 +136,11 @@ struct lowcore {
__u64   vdso_per_cpu_data;  /* 0x03b8 */
__u64   machine_flags;  /* 0x03c0 */
__u64   gmap;   /* 0x03c8 */
-   __u8pad_0x03d0[0x0e00-0x03d0];  /* 0x03d0 */
+   __u8pad_0x03d0[0x0400-0x03d0];  /* 0x03d0 */
+
+   /* br %r1 trampoline */
+   __u16   br_r1_trampoline;   /* 0x0400 */
+   __u8pad_0x0402[0x0e00-0x0402];  /* 0x0402 */
 
/*
 * 0xe00 contains the address of the IPL Parameter Information
diff --git a/arch/s390/include/asm/nospec-branch.h 
b/arch/s390/include/asm/nospec-branch.h
new file mode 100644
index 000..7df48e5
--- /dev/null
+++ b/arch/s390/include/asm/nospec-branch.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_EXPOLINE_H
+#define _ASM_S390_EXPOLINE_H
+
+#ifndef __ASSEMBLY__
+
+#include 
+
+extern int nospec_call_disable;
+extern int nospec_return_disable;
+
+void nospec_init_branches(void);
+void nospec_call_revert(s32 *start, s32 *end);
+void nospec_return_revert(s32 *start, s32 *end);
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_S390_EXPOLINE_H */
diff --git a/arch/s390/kernel/Makefile