Re: [gentoo-user] microcode applied?

[Rich Freeman]

On Tue, Jan 9, 2018 at 4:26 AM, Wols Lists  wrote:
> On 08/01/18 13:52, Rich Freeman wrote:
>> There is also a lot of discussion on lkml about the right fix.  We
>> might very well end up seeing both AMD- and Intel-specific fixes with
>> conditional logic.  The two vendors don't really seem to be
>> coordinating on this.  Intel is pushing patches that apparently don't
>> work on AMD, or aren't necessary on AMD.  AMD for its part hasn't been
>> pushing much in public at all, but has made a few list posts, and they
>> have that mystery microcode update.
>
> Probably not much point co-operating. The *internals* of AMD chips and
> Intel chips are very different. I suspect both of them have a RISC core
> with an x86 instruction set interpretation layer, but that's where the
> similarities end ...
>

The fact that their internals are very different is EXACTLY why they
need to cooperate.

Spectre cannot be remediated in existing CPUs with a microcode update
only.  It might not even be possible to completely fix it in future
CPUs with only a hardware change.  Spectre remediation (at least at
present) requires software changes as well.

Software is written to the ISA, and both Intel and AMD share a common
ISA for the most part.  It is impossible for a programmer to know how
the internals of the CPU actually work, so they write their code to
the ISA specifications.  If the ISA says to insert an LFENCE
instruction immediately after every bounds check then programmers (or
at least compiler designers) will do just that.  If it says to insert
a CPUID instruction after every bounds check then that is what
programmers/compilers will do.

However, if one of the major vendors tells everybody to do one thing,
and the other tells everybody to do another, and each writes their
microcode fixes to work only their way, then programmers are stuck
trying to reconcile them.  The two vendor CPUs are no longer truly
instruction-set compatible for typical software.  Adding a lot of
conditional branching anytime there is a bounds check to try to detect
the CPU and deal with it isn't a great workaround either, because
branching is what causes the problem in the first place - it would be
better to determine the correct fix at compile-time and not runtime.

Intel and AMD don't need to agree on how to fix the internals of their
CPUs.  What they do need to agree on is the changes that need to be
made in the ISA.

Since AMD has been relatively quiet I suspect they intend to just let
Intel define the fix, and then quietly patch their CPUs to accept the
same fix, or at least to not have any issues when the Intel fix is
used.  However, the fact that they quietly issued a microcode update
doesn't go along with that, especially since they haven't provided any
clarification on what it does, or whether anything needs to be done
with software to take advantage of it.  Intel, while not being all
that cooperative, has at least issued unambiguous statements on what
needs to be done to remediate everything.

-- 
Rich

Re: [gentoo-user] microcode applied?

[Adam Carter]

>
> Hopefully there's an equivalent for AMD.
>

Here's what I came up with. This is very hacky and unreliable, but get the
CPUID with;

cpuid -r | grep "0x0001 0x00" | awk '{ print $3}' | uniq | cut -d x -f 3

then grab MCE (thanks Max for the suggestion) from
https://github.com/platomav/MCExtractor

unzip MCExtractor-master.zip
cd MCExtractor-master
chmod +x MCE.py
dos2unix MCE.py
./MCE.py /lib/firmware/amd-ucode/*bin
(at this stage it complained about missing dev-python/colorama and
dev-python/prettytable, so i had to emerge them)
Then press enter and it dumps out CPUID and version for each file.

Other than case differences i found the CPUID for my fam10 AMD system. eg
# cpuid -r | grep "0x0001 0x00" | awk '{ print $3}' | uniq | cut -d x
-f 3
00100f43

And in the MCE output;
| 4  | 00100F43 | 01C8 | 2010-03-11 |  Latest  |

Dmesg had
microcode: CPU0: patch_level=0x01c8

So its confirmed its at the latest microcode, from 2010

Re: [gentoo-user] microcode applied?

[Peter Humphrey]

On Tuesday, 9 January 2018 07:31:35 GMT Mick wrote:
> On Tuesday, 9 January 2018 00:15:03 GMT Peter Humphrey wrote:
> > On Monday, 8 January 2018 10:29:41 GMT Max Zettlmeißl wrote:
> > > > How do you build the microcode into the kernel? The only
> > > > place I can see to do that in menuconfig is under Device Drivers;
> > > > there's no such field under Firmware.
> > > 
> > > The Device Drivers section is exactly where the microcode is included.
> > > CONFIG_EXTRA_FIRMWARE is the relevant symbol.
> > 
> > Right. So which of the 95 files under /lib/firmware/intel-ucode do I
> > specify? That's in addition to the 14 files I have for my amdgpu.
> 
> There may be a cleverer way, but this is how I have been doing it.
> 
> Install sys-apps/iucode_tool.  Run:
> 
> # iucode_tool -S
> 
> It will report the microcode signature for your CPU.  For example, one of
> mine:
> 
> iucode_tool: system has processor(s) with signature 0x000106e5
> 
> 
> (Re)emerge sys-firmware/intel-microcode and capture all its output.  Then
> search for the above signature, again from the same CPU, as an example,
> this matches:
> 
> intel-ucode/06-1e-05
> signature: 0x106e5<==
> flags: 0x13
> revision:  0x07
> date:  2013-08-20
> size:  7168
> 
> 
> Add the first line above in your CONFIG_EXTRA_FIRMWARE and rebuild your
> kernel.

Thanks Mick. After asking that, I discovered this web page, which confirms 
your method: https://wiki.gentoo.org/wiki/Intel_microcode . Then I found 
that I was already loading the right microcode, though I really have no idea 
where I'd got the information from.

Thanks to the others who answered as well.

-- 
Regards,
Peter.

Re: [gentoo-user] microcode applied?

[Wols Lists]

On 08/01/18 13:52, Rich Freeman wrote:
> There is also a lot of discussion on lkml about the right fix.  We
> might very well end up seeing both AMD- and Intel-specific fixes with
> conditional logic.  The two vendors don't really seem to be
> coordinating on this.  Intel is pushing patches that apparently don't
> work on AMD, or aren't necessary on AMD.  AMD for its part hasn't been
> pushing much in public at all, but has made a few list posts, and they
> have that mystery microcode update.

Probably not much point co-operating. The *internals* of AMD chips and
Intel chips are very different. I suspect both of them have a RISC core
with an x86 instruction set interpretation layer, but that's where the
similarities end ...

Bit like expecting a turbo-prop engine company to co-ordinate their
designs with a piston engine company. The two engines might look very
similar on the outside, but the internal machinery bears no resemblance
whatsoever one to the other.

Cheers,
Wol

Re: [gentoo-user] microcode applied?

[Jorge Almeida]

On Tue, Jan 9, 2018 at 12:15 AM, Peter Humphrey  wrote:
> On Monday, 8 January 2018 10:29:41 GMT Max Zettlmeißl wrote:
>> > How do you build the microcode into the kernel? The only
>> > place I can see to do that in menuconfig is under Device Drivers;
>> > there's no such field under Firmware.
>>
>> The Device Drivers section is exactly where the microcode is included.
>> CONFIG_EXTRA_FIRMWARE is the relevant symbol.
>
> Right. So which of the 95 files under /lib/firmware/intel-ucode do I
> specify? That's in addition to the 14 files I have for my amdgpu.
>

In my system (intel):

$ iucode_tool -S
iucode_tool: system has processor(s) with signature 0x000906e9
$ iucode_tool -L /lib/firmware/intel-ucode | grep 0x000906e9 -B 1
microcode bundle 18: /lib/firmware/intel-ucode/06-9e-09
  018/001: sig 0x000906e9, pf_mask 0x2a, 2017-04-06, rev 0x005e, size 97280
  018/002: sig 0x000906e9, pf_mask 0x2a, 2017-04-06, rev 0x005e, size 97280

So in this example "intel-ucode/06-9e-09" is what you'll write in the
kernel form.

There is a amd-ucode dir in /lib/firmware, so I assume it will be the same.

Jorge Almeida

Re: [gentoo-user] microcode applied?

[Adam Carter]

>
> > The Device Drivers section is exactly where the microcode is included.
> > CONFIG_EXTRA_FIRMWARE is the relevant symbol.
>
> Right. So which of the 95 files under /lib/firmware/intel-ucode do I
> specify? That's in addition to the 14 files I have for my amdgpu.
>
>
For intel;

iucode_tool -L /lib/firmware/intel-ucode/* | grep -B 1 `iucode_tool -S 2>&1
| awk '{print $7}'`

As Mick posted, iucode_tool -S will let you know what the CPU signature is,
and running iucode_tool -L against the microcode files dumps out which CPU
sig each file is for, so the above command just searches the files for the
right signature.

Hopefully there's an equivalent for AMD.

Re: [gentoo-user] microcode applied?

[Neil Bothwick]

On Tue, 09 Jan 2018 00:15:03 +, Peter Humphrey wrote:

> > > How do you build the microcode into the kernel? The only
> > > place I can see to do that in menuconfig is under Device Drivers;
> > > there's no such field under Firmware.  
> > 
> > The Device Drivers section is exactly where the microcode is included.
> > CONFIG_EXTRA_FIRMWARE is the relevant symbol.  
> 
> Right. So which of the 95 files under /lib/firmware/intel-ucode do I 
> specify? That's in addition to the 14 files I have for my amdgpu.

I found the simplest way to do it was to emerge intel-microcode with the
initramfs uSE flag. Then copy /lib/firmware/microcode.cpio to /boot and
add it as an initrd - before the existing one if you already use one.

That way it is still loaded at the same time but you don't need to
recompile your kernels (most of us probably have more than one in /boot)
each time there is a microcode update. The USE flag takes care of
selecting the correct file for your CPU.


-- 
Neil Bothwick

Top Oxymorons Number 20: Synthetic natural gas


pgpNQjdJJjvIs.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] microcode applied?

[Mick]

On Tuesday, 9 January 2018 00:15:03 GMT Peter Humphrey wrote:
> On Monday, 8 January 2018 10:29:41 GMT Max Zettlmeißl wrote:
> > > How do you build the microcode into the kernel? The only
> > > place I can see to do that in menuconfig is under Device Drivers;
> > > there's no such field under Firmware.
> > 
> > The Device Drivers section is exactly where the microcode is included.
> > CONFIG_EXTRA_FIRMWARE is the relevant symbol.
> 
> Right. So which of the 95 files under /lib/firmware/intel-ucode do I
> specify? That's in addition to the 14 files I have for my amdgpu.

There may be a cleverer way, but this is how I have been doing it.

Install sys-apps/iucode_tool.  Run:

# iucode_tool -S

It will report the microcode signature for your CPU.  For example, one of 
mine:

iucode_tool: system has processor(s) with signature 0x000106e5


(Re)emerge sys-firmware/intel-microcode and capture all its output.  Then 
search for the above signature, again from the same CPU, as an example, this 
matches:

intel-ucode/06-1e-05
signature: 0x106e5<==
flags: 0x13
revision:  0x07
date:  2013-08-20
size:  7168


Add the first line above in your CONFIG_EXTRA_FIRMWARE and rebuild your 
kernel.

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] microcode applied?

[Rich Freeman]

On Sun, Jan 7, 2018 at 11:42 PM,   wrote:
> You really can't fix it completely in
> software on either brand, at best you are counting on code to protect code
> from a hardware on intel, and  more mild but still dangerous design issues
> on both.

As far as I can tell from the various emails/postings you can actually
fix this entirely in software on AMD, though it might be better solved
with a combination of microcode and software.

Variant 3 doesn't impact AMD.

Regarding variant 2:
https://lkml.org/lkml/2018/1/4/742
(which seems to be down right now, so I'll also post:)
https://webcache.googleusercontent.com/search?q=cache:i47fyooNn4UJ:https://lkml.org/lkml/2018/1/4/742+=1=en=clnk=us

Regarding variant 1, I suspect this could be fixed with a call to
something like CPUID, though that probably will impact performance a
bit in critical code, and it probably could also be fixed by tossing
in some instructions to manipulate either speculative execution or the
cache (such as forcing the CPU to fetch both possible target addresses
into the cache to make it impossible to tell which branch was
followed).  Using LFENCE (which is what Intel recommends) apparently
requires an MSR setting or maybe a microcode update.  I haven't
actually tested CPUID on the released spectre exploit code, but I have
confirmed that LFENCE doesn't fix it at all without the microcode/MSR
fix.  The main advantage of microcode updates would be that you could
probably reduce the complexity of the software fix and maybe improve
performance.  Not speculatively executing something will always have
some performance hit, but it could be minimal.

There is also an AMD microcode update floating around (which Gentoo
just deployed), and I can't figure out what it actually does, because
AMD hasn't said a word about it.  I can't imagine that anybody other
than AMD wrote it, so I assume it went through back channels (Suse was
the first to come out with it).  Suse says that it disables branch
prediction, and everybody else seems to be going with that description
(though the upstream kernel team hasn't accepted the change).
Obviously it can't completely disable branch prediction without
clobbering performance so it is a mystery as to when it actually does
disable it.  I haven't deployed the kernel patch to load it yet so I
haven't had a chance to test the spectre variant 1 code with it.

There is also a lot of discussion on lkml about the right fix.  We
might very well end up seeing both AMD- and Intel-specific fixes with
conditional logic.  The two vendors don't really seem to be
coordinating on this.  Intel is pushing patches that apparently don't
work on AMD, or aren't necessary on AMD.  AMD for its part hasn't been
pushing much in public at all, but has made a few list posts, and they
have that mystery microcode update.

I suspect that that Linux will either adopt conditional AMD vs Intel
code, or will force a compromise that works on both.  I have no idea
what that will end up being.  Once that happens I wouldn't be
surprised if we see GCC adopt a fix to apply the software side of that
automatically.

-- 
Rich

Re: [gentoo-user] microcode applied?

[Corbin Bird]

On 01/07/2018 09:24 PM, Adam Carter wrote:
> Does the absence of a "microcode updated" message in dmesg imply that
> the microcode was not updated?
>
> I believe my fam10/barcelona AMD CPU will use
> amd-ucode/microcode_amd.bin but there's no update message.
>
> I've checked the config against another system that works and cant see
> any errors. Is there a way to turn on debugging?

Sample "dmesg" output for a Fam15h AMD ( Kernel 4.9.xx ) :
> [   10.336395] microcode: microcode updated early to new
> patch_level=0x0600084f
> [   10.337132] microcode: CPU0: patch_level=0x0600084f
> [   10.337839] microcode: CPU1: patch_level=0x0600084f
> [   10.338631] microcode: CPU2: patch_level=0x0600084f
> [   10.342083] microcode: CPU3: patch_level=0x0600084f
> [   10.342756] microcode: CPU4: patch_level=0x0600084f
> [   10.343468] microcode: CPU5: patch_level=0x0600084f
> [   10.344117] microcode: CPU6: patch_level=0x0600084f
> [   10.344786] microcode: CPU7: patch_level=0x0600084f
> [   10.345615] microcode: Microcode Update Driver: v2.01
> , Peter Oruba
Output is very similar for Fam10h AMDs as well ( Phenom II x4 980 )

Re: [gentoo-user] microcode applied?

[Peter Humphrey]

On Monday, 8 January 2018 10:29:41 GMT Max Zettlmeißl wrote:
> > How do you build the microcode into the kernel? The only
> > place I can see to do that in menuconfig is under Device Drivers;
> > there's no such field under Firmware.
> 
> The Device Drivers section is exactly where the microcode is included.
> CONFIG_EXTRA_FIRMWARE is the relevant symbol.

Right. So which of the 95 files under /lib/firmware/intel-ucode do I 
specify? That's in addition to the 14 files I have for my amdgpu.

-- 
Regards,
Peter.

Re: [gentoo-user] microcode applied?

[Mick]

On Monday, 8 January 2018 09:05:02 GMT Max Zettlmeißl wrote:
> It seems like there are no microcode updates for your specific CPU
> bundled in linux-firmware.

Only two out of three Intel boxen here report an early update of microcode in 
dmesg.  Even when they do, it is not certain the latest firmware has brought 
new code:

[0.00] microcode: microcode updated early to revision 0x7, date = 
2013-08-20

This date above puzzles me.  Is it that on this PC's CPU the Intel bugs cannot 
be ameliorated by the latest intel-ucode release, or is it that Intel have not 
bothered to release microcode revisions for all their products.

Running in the directory /lib/firmware/intel-ucode the following command:

iucode_tool -v --date-after=2017-01-01  --write-all-named-to=TEST .

spews out into TEST/ the microcode for this CPU:

s000106E5_m0013_r0007.fw

Therefore if microcode for my CPU was included in intel-ucode releases since 
2017-01-01, is this the same unchanged microcode being released since the date 
reported in dmesg of 2013-08-20?

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] microcode applied?

[Adam Carter]

On Mon, Jan 8, 2018 at 9:14 PM, Peter Humphrey 
wrote:

> On Monday, 8 January 2018 04:55:58 GMT Max Zettlmeißl wrote:
>
> > You can either use an initrd or build the microcode into your kernel
> > image. I prefer the latter.
>
> I'm confused now. How do you build the microcode into the kernel? The only
> place I can see to do that in menuconfig is under Device Drivers; there's
> no
> such field under Firmware.
>

Device Drivers -> Generic Driver Options -> Include in-kernel firmware
blobs in kernel binary

which is CONFIG_FIRMWARE_IN_KERNEL=y

Re: [gentoo-user] microcode applied?

[Max Zettlmeißl]

> How do you build the microcode into the kernel? The only
> place I can see to do that in menuconfig is under Device Drivers; there's no
> such field under Firmware.

The Device Drivers section is exactly where the microcode is included.
CONFIG_EXTRA_FIRMWARE is the relevant symbol.

Re: [gentoo-user] microcode applied?

[Peter Humphrey]

On Monday, 8 January 2018 04:55:58 GMT Max Zettlmeißl wrote:

> You can either use an initrd or build the microcode into your kernel
> image. I prefer the latter.

I'm confused now. How do you build the microcode into the kernel? The only 
place I can see to do that in menuconfig is under Device Drivers; there's no 
such field under Firmware.

-- 
Regards,
Peter.

Re: [gentoo-user] microcode applied?

[Max Zettlmeißl]

> Since I dont know where look up firmware version numbers i'm in the dark.

You can use MC Extractor to extract the metadata associated with the
AMD microcode updates.

The microcode_amd.bin which is part of
sys-kernel/linux-firmware-20180103-r1 contains the following microcode
updates:

CPUID   Version
00100F22 0183
00100F20 0184
00100F62 01C7
00100F43 01C8
00100F81 01D9
00100F80 01DA
00100F41 01DB
00100FA0 01DC
00200F31 0232
00300F10 0327
00500F10 0529
00500F20 05000119

According to your first message your CPU is likely to have either the
ID 00100F21 or the ID 00100F2A.
It seems like there are no microcode updates for your specific CPU
bundled in linux-firmware.

Re: [gentoo-user] microcode applied?

[Adam Carter]

On Mon, Jan 8, 2018 at 3:55 PM, Max Zettlmeißl  wrote:

> > The contents of cpuinfo is the same as the messages in dmesg. What does
> that
> > imply?
>
> Your BIOS or EFI might already install the same version or a later
> version than what the microcode package provides. Although the second
> case is highly unlikely.
> The update might also just not get applied properly.
>
> You should check whether the actual microcode version is the version
> which you expect.
>
> How are you trying to apply the microcode?
> The best way to update your processor's microcode is via early
> microcode loading.
> You can either use an initrd or build the microcode into your kernel
> image. I prefer the latter.
>
>
Yes the microcode is in the the kernel. Equivalent config works fine on
another machine.

The problem could be;
The firmware from /lib/firmware is that same as that on the system, so its
not required
The firmware loaded successfully, but didnt write a success log
The firmware failed to load (and didnt write a failure log, but we dont
expect it to)

Since I dont know where look up firmware version numbers i'm in the dark.

Re: [gentoo-user] microcode applied?

[Max Zettlmeißl]

> The contents of cpuinfo is the same as the messages in dmesg. What does that
> imply?

Your BIOS or EFI might already install the same version or a later
version than what the microcode package provides. Although the second
case is highly unlikely.
The update might also just not get applied properly.

You should check whether the actual microcode version is the version
which you expect.

How are you trying to apply the microcode?
The best way to update your processor's microcode is via early
microcode loading.
You can either use an initrd or build the microcode into your kernel
image. I prefer the latter.

Re: [gentoo-user] microcode applied?

[mad.scientist.at.large]

There is also a test program to see if the vulnerability is there, i'd 
definately check that as well, best to check both considering how terrible the 
but is.  frankly amd and intel will still have software vulnerabilities, 
particular apps are being patched but if an exploit is developed in the "wild" 
or the info leaks it will be used with other vulnerabilities, with user 
privilages i believe or does it require root/susceptable root code.  

Frankly, i suspect with more research, or possibly unreleased details that you 
could likely use the larger issues in other bad ways, hopefully not easily 
(there will always be other easier exploits, this one just makes everything 
else easy if you know it, most people take the easy way whether breaking in or 
doing anything else).  You really can't fix it completely in software on either 
brand, at best you are counting on code to protect code from a hardware on 
intel, and  more mild but still dangerous design issues on both.  hopefully 
microcode update will help, hopefully it doesn't disable features that are hard 
to live without.

Hopefully things will get better, and hopefully new features on new chips will 
help or prevent this issue after the OS is rewritten to use them and if you can 
block code that doesn't work with new features, i.e. no backwards 
compatibility.  modern cpu design has many potential security issues and chips 
that use things like parallel execution have to be  very carefully designed to 
hopefully avoid such issues, obviously hardware at this complexity is as 
impossible to fully test and debug as any large modern piece of software.  Many 
hacks result from thinking about things sideways or in ways no one on the 
engineering team has, no one sees and knows it all, there are simply too many 
possibilities to test completely.  You have to depend on trying not to get 
weakness in and on protecting from the unseen by keeping everything else secure 
so that hopefully one good thing will block the exploitation of a flaw.  
Security in Depth is your' best option, absolute security is unlikely even on 
quantum computers, and impossible on anything less of any complexity with 
features modern computing depends on.


mad.scientist.at.large (a good madscientist)
--
God bless the rich, the greedy and the corrupt politicians they have put into 
office.   God bless them for helping me do the right thing by giving the rich 
my little pile of cash.  After all, the rich know what to do with money.


7. Jan 2018 21:01 by m...@zettlmeissl.de:


>> Does the absence of a "microcode updated" message in dmesg imply that the
>> microcode was not updated?
>
> Not necessarily.
>
>> Is there a way to turn on debugging?
>
> The easiest way to check whether the microcode update was applied
> correctly would be to check the microcode version in /proc/cpuinfo

Re: [gentoo-user] microcode applied?

[Adam Carter]

>
> The easiest way to check whether the microcode update was applied
> correctly would be to check the microcode version in /proc/cpuinfo
>

The contents of cpuinfo is the same as the messages in dmesg. What does
that imply?

Re: [gentoo-user] microcode applied?

[Max Zettlmeißl]

> Does the absence of a "microcode updated" message in dmesg imply that the
> microcode was not updated?

Not necessarily.

> Is there a way to turn on debugging?

The easiest way to check whether the microcode update was applied
correctly would be to check the microcode version in /proc/cpuinfo