Re: [PATCH 2/2] iommu: add warning when sharing groups

[Sinan Kaya]

Hi Alex,

On 2/15/2017 4:43 PM, Sinan Kaya wrote:
> On 2/15/2017 2:36 PM, Alex Williamson wrote:
>> On Tue, 14 Feb 2017 22:53:35 -0500
>> ok...@codeaurora.org wrote:
>>
>>> On 2017-02-14 18:51, Alex Williamson wrote:
 On Tue, 14 Feb 2017 16:25:22 -0500
 Sinan Kaya  wrote:
   
> The ACS requirement has been obscured in the current code and is only
> known by certain individuals who happen to read the code. Print out a
> warning with ACS path failure when ACS requirement is not met.
>
> Signed-off-by: Sinan Kaya 
> ---
>  drivers/iommu/iommu.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index dbe7f65..049ee0a 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -811,6 +811,9 @@ struct iommu_group *pci_device_group(struct device 
> *dev)
>   if (IS_ERR(group))
>   return NULL;
>
> + if (pci_is_root_bus(bus))
> + dev_warn_once(>dev, "using shared group due to ACS path 
> failure\n");
> +
>   return group;
>  }
>   

 The premise here is flawed.  An IOMMU group based at the root bus
 doesn't necessarily imply a lack of ACS.  There are devices on root
 buses, integrated endpoints and root ports.  Naturally an IOMMU group
 for these devices needs to be based at the root bus.  Additionally,
 there can be IOMMU groups developed around a lack of ACS that don't
 intersect with the root bus.  Since this is a warn_once, the false
 positives for root bus devices are going to be enumerated first.  On an
 Intel system there's typically a device as 00.0 that will always be
 pointlessly listed first.  Also, it's not clear that grouping devices
 together is always wrong, as Robin pointed out in the EHCI/OHCI
 example.  Lack of ACS on downtream ports is likely to cause problems,
 especially if that downstream port exposes a slot.  Maybe that would be
 a good place to start.  Also, what is someone supposed to do when they
 see this error?  If we can hope they'll look for the error in the code
 (unlikely) a big comment with useful external links might be
 necessary.  Based on how easily vendors ignore kernel warnings, I'm
 dubious there's any value to this path though.  Thanks,  
>>>
>>> Maybe, a better solution would be to add some sentences into vfio.txt 
>>> documentation.
>>>
>>> I'm ready to drop this patch. I just don't want ACS requirement to be 
>>> hidden between the source code.
>>>

I posted V2 to linux-pci maillist but forgot to CC the iommu group.

[PATCH V2] PCI: add QCOM root port quirks for ACS

I dropped the second patch (this one I'm replying to) as discussed. 
I did minor cleanups in the first commit including

1- commit message change
2- replace dev_info_once with dev_info

https://patchwork.codeaurora.org/patch/177033/

Sinan

-- 
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm 
Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux 
Foundation Collaborative Project.
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [RFC PATCH v4 03/28] x86: Add the Secure Memory Encryption CPU feature

[Borislav Petkov]

On Thu, Feb 16, 2017 at 01:42:13PM -0600, Tom Lendacky wrote:
> I realize it's a bit more code and expands the changes but I thought it
> would be a bit clearer as to what was going on this way. And then the
> follow on patch for the physical address reduction goes in nicely, too.

Well, the code from the next patch should go to AMD-specific place like
arch/x86/kernel/cpu/amd.c anyway, where you don't have to do vendor
checks.

> If you prefer I stay with the scattered feature approach and then clear
> the bit based on the MSR at the end of init_amd() I can do that. I'm
> not attached to either method.

Yes please. We should keep the shole X86_FEATURE machinery from
exploding in size. Especially if CPUID_0x801f is not a leaf we're
going to be adding the majority of its bits, to warrant a separate
->x86_capability array element.

  [If it does later, we can always move it to a separate element. ]

Thanks.

-- 
Regards/Gruss,
Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [RFC PATCH v4 01/28] x86: Documentation for AMD Secure Memory Encryption (SME)

[Tom Lendacky]

On 02/16/2017 11:56 AM, Borislav Petkov wrote:
> Ok, this time detailed review :-)
> 
> On Thu, Feb 16, 2017 at 09:42:11AM -0600, Tom Lendacky wrote:
>> This patch adds a Documenation entry to decribe the AMD Secure Memory
>> Encryption (SME) feature.
> 
> Please introduce a spellchecker into your patch creation workflow. I see
> two typos in one line.
> 
> Also, never start patch commit messages with "This patch" - we know it
> is this patch. Always write a doer-sentences explaining the why, not the
> what. Something like:
> 
> "Add a SME and mem_encrypt= kernel parameter documentation."
> 
> for example.

Ok, will do.

> 
>> Signed-off-by: Tom Lendacky 
>> ---
>>  Documentation/admin-guide/kernel-parameters.txt |   11 
>>  Documentation/x86/amd-memory-encryption.txt |   57 
>> +++
>>  2 files changed, 68 insertions(+)
>>  create mode 100644 Documentation/x86/amd-memory-encryption.txt
>>
>> diff --git a/Documentation/admin-guide/kernel-parameters.txt 
>> b/Documentation/admin-guide/kernel-parameters.txt
>> index 110745e..91c40fa 100644
>> --- a/Documentation/admin-guide/kernel-parameters.txt
>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>> @@ -2145,6 +2145,17 @@
>>  memory contents and reserves bad memory
>>  regions that are detected.
>>  
>> +mem_encrypt=[X86-64] AMD Secure Memory Encryption (SME) control
>> +Valid arguments: on, off
>> +Default (depends on kernel configuration option):
>> +  on  (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y)
>> +  off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n)
>> +mem_encrypt=on: Activate SME
>> +mem_encrypt=off:Do not activate SME
>> +
>> +Refer to the SME documentation for details on when
> 
> "Refer to Documentation/x86/amd-memory-encryption.txt .."

Ok.

> 
>> +memory encryption can be activated.
>> +
>>  mem_sleep_default=  [SUSPEND] Default system suspend mode:
>>  s2idle  - Suspend-To-Idle
>>  shallow - Power-On Suspend or equivalent (if supported)
>> diff --git a/Documentation/x86/amd-memory-encryption.txt 
>> b/Documentation/x86/amd-memory-encryption.txt
>> new file mode 100644
>> index 000..0938e89
>> --- /dev/null
>> +++ b/Documentation/x86/amd-memory-encryption.txt
>> @@ -0,0 +1,57 @@
>> +Secure Memory Encryption (SME) is a feature found on AMD processors.
>> +
>> +SME provides the ability to mark individual pages of memory as encrypted 
>> using
>> +the standard x86 page tables.  A page that is marked encrypted will be
>> +automatically decrypted when read from DRAM and encrypted when written to
>> +DRAM.  SME can therefore be used to protect the contents of DRAM from 
>> physical
>> +attacks on the system.
>> +
>> +A page is encrypted when a page table entry has the encryption bit set (see
>> +below how to determine the position of the bit).  The encryption bit can be
> 
> "... how to determine its position)."

Ok.

> 
>> +specified in the cr3 register, allowing the PGD table to be encrypted. Each
>> +successive level of page tables can also be encrypted.
>> +
>> +Support for SME can be determined through the CPUID instruction. The CPUID
>> +function 0x801f reports information related to SME:
>> +
>> +0x801f[eax]:
>> +Bit[0] indicates support for SME
>> +0x801f[ebx]:
>> +Bit[5:0]  pagetable bit number used to activate memory
>> +  encryption
> 
> s/Bit/Bits/

Ok.

> 
>> +Bit[11:6] reduction in physical address space, in bits, when
> 
> Ditto.
> 
>> +  memory encryption is enabled (this only affects system
>> +  physical addresses, not guest physical addresses)
>> +
>> +If support for SME is present, MSR 0xc00100010 (SYS_CFG) can be used to
> 
> Let's use the kernel's define name MSR_K8_SYSCFG to avoid ambiguity.

Will do.

> 
>> +determine if SME is enabled and/or to enable memory encryption:
>> +
>> +0xc0010010:
>> +Bit[23]   0 = memory encryption features are disabled
>> +  1 = memory encryption features are enabled
>> +
>> +Linux relies on BIOS to set this bit if BIOS has determined that the 
>> reduction
>> +in the physical address space as a result of enabling memory encryption (see
>> +CPUID information above) will not conflict with the address space resource
>> +requirements for the system.  If this bit is not set upon Linux startup then
>> +Linux itself will not set it and memory encryption will not be possible.
>> +
>> +The state of SME in the Linux kernel can be documented as follows:
>> +- Supported:
>> +  The CPU supports SME (determined through CPUID instruction).
>> +
>> +- Enabled:
>> +  Supported and bit 23 of the SYS_CFG MSR is set.
> 
> 

Re: [RFC PATCH v4 03/28] x86: Add the Secure Memory Encryption CPU feature

[Tom Lendacky]

On 02/16/2017 12:13 PM, Borislav Petkov wrote:
> On Thu, Feb 16, 2017 at 09:42:36AM -0600, Tom Lendacky wrote:
>> Update the CPU features to include identifying and reporting on the
>> Secure Memory Encryption (SME) feature.  SME is identified by CPUID
>> 0x801f, but requires BIOS support to enable it (set bit 23 of
>> SYS_CFG MSR).  Only show the SME feature as available if reported by
>> CPUID and enabled by BIOS.
>>
>> Signed-off-by: Tom Lendacky 
>> ---
>>  arch/x86/include/asm/cpufeature.h|7 +--
>>  arch/x86/include/asm/cpufeatures.h   |5 -
>>  arch/x86/include/asm/disabled-features.h |3 ++-
>>  arch/x86/include/asm/msr-index.h |2 ++
>>  arch/x86/include/asm/required-features.h |3 ++-
>>  arch/x86/kernel/cpu/common.c |   19 +++
>>  6 files changed, 34 insertions(+), 5 deletions(-)
> 
> What happened here?
> 
> You had it already:
> 
> https://lkml.kernel.org/r/20161110003459.3280.25796.st...@tlendack-t1.amdoffice.net
> 
> The bit in get_cpu_cap() with checking the MSR you can add at the end of
> init_amd() for example.

I realize it's a bit more code and expands the changes but I thought it
would be a bit clearer as to what was going on this way. And then the
follow on patch for the physical address reduction goes in nicely, too.

If you prefer I stay with the scattered feature approach and then clear
the bit based on the MSR at the end of init_amd() I can do that. I'm
not attached to either method.

Thanks,
Tom

> 
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [RFC PATCH v4 03/28] x86: Add the Secure Memory Encryption CPU feature

[Borislav Petkov]

On Thu, Feb 16, 2017 at 09:42:36AM -0600, Tom Lendacky wrote:
> Update the CPU features to include identifying and reporting on the
> Secure Memory Encryption (SME) feature.  SME is identified by CPUID
> 0x801f, but requires BIOS support to enable it (set bit 23 of
> SYS_CFG MSR).  Only show the SME feature as available if reported by
> CPUID and enabled by BIOS.
> 
> Signed-off-by: Tom Lendacky 
> ---
>  arch/x86/include/asm/cpufeature.h|7 +--
>  arch/x86/include/asm/cpufeatures.h   |5 -
>  arch/x86/include/asm/disabled-features.h |3 ++-
>  arch/x86/include/asm/msr-index.h |2 ++
>  arch/x86/include/asm/required-features.h |3 ++-
>  arch/x86/kernel/cpu/common.c |   19 +++
>  6 files changed, 34 insertions(+), 5 deletions(-)

What happened here?

You had it already:

https://lkml.kernel.org/r/20161110003459.3280.25796.st...@tlendack-t1.amdoffice.net

The bit in get_cpu_cap() with checking the MSR you can add at the end of
init_amd() for example.

-- 
Regards/Gruss,
Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [RFC PATCH v4 01/28] x86: Documentation for AMD Secure Memory Encryption (SME)

[Borislav Petkov]

Ok, this time detailed review :-)

On Thu, Feb 16, 2017 at 09:42:11AM -0600, Tom Lendacky wrote:
> This patch adds a Documenation entry to decribe the AMD Secure Memory
> Encryption (SME) feature.

Please introduce a spellchecker into your patch creation workflow. I see
two typos in one line.

Also, never start patch commit messages with "This patch" - we know it
is this patch. Always write a doer-sentences explaining the why, not the
what. Something like:

"Add a SME and mem_encrypt= kernel parameter documentation."

for example.

> Signed-off-by: Tom Lendacky 
> ---
>  Documentation/admin-guide/kernel-parameters.txt |   11 
>  Documentation/x86/amd-memory-encryption.txt |   57 
> +++
>  2 files changed, 68 insertions(+)
>  create mode 100644 Documentation/x86/amd-memory-encryption.txt
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt 
> b/Documentation/admin-guide/kernel-parameters.txt
> index 110745e..91c40fa 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -2145,6 +2145,17 @@
>   memory contents and reserves bad memory
>   regions that are detected.
>  
> + mem_encrypt=[X86-64] AMD Secure Memory Encryption (SME) control
> + Valid arguments: on, off
> + Default (depends on kernel configuration option):
> +   on  (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y)
> +   off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n)
> + mem_encrypt=on: Activate SME
> + mem_encrypt=off:Do not activate SME
> +
> + Refer to the SME documentation for details on when

"Refer to Documentation/x86/amd-memory-encryption.txt .."

> + memory encryption can be activated.
> +
>   mem_sleep_default=  [SUSPEND] Default system suspend mode:
>   s2idle  - Suspend-To-Idle
>   shallow - Power-On Suspend or equivalent (if supported)
> diff --git a/Documentation/x86/amd-memory-encryption.txt 
> b/Documentation/x86/amd-memory-encryption.txt
> new file mode 100644
> index 000..0938e89
> --- /dev/null
> +++ b/Documentation/x86/amd-memory-encryption.txt
> @@ -0,0 +1,57 @@
> +Secure Memory Encryption (SME) is a feature found on AMD processors.
> +
> +SME provides the ability to mark individual pages of memory as encrypted 
> using
> +the standard x86 page tables.  A page that is marked encrypted will be
> +automatically decrypted when read from DRAM and encrypted when written to
> +DRAM.  SME can therefore be used to protect the contents of DRAM from 
> physical
> +attacks on the system.
> +
> +A page is encrypted when a page table entry has the encryption bit set (see
> +below how to determine the position of the bit).  The encryption bit can be

"... how to determine its position)."

> +specified in the cr3 register, allowing the PGD table to be encrypted. Each
> +successive level of page tables can also be encrypted.
> +
> +Support for SME can be determined through the CPUID instruction. The CPUID
> +function 0x801f reports information related to SME:
> +
> + 0x801f[eax]:
> + Bit[0] indicates support for SME
> + 0x801f[ebx]:
> + Bit[5:0]  pagetable bit number used to activate memory
> +   encryption

s/Bit/Bits/

> + Bit[11:6] reduction in physical address space, in bits, when

Ditto.

> +   memory encryption is enabled (this only affects system
> +   physical addresses, not guest physical addresses)
> +
> +If support for SME is present, MSR 0xc00100010 (SYS_CFG) can be used to

Let's use the kernel's define name MSR_K8_SYSCFG to avoid ambiguity.

> +determine if SME is enabled and/or to enable memory encryption:
> +
> + 0xc0010010:
> + Bit[23]   0 = memory encryption features are disabled
> +   1 = memory encryption features are enabled
> +
> +Linux relies on BIOS to set this bit if BIOS has determined that the 
> reduction
> +in the physical address space as a result of enabling memory encryption (see
> +CPUID information above) will not conflict with the address space resource
> +requirements for the system.  If this bit is not set upon Linux startup then
> +Linux itself will not set it and memory encryption will not be possible.
> +
> +The state of SME in the Linux kernel can be documented as follows:
> + - Supported:
> +   The CPU supports SME (determined through CPUID instruction).
> +
> + - Enabled:
> +   Supported and bit 23 of the SYS_CFG MSR is set.

Ditto.

> +
> + - Active:
> +   Supported, Enabled and the Linux kernel is actively applying
> +   the encryption bit to page table entries (the SME mask in the
> +   kernel is non-zero).
> +
> 

[RFC PATCH v4 17/28] x86: Decrypt trampoline area if memory encryption is active

[Tom Lendacky]

When Secure Memory Encryption is enabled, the trampoline area must not
be encrypted. A CPU running in real mode will not be able to decrypt
memory that has been encrypted because it will not be able to use addresses
with the memory encryption mask.

Signed-off-by: Tom Lendacky 
---
 arch/x86/realmode/init.c |   12 
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index 5db706f1..21d7506 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -6,6 +6,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 
 struct real_mode_header *real_mode_header;
 u32 *trampoline_cr4_features;
@@ -130,6 +132,16 @@ static void __init set_real_mode_permissions(void)
unsigned long text_start =
(unsigned long) __va(real_mode_header->text_start);
 
+   /*
+* If SME is active, the trampoline area will need to be in
+* decrypted memory in order to bring up other processors
+* successfully.
+*/
+   if (sme_active()) {
+   sme_early_decrypt(__pa(base), size);
+   set_memory_decrypted((unsigned long)base, size >> PAGE_SHIFT);
+   }
+
set_memory_nx((unsigned long) base, size >> PAGE_SHIFT);
set_memory_ro((unsigned long) base, ro_size >> PAGE_SHIFT);
set_memory_x((unsigned long) text_start, text_size >> PAGE_SHIFT);

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 00/28] x86: Secure Memory Encryption (AMD)

[Tom Lendacky]

This RFC patch series provides support for AMD's new Secure Memory
Encryption (SME) feature.

SME can be used to mark individual pages of memory as encrypted through the
page tables. A page of memory that is marked encrypted will be automatically
decrypted when read from DRAM and will be automatically encrypted when
written to DRAM. Details on SME can found in the links below.

The SME feature is identified through a CPUID function and enabled through
the SYSCFG MSR. Once enabled, page table entries will determine how the
memory is accessed. If a page table entry has the memory encryption mask set,
then that memory will be accessed as encrypted memory. The memory encryption
mask (as well as other related information) is determined from settings
returned through the same CPUID function that identifies the presence of the
feature.

The approach that this patch series takes is to encrypt everything possible
starting early in the boot where the kernel is encrypted. Using the page
table macros the encryption mask can be incorporated into all page table
entries and page allocations. By updating the protection map, userspace
allocations are also marked encrypted. Certain data must be accounted for
as having been placed in memory before SME was enabled (EFI, initrd, etc.)
and accessed accordingly.

This patch series is a pre-cursor to another AMD processor feature called
Secure Encrypted Virtualization (SEV). The support for SEV will build upon
the SME support and will be submitted later. Details on SEV can be found
in the links below.

The following links provide additional detail:

AMD Memory Encryption whitepaper:
   
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf

AMD64 Architecture Programmer's Manual:
   http://support.amd.com/TechDocs/24593.pdf
   SME is section 7.10
   SEV is section 15.34

This patch series is based off of the master branch of tip.
  Commit a27cb9e1b2b4 ("Merge branch 'WIP.sched/core'")

---

Still to do: IOMMU enablement support

Changes since v3:
- Broke out some of the patches into smaller individual patches
- Updated Documentation
- Added a message to indicate why the IOMMU was disabled
- Updated CPU feature support for SME by taking into account whether
  BIOS has enabled SME
- Eliminated redundant functions
- Added some warning messages for DMA usage of bounce buffers when SME
  is active
- Added support for persistent memory
- Added support to determine when setup data is being mapped and be sure
  to map it un-encrypted
- Added CONFIG support to set the default action of whether to activate
  SME if it is supported/enabled
- Added support for (re)booting with kexec

Changes since v2:
- Updated Documentation
- Make the encryption mask available outside of arch/x86 through a
  standard include file
- Conversion of assembler routines to C where possible (not everything
  could be converted, e.g. the routine that does the actual encryption
  needs to be copied into a safe location and it is difficult to
  determine the actual length of the function in order to copy it)
- Fix SME feature use of scattered CPUID feature
- Creation of SME specific functions for things like encrypting
  the setup data, ramdisk, etc.
- New take on early_memremap / memremap encryption support
- Additional support for accessing video buffers (fbdev/gpu) as
  un-encrypted
- Disable IOMMU for now - need to investigate further in relation to
  how it needs to be programmed relative to accessing physical memory

Changes since v1:
- Added Documentation.
- Removed AMD vendor check for setting the PAT write protect mode
- Updated naming of trampoline flag for SME as well as moving of the
  SME check to before paging is enabled.
- Change to early_memremap to identify the data being mapped as either
  boot data or kernel data.  The idea being that boot data will have
  been placed in memory as un-encrypted data and would need to be accessed
  as such.
- Updated debugfs support for the bootparams to access the data properly.
- Do not set the SYSCFG[MEME] bit, only check it.  The setting of the
  MemEncryptionModeEn bit results in a reduction of physical address size
  of the processor.  It is possible that BIOS could have configured resources
  resources into a range that will now not be addressable.  To prevent this,
  rely on BIOS to set the SYSCFG[MEME] bit and only then enable memory
  encryption support in the kernel.

Tom Lendacky (28):
  x86: Documentation for AMD Secure Memory Encryption (SME)
  x86: Set the write-protect cache mode for full PAT support
  x86: Add the Secure Memory Encryption CPU feature
  x86: Handle reduction in physical address size with SME
  x86: Add Secure Memory Encryption (SME) support
  x86: Add support to enable SME during early boot processing
  x86: Provide general kernel support for memory encryption
  x86: Extend the early_memremap support with additional attrs
  x86: Add support 

[RFC PATCH v4 02/28] x86: Set the write-protect cache mode for full PAT support

[Tom Lendacky]

For processors that support PAT, set the write-protect cache mode
(_PAGE_CACHE_MODE_WP) entry to the actual write-protect value (x05).

Acked-by: Borislav Petkov 
Signed-off-by: Tom Lendacky 
---
 arch/x86/mm/pat.c |6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 9b78685..6753d9c 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -295,7 +295,7 @@ static void init_cache_modes(void)
  * pat_init - Initialize PAT MSR and PAT table
  *
  * This function initializes PAT MSR and PAT table with an OS-defined value
- * to enable additional cache attributes, WC and WT.
+ * to enable additional cache attributes, WC, WT and WP.
  *
  * This function must be called on all CPUs using the specific sequence of
  * operations defined in Intel SDM. mtrr_rendezvous_handler() provides this
@@ -356,7 +356,7 @@ void pat_init(void)
 *  0102UC-: _PAGE_CACHE_MODE_UC_MINUS
 *  0113UC : _PAGE_CACHE_MODE_UC
 *  1004WB : Reserved
-*  1015WC : Reserved
+*  1015WP : _PAGE_CACHE_MODE_WP
 *  1106UC-: Reserved
 *  1117WT : _PAGE_CACHE_MODE_WT
 *
@@ -364,7 +364,7 @@ void pat_init(void)
 * corresponding types in the presence of PAT errata.
 */
pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) |
- PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT);
+ PAT(4, WB) | PAT(5, WP) | PAT(6, UC_MINUS) | PAT(7, WT);
}
 
if (!boot_cpu_done) {

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 27/28] x86: Add support to encrypt the kernel in-place

[Tom Lendacky]

This patch adds the support to encrypt the kernel in-place. This is
done by creating new page mappings for the kernel - a decrypted
write-protected mapping and an encrypted mapping. The kernel is encyrpted
by copying the kernel through a temporary buffer.

Signed-off-by: Tom Lendacky 
---
 arch/x86/kernel/Makefile   |1 
 arch/x86/kernel/mem_encrypt_boot.S |  156 +
 arch/x86/kernel/mem_encrypt_init.c |  191 
 3 files changed, 348 insertions(+)
 create mode 100644 arch/x86/kernel/mem_encrypt_boot.S

diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 33af80a..dc3ed84 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -142,4 +142,5 @@ ifeq ($(CONFIG_X86_64),y)
obj-y   += vsmp_64.o
 
obj-y   += mem_encrypt_init.o
+   obj-$(CONFIG_AMD_MEM_ENCRYPT)   += mem_encrypt_boot.o
 endif
diff --git a/arch/x86/kernel/mem_encrypt_boot.S 
b/arch/x86/kernel/mem_encrypt_boot.S
new file mode 100644
index 000..58e1756
--- /dev/null
+++ b/arch/x86/kernel/mem_encrypt_boot.S
@@ -0,0 +1,156 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+   .text
+   .code64
+ENTRY(sme_encrypt_execute)
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+   /*
+* Entry parameters:
+*   RDI - virtual address for the encrypted kernel mapping
+*   RSI - virtual address for the decrypted kernel mapping
+*   RDX - length of kernel
+*   RCX - address of the encryption workarea
+* - stack page (PAGE_SIZE)
+* - encryption routine page (PAGE_SIZE)
+* - intermediate copy buffer (PMD_PAGE_SIZE)
+*R8 - address of the pagetables to use for encryption
+*/
+
+   /* Set up a one page stack in the non-encrypted memory area */
+   movq%rcx, %rax
+   addq$PAGE_SIZE, %rax
+   movq%rsp, %rbp
+   movq%rax, %rsp
+   push%rbp
+
+   push%r12
+   push%r13
+
+   movq%rdi, %r10
+   movq%rsi, %r11
+   movq%rdx, %r12
+   movq%rcx, %r13
+
+   /* Copy encryption routine into the workarea */
+   movq%rax, %rdi
+   leaq.Lencrypt_start(%rip), %rsi
+   movq$(.Lencrypt_stop - .Lencrypt_start), %rcx
+   rep movsb
+
+   /* Setup registers for call */
+   movq%r10, %rdi
+   movq%r11, %rsi
+   movq%r8, %rdx
+   movq%r12, %rcx
+   movq%rax, %r8
+   addq$PAGE_SIZE, %r8
+
+   /* Call the encryption routine */
+   call*%rax
+
+   pop %r13
+   pop %r12
+
+   pop %rsp/* Restore original stack pointer */
+.Lencrypt_exit:
+#endif /* CONFIG_AMD_MEM_ENCRYPT */
+
+   ret
+ENDPROC(sme_encrypt_execute)
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+/*
+ * Routine used to encrypt kernel.
+ *   This routine must be run outside of the kernel proper since
+ *   the kernel will be encrypted during the process. So this
+ *   routine is defined here and then copied to an area outside
+ *   of the kernel where it will remain and run decrypted
+ *   during execution.
+ *
+ *   On entry the registers must be:
+ * RDI - virtual address for the encrypted kernel mapping
+ * RSI - virtual address for the decrypted kernel mapping
+ * RDX - address of the pagetables to use for encryption
+ * RCX - length of kernel
+ *  R8 - intermediate copy buffer
+ *
+ * RAX - points to this routine
+ *
+ * The kernel will be encrypted by copying from the non-encrypted
+ * kernel space to an intermediate buffer and then copying from the
+ * intermediate buffer back to the encrypted kernel space. The physical
+ * addresses of the two kernel space mappings are the same which
+ * results in the kernel being encrypted "in place".
+ */
+.Lencrypt_start:
+   /* Enable the new page tables */
+   mov %rdx, %cr3
+
+   /* Flush any global TLBs */
+   mov %cr4, %rdx
+   andq$~X86_CR4_PGE, %rdx
+   mov %rdx, %cr4
+   orq $X86_CR4_PGE, %rdx
+   mov %rdx, %cr4
+
+   /* Set the PAT register PA5 entry to write-protect */
+   push%rcx
+   movl$MSR_IA32_CR_PAT, %ecx
+   rdmsr
+   push%rdx/* Save original PAT value */
+   andl$0x00ff, %edx   /* Clear PA5 */
+   orl $0x0500, %edx   /* Set PA5 to WP */
+   wrmsr
+   pop %rdx/* RDX contains original PAT value */
+   pop %rcx
+

[RFC PATCH v4 28/28] x86: Add support to make use of Secure Memory Encryption

[Tom Lendacky]

This patch adds the support to check if SME has been enabled and if
memory encryption should be activated (checking of command line option
based on the configuration of the default state).  If memory encryption
is to be activated, then the encryption mask is set and the kernel is
encrypted "in place."

Signed-off-by: Tom Lendacky 
---
 arch/x86/kernel/head_64.S  |1 +
 arch/x86/kernel/mem_encrypt_init.c |   71 +++-
 arch/x86/mm/mem_encrypt.c  |2 +
 3 files changed, 73 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index edd2f14..e6820e7 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -97,6 +97,7 @@ startup_64:
 * Save the returned mask in %r12 for later use.
 */
push%rsi
+   movq%rsi, %rdi
callsme_enable
pop %rsi
movq%rax, %r12
diff --git a/arch/x86/kernel/mem_encrypt_init.c 
b/arch/x86/kernel/mem_encrypt_init.c
index 07cbb90..35c5e3d 100644
--- a/arch/x86/kernel/mem_encrypt_init.c
+++ b/arch/x86/kernel/mem_encrypt_init.c
@@ -19,6 +19,12 @@
 #include 
 
 #include 
+#include 
+#include 
+#include 
+
+static char sme_cmdline_arg_on[] __initdata = "mem_encrypt=on";
+static char sme_cmdline_arg_off[] __initdata = "mem_encrypt=off";
 
 extern void sme_encrypt_execute(unsigned long, unsigned long, unsigned long,
void *, pgd_t *);
@@ -217,8 +223,71 @@ unsigned long __init sme_get_me_mask(void)
return sme_me_mask;
 }
 
-unsigned long __init sme_enable(void)
+unsigned long __init sme_enable(void *boot_data)
 {
+   struct boot_params *bp = boot_data;
+   unsigned int eax, ebx, ecx, edx;
+   unsigned long cmdline_ptr;
+   bool enable_if_found;
+   void *cmdline_arg;
+   u64 msr;
+
+   /* Check for an AMD processor */
+   eax = 0;
+   ecx = 0;
+   native_cpuid(, , , );
+   if ((ebx != 0x68747541) || (edx != 0x69746e65) || (ecx != 0x444d4163))
+   goto out;
+
+   /* Check for the SME support leaf */
+   eax = 0x8000;
+   ecx = 0;
+   native_cpuid(, , , );
+   if (eax < 0x801f)
+   goto out;
+
+   /*
+* Check for the SME feature:
+*   CPUID Fn8000_001F[EAX] - Bit 0
+* Secure Memory Encryption support
+*   CPUID Fn8000_001F[EBX] - Bits 5:0
+* Pagetable bit position used to indicate encryption
+*/
+   eax = 0x801f;
+   ecx = 0;
+   native_cpuid(, , , );
+   if (!(eax & 1))
+   goto out;
+
+   /* Check if SME is enabled */
+   msr = native_read_msr(MSR_K8_SYSCFG);
+   if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+   goto out;
+
+   /*
+* Fixups have not been to applied phys_base yet, so we must obtain
+* the address to the SME command line option in the following way.
+*/
+   if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT)) {
+   asm ("lea sme_cmdline_arg_off(%%rip), %0"
+: "=r" (cmdline_arg)
+: "p" (sme_cmdline_arg_off));
+   enable_if_found = false;
+   } else {
+   asm ("lea sme_cmdline_arg_on(%%rip), %0"
+: "=r" (cmdline_arg)
+: "p" (sme_cmdline_arg_on));
+   enable_if_found = true;
+   }
+
+   cmdline_ptr = bp->hdr.cmd_line_ptr | ((u64)bp->ext_cmd_line_ptr << 32);
+
+   if (cmdline_find_option_bool((char *)cmdline_ptr, cmdline_arg))
+   sme_me_mask = enable_if_found ? 1UL << (ebx & 0x3f) : 0;
+   else
+   sme_me_mask = enable_if_found ? 0 : 1UL << (ebx & 0x3f);
+
+out:
return sme_me_mask;
 }
 
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index a46bcf4..c5062e1 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -204,6 +204,8 @@ void __init mem_encrypt_init(void)
 
/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
swiotlb_update_mem_attributes();
+
+   pr_info("AMD Secure Memory Encryption (SME) active\n");
 }
 
 void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 26/28] x86: Allow kexec to be used with SME

[Tom Lendacky]

Provide support so that kexec can be used to boot a kernel when SME is
enabled.

Support is needed to allocate pages for kexec without encryption.  This
is needed in order to be able to reboot in the kernel in the same manner
as originally booted.

Additionally, when shutting down all of the CPUs we need to be sure to
disable caches, flush the caches and then halt. This is needed when booting
from a state where SME was not active into a state where SME is active.
Without these steps, it is possible for cache lines to exist for the same
physical location but tagged both with and without the encryption bit. This
can cause random memory corruption when caches are flushed depending on
which cacheline is written last.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/cacheflush.h|2 ++
 arch/x86/include/asm/init.h  |1 +
 arch/x86/include/asm/mem_encrypt.h   |   10 
 arch/x86/include/asm/pgtable_types.h |1 +
 arch/x86/kernel/machine_kexec_64.c   |3 ++
 arch/x86/kernel/process.c|   43 +-
 arch/x86/kernel/smp.c|4 ++-
 arch/x86/mm/ident_map.c  |6 +++--
 arch/x86/mm/pageattr.c   |2 ++
 include/linux/mem_encrypt.h  |   10 
 kernel/kexec_core.c  |   24 +++
 11 files changed, 100 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/cacheflush.h 
b/arch/x86/include/asm/cacheflush.h
index 33ae60a..2180cd5 100644
--- a/arch/x86/include/asm/cacheflush.h
+++ b/arch/x86/include/asm/cacheflush.h
@@ -48,8 +48,10 @@
 int set_memory_rw(unsigned long addr, int numpages);
 int set_memory_np(unsigned long addr, int numpages);
 int set_memory_4k(unsigned long addr, int numpages);
+#ifdef CONFIG_AMD_MEM_ENCRYPT
 int set_memory_encrypted(unsigned long addr, int numpages);
 int set_memory_decrypted(unsigned long addr, int numpages);
+#endif
 
 int set_memory_array_uc(unsigned long *addr, int addrinarray);
 int set_memory_array_wc(unsigned long *addr, int addrinarray);
diff --git a/arch/x86/include/asm/init.h b/arch/x86/include/asm/init.h
index 737da62..b2ec511 100644
--- a/arch/x86/include/asm/init.h
+++ b/arch/x86/include/asm/init.h
@@ -6,6 +6,7 @@ struct x86_mapping_info {
void *context;   /* context for alloc_pgt_page */
unsigned long pmd_flag;  /* page flag for PMD entry */
unsigned long offset;/* ident mapping offset */
+   unsigned long kernpg_flag;   /* kernel pagetable flag override */
 };
 
 int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index 5a17f1b..1fd5426 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -64,6 +64,16 @@ static inline u64 sme_dma_mask(void)
return 0ULL;
 }
 
+static inline int set_memory_encrypted(unsigned long vaddr, int numpages)
+{
+   return 0;
+}
+
+static inline int set_memory_decrypted(unsigned long vaddr, int numpages)
+{
+   return 0;
+}
+
 #endif
 
 static inline void __init sme_early_encrypt(resource_size_t paddr,
diff --git a/arch/x86/include/asm/pgtable_types.h 
b/arch/x86/include/asm/pgtable_types.h
index f00e70f..456c5cc 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -213,6 +213,7 @@ enum page_cache_mode {
 #define PAGE_KERNEL__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO __pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC   __pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC_NOENC __pgprot(__PAGE_KERNEL_EXEC)
 #define PAGE_KERNEL_RX __pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
 #define PAGE_KERNEL_NOCACHE__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
 #define PAGE_KERNEL_LARGE  __pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
diff --git a/arch/x86/kernel/machine_kexec_64.c 
b/arch/x86/kernel/machine_kexec_64.c
index 307b1f4..b01648c 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -76,7 +76,7 @@ static int init_transition_pgtable(struct kimage *image, 
pgd_t *pgd)
set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
}
pte = pte_offset_kernel(pmd, vaddr);
-   set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC));
+   set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC_NOENC));
return 0;
 err:
free_transition_pgtable(image);
@@ -104,6 +104,7 @@ static int init_pgtable(struct kimage *image, unsigned long 
start_pgtable)
.alloc_pgt_page = alloc_pgt_page,
.context= image,
.pmd_flag   = __PAGE_KERNEL_LARGE_EXEC,
+   .kernpg_flag= _KERNPG_TABLE_NOENC,
};
unsigned long mstart, mend;
pgd_t *level4p;
diff --git 

[RFC PATCH v4 25/28] x86: Access the setup data through sysfs decrypted

[Tom Lendacky]

Use memremap() to map the setup data.  This will make the appropriate
decision as to whether a RAM remapping can be done or if a fallback to
ioremap_cache() is needed (similar to the setup data debugfs support).

Signed-off-by: Tom Lendacky 
---
 arch/x86/kernel/ksysfs.c |   27 ++-
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/arch/x86/kernel/ksysfs.c b/arch/x86/kernel/ksysfs.c
index 4afc67f..d653b3e 100644
--- a/arch/x86/kernel/ksysfs.c
+++ b/arch/x86/kernel/ksysfs.c
@@ -16,6 +16,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -79,12 +80,12 @@ static int get_setup_data_paddr(int nr, u64 *paddr)
*paddr = pa_data;
return 0;
}
-   data = ioremap_cache(pa_data, sizeof(*data));
+   data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
if (!data)
return -ENOMEM;
 
pa_data = data->next;
-   iounmap(data);
+   memunmap(data);
i++;
}
return -EINVAL;
@@ -97,17 +98,17 @@ static int __init get_setup_data_size(int nr, size_t *size)
u64 pa_data = boot_params.hdr.setup_data;
 
while (pa_data) {
-   data = ioremap_cache(pa_data, sizeof(*data));
+   data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
if (!data)
return -ENOMEM;
if (nr == i) {
*size = data->len;
-   iounmap(data);
+   memunmap(data);
return 0;
}
 
pa_data = data->next;
-   iounmap(data);
+   memunmap(data);
i++;
}
return -EINVAL;
@@ -127,12 +128,12 @@ static ssize_t type_show(struct kobject *kobj,
ret = get_setup_data_paddr(nr, );
if (ret)
return ret;
-   data = ioremap_cache(paddr, sizeof(*data));
+   data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
if (!data)
return -ENOMEM;
 
ret = sprintf(buf, "0x%x\n", data->type);
-   iounmap(data);
+   memunmap(data);
return ret;
 }
 
@@ -154,7 +155,7 @@ static ssize_t setup_data_data_read(struct file *fp,
ret = get_setup_data_paddr(nr, );
if (ret)
return ret;
-   data = ioremap_cache(paddr, sizeof(*data));
+   data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
if (!data)
return -ENOMEM;
 
@@ -170,15 +171,15 @@ static ssize_t setup_data_data_read(struct file *fp,
goto out;
 
ret = count;
-   p = ioremap_cache(paddr + sizeof(*data), data->len);
+   p = memremap(paddr + sizeof(*data), data->len, MEMREMAP_WB);
if (!p) {
ret = -ENOMEM;
goto out;
}
memcpy(buf, p + off, count);
-   iounmap(p);
+   memunmap(p);
 out:
-   iounmap(data);
+   memunmap(data);
return ret;
 }
 
@@ -250,13 +251,13 @@ static int __init get_setup_data_total_num(u64 pa_data, 
int *nr)
*nr = 0;
while (pa_data) {
*nr += 1;
-   data = ioremap_cache(pa_data, sizeof(*data));
+   data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
if (!data) {
ret = -ENOMEM;
goto out;
}
pa_data = data->next;
-   iounmap(data);
+   memunmap(data);
}
 
 out:

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 24/28] x86: Access the setup data through debugfs decrypted

[Tom Lendacky]

Use memremap() to map the setup data.  This simplifies the code and will
make the appropriate decision as to whether a RAM remapping can be done
or if a fallback to ioremap_cache() is needed (which includes checking
PageHighMem).

Signed-off-by: Tom Lendacky 
---
 arch/x86/kernel/kdebugfs.c |   30 +++---
 1 file changed, 11 insertions(+), 19 deletions(-)

diff --git a/arch/x86/kernel/kdebugfs.c b/arch/x86/kernel/kdebugfs.c
index bdb83e4..c3d354d 100644
--- a/arch/x86/kernel/kdebugfs.c
+++ b/arch/x86/kernel/kdebugfs.c
@@ -48,17 +48,13 @@ static ssize_t setup_data_read(struct file *file, char 
__user *user_buf,
 
pa = node->paddr + sizeof(struct setup_data) + pos;
pg = pfn_to_page((pa + count - 1) >> PAGE_SHIFT);
-   if (PageHighMem(pg)) {
-   p = ioremap_cache(pa, count);
-   if (!p)
-   return -ENXIO;
-   } else
-   p = __va(pa);
+   p = memremap(pa, count, MEMREMAP_WB);
+   if (!p)
+   return -ENXIO;
 
remain = copy_to_user(user_buf, p, count);
 
-   if (PageHighMem(pg))
-   iounmap(p);
+   memunmap(p);
 
if (remain)
return -EFAULT;
@@ -127,15 +123,12 @@ static int __init create_setup_data_nodes(struct dentry 
*parent)
}
 
pg = pfn_to_page((pa_data+sizeof(*data)-1) >> PAGE_SHIFT);
-   if (PageHighMem(pg)) {
-   data = ioremap_cache(pa_data, sizeof(*data));
-   if (!data) {
-   kfree(node);
-   error = -ENXIO;
-   goto err_dir;
-   }
-   } else
-   data = __va(pa_data);
+   data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
+   if (!data) {
+   kfree(node);
+   error = -ENXIO;
+   goto err_dir;
+   }
 
node->paddr = pa_data;
node->type = data->type;
@@ -143,8 +136,7 @@ static int __init create_setup_data_nodes(struct dentry 
*parent)
error = create_setup_data_node(d, no, node);
pa_data = data->next;
 
-   if (PageHighMem(pg))
-   iounmap(data);
+   memunmap(data);
if (error)
goto err_dir;
no++;

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 23/28] x86/kvm: Enable Secure Memory Encryption of nested page tables

[Tom Lendacky]

Update the KVM support to include the memory encryption mask when creating
and using nested page tables.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/kvm_host.h |3 ++-
 arch/x86/kvm/mmu.c  |8 ++--
 arch/x86/kvm/vmx.c  |3 ++-
 arch/x86/kvm/x86.c  |3 ++-
 4 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index a7066dc..37326b5 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1050,7 +1050,8 @@ struct kvm_arch_async_pf {
 void kvm_mmu_init_vm(struct kvm *kvm);
 void kvm_mmu_uninit_vm(struct kvm *kvm);
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
-   u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask);
+   u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
+   u64 me_mask);
 
 void kvm_mmu_reset_context(struct kvm_vcpu *vcpu);
 void kvm_mmu_slot_remove_write_access(struct kvm *kvm,
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index d8d235b..46f246c 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -123,7 +123,7 @@ enum {
* PT32_LEVEL_BITS))) - 1))
 
 #define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \
-   | shadow_x_mask | shadow_nx_mask)
+   | shadow_x_mask | shadow_nx_mask | shadow_me_mask)
 
 #define ACC_EXEC_MASK1
 #define ACC_WRITE_MASK   PT_WRITABLE_MASK
@@ -178,6 +178,7 @@ struct kvm_shadow_walk_iterator {
 static u64 __read_mostly shadow_dirty_mask;
 static u64 __read_mostly shadow_mmio_mask;
 static u64 __read_mostly shadow_present_mask;
+static u64 __read_mostly shadow_me_mask;
 
 static void mmu_spte_set(u64 *sptep, u64 spte);
 static void mmu_free_roots(struct kvm_vcpu *vcpu);
@@ -285,7 +286,8 @@ static bool check_mmio_spte(struct kvm_vcpu *vcpu, u64 spte)
 }
 
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
-   u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask)
+   u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
+   u64 me_mask)
 {
shadow_user_mask = user_mask;
shadow_accessed_mask = accessed_mask;
@@ -293,6 +295,7 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
shadow_nx_mask = nx_mask;
shadow_x_mask = x_mask;
shadow_present_mask = p_mask;
+   shadow_me_mask = me_mask;
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes);
 
@@ -2546,6 +2549,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
pte_access &= ~ACC_WRITE_MASK;
 
spte |= (u64)pfn << PAGE_SHIFT;
+   spte |= shadow_me_mask;
 
if (pte_access & ACC_WRITE_MASK) {
 
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index a236dec..fac3c27 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -6703,7 +6703,8 @@ static __init int hardware_setup(void)
(enable_ept_ad_bits) ? VMX_EPT_DIRTY_BIT : 0ull,
0ull, VMX_EPT_EXECUTABLE_MASK,
cpu_has_vmx_ept_execute_only() ?
- 0ull : VMX_EPT_READABLE_MASK);
+ 0ull : VMX_EPT_READABLE_MASK,
+   0ull);
ept_set_mmio_spte_mask();
kvm_enable_tdp();
} else
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index a719783..9e6a593 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -67,6 +67,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #define CREATE_TRACE_POINTS
 #include "trace.h"
@@ -6027,7 +6028,7 @@ int kvm_arch_init(void *opaque)
 
kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK,
PT_DIRTY_MASK, PT64_NX_MASK, 0,
-   PT_PRESENT_MASK);
+   PT_PRESENT_MASK, sme_me_mask);
kvm_timer_init();
 
perf_register_guest_info_callbacks(_guest_cbs);

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 22/28] x86: Do not specify encrypted memory for video mappings

[Tom Lendacky]

Since video memory needs to be accessed decrypted, be sure that the
memory encryption mask is not set for the video ranges.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/vga.h   |   13 +
 drivers/gpu/drm/drm_gem.c|2 ++
 drivers/gpu/drm/drm_vm.c |4 
 drivers/gpu/drm/ttm/ttm_bo_vm.c  |7 +--
 drivers/gpu/drm/udl/udl_fb.c |4 
 drivers/video/fbdev/core/fbmem.c |   12 
 6 files changed, 40 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/vga.h b/arch/x86/include/asm/vga.h
index c4b9dc2..5c7567a 100644
--- a/arch/x86/include/asm/vga.h
+++ b/arch/x86/include/asm/vga.h
@@ -7,12 +7,25 @@
 #ifndef _ASM_X86_VGA_H
 #define _ASM_X86_VGA_H
 
+#include 
+
 /*
  * On the PC, we can just recalculate addresses and then
  * access the videoram directly without any black magic.
+ * To support memory encryption however, we need to access
+ * the videoram as decrypted memory.
  */
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+#define VGA_MAP_MEM(x, s)  \
+({ \
+   unsigned long start = (unsigned long)phys_to_virt(x);   \
+   set_memory_decrypted(start, (s) >> PAGE_SHIFT); \
+   start;  \
+})
+#else
 #define VGA_MAP_MEM(x, s) (unsigned long)phys_to_virt(x)
+#endif
 
 #define vga_readb(x) (*(x))
 #define vga_writeb(x, y) (*(y) = (x))
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 465bacd..f9b3be0 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -36,6 +36,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -928,6 +929,7 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned 
long obj_size,
vma->vm_ops = dev->driver->gem_vm_ops;
vma->vm_private_data = obj;
vma->vm_page_prot = 
pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+   vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
/* Take a ref for this mapping of the object, so that the fault
 * handler can dereference the mmap offset's pointer to the object.
diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index bd311c7..f0fc52a 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -40,6 +40,7 @@
 #include 
 #include 
 #endif
+#include 
 #include 
 #include "drm_internal.h"
 #include "drm_legacy.h"
@@ -58,6 +59,9 @@ static pgprot_t drm_io_prot(struct drm_local_map *map,
 {
pgprot_t tmp = vm_get_page_prot(vma->vm_flags);
 
+   /* We don't want graphics memory to be mapped encrypted */
+   tmp = pgprot_decrypted(tmp);
+
 #if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__)
if (map->type == _DRM_REGISTERS && !(map->flags & _DRM_WRITE_COMBINING))
tmp = pgprot_noncached(tmp);
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 68ef993..09f2e73 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -39,6 +39,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #define TTM_BO_VM_NUM_PREFAULT 16
 
@@ -218,9 +219,11 @@ static int ttm_bo_vm_fault(struct vm_area_struct *vma, 
struct vm_fault *vmf)
 * first page.
 */
for (i = 0; i < TTM_BO_VM_NUM_PREFAULT; ++i) {
-   if (bo->mem.bus.is_iomem)
+   if (bo->mem.bus.is_iomem) {
+   /* Iomem should not be marked encrypted */
+   cvma.vm_page_prot = pgprot_decrypted(cvma.vm_page_prot);
pfn = ((bo->mem.bus.base + bo->mem.bus.offset) >> 
PAGE_SHIFT) + page_offset;
-   else {
+   } else {
page = ttm->pages[page_offset];
if (unlikely(!page && i == 0)) {
retval = VM_FAULT_OOM;
diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c
index 167f42c..2207ec0 100644
--- a/drivers/gpu/drm/udl/udl_fb.c
+++ b/drivers/gpu/drm/udl/udl_fb.c
@@ -14,6 +14,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -169,6 +170,9 @@ static int udl_fb_mmap(struct fb_info *info, struct 
vm_area_struct *vma)
pr_notice("mmap() framebuffer addr:%lu size:%lu\n",
  pos, size);
 
+   /* We don't want the framebuffer to be mapped encrypted */
+   vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+
while (size > 0) {
page = vmalloc_to_pfn((void *)pos);
if (remap_pfn_range(vma, start, page, PAGE_SIZE, PAGE_SHARED))
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 76c1ad9..b895e60 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -32,6 +32,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 

[RFC PATCH v4 21/28] x86: Check for memory encryption on the APs

[Tom Lendacky]

Add support to check if memory encryption is active in the kernel and that
it has been enabled on the AP. If memory encryption is active in the kernel
but has not been enabled on the AP, then set the SYS_CFG MSR bit to enable
memory encryption on that AP and allow the AP to continue start up.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/realmode.h  |   12 
 arch/x86/realmode/init.c |4 
 arch/x86/realmode/rm/trampoline_64.S |   17 +
 3 files changed, 33 insertions(+)

diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h
index 230e190..4f7ef53 100644
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -1,6 +1,15 @@
 #ifndef _ARCH_X86_REALMODE_H
 #define _ARCH_X86_REALMODE_H
 
+/*
+ * Flag bit definitions for use with the flags field of the trampoline header
+ * int the CONFIG_X86_64 variant.
+ */
+#define TH_FLAGS_SME_ACTIVE_BIT0
+#define TH_FLAGS_SME_ACTIVEBIT(TH_FLAGS_SME_ACTIVE_BIT)
+
+#ifndef __ASSEMBLY__
+
 #include 
 #include 
 
@@ -38,6 +47,7 @@ struct trampoline_header {
u64 start;
u64 efer;
u32 cr4;
+   u32 flags;
 #endif
 };
 
@@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void)
 void set_real_mode_mem(phys_addr_t mem, size_t size);
 void reserve_real_mode(void);
 
+#endif /* __ASSEMBLY__ */
+
 #endif /* _ARCH_X86_REALMODE_H */
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index 21d7506..5010089 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -102,6 +102,10 @@ static void __init setup_real_mode(void)
trampoline_cr4_features = _header->cr4;
*trampoline_cr4_features = mmu_cr4_features;
 
+   trampoline_header->flags = 0;
+   if (sme_active())
+   trampoline_header->flags |= TH_FLAGS_SME_ACTIVE;
+
trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
trampoline_pgd[0] = trampoline_pgd_entry.pgd;
trampoline_pgd[511] = init_level4_pgt[511].pgd;
diff --git a/arch/x86/realmode/rm/trampoline_64.S 
b/arch/x86/realmode/rm/trampoline_64.S
index dac7b20..a88c3d1 100644
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -30,6 +30,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "realmode.h"
 
.text
@@ -92,6 +93,21 @@ ENTRY(startup_32)
movl%edx, %fs
movl%edx, %gs
 
+   /* Check for memory encryption support */
+   bt  $TH_FLAGS_SME_ACTIVE_BIT, pa_tr_flags
+   jnc .Ldone
+   movl$MSR_K8_SYSCFG, %ecx
+   rdmsr
+   bts $MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax
+   jc  .Ldone
+
+   /*
+* Memory encryption is enabled but the SME enable bit for this
+* CPU has has not been set.  It is safe to set it, so do so.
+*/
+   wrmsr
+.Ldone:
+
movlpa_tr_cr4, %eax
movl%eax, %cr4  # Enable PAE mode
 
@@ -147,6 +163,7 @@ GLOBAL(trampoline_header)
tr_start:   .space  8
GLOBAL(tr_efer) .space  8
GLOBAL(tr_cr4)  .space  4
+   GLOBAL(tr_flags).space  4
 END(trampoline_header)
 
 #include "trampoline_common.S"

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 20/28] iommu/amd: Disable AMD IOMMU if memory encryption is active

[Tom Lendacky]

For now, disable the AMD IOMMU if memory encryption is active. A future
patch will re-enable the function with full memory encryption support.

Signed-off-by: Tom Lendacky 
---
 drivers/iommu/amd_iommu_init.c |7 +++
 1 file changed, 7 insertions(+)

diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index 6799cf9..6df2dd5 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -29,6 +29,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -2544,6 +2545,12 @@ int __init amd_iommu_detect(void)
if (amd_iommu_disabled)
return -ENODEV;
 
+   /* For now, disable the IOMMU if SME is active */
+   if (sme_active()) {
+   pr_notice("AMD-Vi: SME is active, disabling the IOMMU\n");
+   return -ENODEV;
+   }
+
ret = iommu_go_to_state(IOMMU_IVRS_DETECTED);
if (ret)
return ret;

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 19/28] swiotlb: Add warnings for use of bounce buffers with SME

[Tom Lendacky]

Add warnings to let the user know when bounce buffers are being used for
DMA when SME is active.  Since the bounce buffers are not in encrypted
memory, these notifications are to allow the user to determine some
appropriate action - if necessary.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/mem_encrypt.h |   11 +++
 include/linux/dma-mapping.h|   11 +++
 include/linux/mem_encrypt.h|6 ++
 lib/swiotlb.c  |3 +++
 4 files changed, 31 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index 87e816f..5a17f1b 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -26,6 +26,11 @@ static inline bool sme_active(void)
return (sme_me_mask) ? true : false;
 }
 
+static inline u64 sme_dma_mask(void)
+{
+   return ((u64)sme_me_mask << 1) - 1;
+}
+
 void __init sme_early_encrypt(resource_size_t paddr,
  unsigned long size);
 void __init sme_early_decrypt(resource_size_t paddr,
@@ -53,6 +58,12 @@ static inline bool sme_active(void)
 {
return false;
 }
+
+static inline u64 sme_dma_mask(void)
+{
+   return 0ULL;
+}
+
 #endif
 
 static inline void __init sme_early_encrypt(resource_size_t paddr,
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 10c5a17..130bef7 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -10,6 +10,7 @@
 #include 
 #include 
 #include 
+#include 
 
 /**
  * List of possible attributes associated with a DMA mapping. The semantics
@@ -557,6 +558,11 @@ static inline int dma_set_mask(struct device *dev, u64 
mask)
 
if (!dev->dma_mask || !dma_supported(dev, mask))
return -EIO;
+
+   if (sme_active() && (mask < sme_dma_mask()))
+   dev_warn(dev,
+"SME is active, device will require DMA bounce 
buffers\n");
+
*dev->dma_mask = mask;
return 0;
 }
@@ -576,6 +582,11 @@ static inline int dma_set_coherent_mask(struct device 
*dev, u64 mask)
 {
if (!dma_supported(dev, mask))
return -EIO;
+
+   if (sme_active() && (mask < sme_dma_mask()))
+   dev_warn(dev,
+"SME is active, device will require DMA bounce 
buffers\n");
+
dev->coherent_dma_mask = mask;
return 0;
 }
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
index 14a7b9f..6829ff1 100644
--- a/include/linux/mem_encrypt.h
+++ b/include/linux/mem_encrypt.h
@@ -28,6 +28,12 @@ static inline bool sme_active(void)
 {
return false;
 }
+
+static inline u64 sme_dma_mask(void)
+{
+   return 0ULL;
+}
+
 #endif
 
 #endif /* CONFIG_AMD_MEM_ENCRYPT */
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index c463067..aff9353 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -509,6 +509,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
if (no_iotlb_memory)
panic("Can not allocate SWIOTLB buffer earlier and can't now 
provide you with the DMA bounce buffer");
 
+   WARN_ONCE(sme_active(),
+ "SME is active and system is using DMA bounce buffers\n");
+
mask = dma_get_seg_boundary(hwdev);
 
tbl_dma_addr &= mask;

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 18/28] x86: DMA support for memory encryption

[Tom Lendacky]

Since DMA addresses will effectively look like 48-bit addresses when the
memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
device performing the DMA does not support 48-bits. SWIOTLB will be
initialized to create decrypted bounce buffers for use by these devices.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/dma-mapping.h |5 ++-
 arch/x86/include/asm/mem_encrypt.h |5 +++
 arch/x86/kernel/pci-dma.c  |   11 +--
 arch/x86/kernel/pci-nommu.c|2 +
 arch/x86/kernel/pci-swiotlb.c  |8 -
 arch/x86/mm/mem_encrypt.c  |   22 ++
 include/linux/swiotlb.h|1 +
 init/main.c|   13 
 lib/swiotlb.c  |   56 +++-
 9 files changed, 106 insertions(+), 17 deletions(-)

diff --git a/arch/x86/include/asm/dma-mapping.h 
b/arch/x86/include/asm/dma-mapping.h
index 4446162..c9cdcae 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -12,6 +12,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #ifdef CONFIG_ISA
 # define ISA_DMA_BIT_MASK DMA_BIT_MASK(24)
@@ -69,12 +70,12 @@ static inline bool dma_capable(struct device *dev, 
dma_addr_t addr, size_t size)
 
 static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-   return paddr;
+   return paddr | sme_me_mask;
 }
 
 static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
 {
-   return daddr;
+   return daddr & ~sme_me_mask;
 }
 #endif /* CONFIG_X86_DMA_REMAP */
 
diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index e2b7364..87e816f 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -36,6 +36,11 @@ void __init sme_early_decrypt(resource_size_t paddr,
 
 void __init sme_early_init(void);
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void);
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size);
+
 #define __sme_pa(x)(__pa((x)) | sme_me_mask)
 #define __sme_pa_nodebug(x)(__pa_nodebug((x)) | sme_me_mask)
 
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index d30c377..0ce28df 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -92,9 +92,12 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t 
size,
/* CMA can be used only in the context which permits sleeping */
if (gfpflags_allow_blocking(flag)) {
page = dma_alloc_from_contiguous(dev, count, get_order(size));
-   if (page && page_to_phys(page) + size > dma_mask) {
-   dma_release_from_contiguous(dev, page, count);
-   page = NULL;
+   if (page) {
+   addr = phys_to_dma(dev, page_to_phys(page));
+   if (addr + size > dma_mask) {
+   dma_release_from_contiguous(dev, page, count);
+   page = NULL;
+   }
}
}
/* fallback */
@@ -103,7 +106,7 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t 
size,
if (!page)
return NULL;
 
-   addr = page_to_phys(page);
+   addr = phys_to_dma(dev, page_to_phys(page));
if (addr + size > dma_mask) {
__free_pages(page, get_order(size));
 
diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index 00e71ce..922c10d 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -30,7 +30,7 @@ static dma_addr_t nommu_map_page(struct device *dev, struct 
page *page,
 enum dma_data_direction dir,
 unsigned long attrs)
 {
-   dma_addr_t bus = page_to_phys(page) + offset;
+   dma_addr_t bus = phys_to_dma(dev, page_to_phys(page)) + offset;
WARN_ON(size == 0);
if (!check_addr("map_single", dev, bus, size))
return DMA_ERROR_CODE;
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 410efb2..a0677a9 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -12,6 +12,8 @@
 #include 
 #include 
 #include 
+#include 
+
 int swiotlb __read_mostly;
 
 void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
@@ -64,11 +66,13 @@ void x86_swiotlb_free_coherent(struct device *dev, size_t 
size,
  * pci_swiotlb_detect_override - set swiotlb to 1 if necessary
  *
  * This returns non-zero if we are forced to use swiotlb (by the boot
- * option).
+ * option). If memory encryption is enabled then swiotlb will be set
+ * to 1 so that bounce buffers are allocated and used for devices that
+ * do not support the addressing range required for the encryption mask.
  */
 int __init pci_swiotlb_detect_override(void)
 {
-   if 

[RFC PATCH v4 15/28] Add support to access persistent memory in the clear

[Tom Lendacky]

Persistent memory is expected to persist across reboots. The encryption
key used by SME will change across reboots which will result in corrupted
persistent memory.  Persistent memory is handed out by block devices
through memory remapping functions, so be sure not to map this memory as
encrypted.

Signed-off-by: Tom Lendacky 
---
 arch/x86/mm/ioremap.c |2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index b0ff6bc..c6cb921 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -498,6 +498,8 @@ static bool memremap_should_map_encrypted(resource_size_t 
phys_addr,
case E820_TYPE_ACPI:
case E820_TYPE_NVS:
case E820_TYPE_UNUSABLE:
+   case E820_TYPE_PMEM:
+   case E820_TYPE_PRAM:
return false;
default:
break;

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 16/28] x86: Add support for changing memory encryption attribute

[Tom Lendacky]

Add support for changing the memory encryption attribute for one or more
memory pages.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/cacheflush.h |3 ++
 arch/x86/mm/pageattr.c|   66 +
 2 files changed, 69 insertions(+)

diff --git a/arch/x86/include/asm/cacheflush.h 
b/arch/x86/include/asm/cacheflush.h
index 872877d..33ae60a 100644
--- a/arch/x86/include/asm/cacheflush.h
+++ b/arch/x86/include/asm/cacheflush.h
@@ -12,6 +12,7 @@
  * Executability : eXeutable, NoteXecutable
  * Read/Write: ReadOnly, ReadWrite
  * Presence  : NotPresent
+ * Encryption: Encrypted, Decrypted
  *
  * Within a category, the attributes are mutually exclusive.
  *
@@ -47,6 +48,8 @@
 int set_memory_rw(unsigned long addr, int numpages);
 int set_memory_np(unsigned long addr, int numpages);
 int set_memory_4k(unsigned long addr, int numpages);
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
 
 int set_memory_array_uc(unsigned long *addr, int addrinarray);
 int set_memory_array_wc(unsigned long *addr, int addrinarray);
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index 91c5c63..9710f5c 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1742,6 +1742,72 @@ int set_memory_4k(unsigned long addr, int numpages)
__pgprot(0), 1, 0, NULL);
 }
 
+static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
+{
+   struct cpa_data cpa;
+   unsigned long start;
+   int ret;
+
+   /* Nothing to do if the _PAGE_ENC attribute is zero */
+   if (_PAGE_ENC == 0)
+   return 0;
+
+   /* Save original start address since it will be modified */
+   start = addr;
+
+   memset(, 0, sizeof(cpa));
+   cpa.vaddr = 
+   cpa.numpages = numpages;
+   cpa.mask_set = enc ? __pgprot(_PAGE_ENC) : __pgprot(0);
+   cpa.mask_clr = enc ? __pgprot(0) : __pgprot(_PAGE_ENC);
+   cpa.pgd = init_mm.pgd;
+
+   /* Should not be working on unaligned addresses */
+   if (WARN_ONCE(*cpa.vaddr & ~PAGE_MASK,
+ "misaligned address: %#lx\n", *cpa.vaddr))
+   *cpa.vaddr &= PAGE_MASK;
+
+   /* Must avoid aliasing mappings in the highmem code */
+   kmap_flush_unused();
+   vm_unmap_aliases();
+
+   /*
+* Before changing the encryption attribute, we need to flush caches.
+*/
+   if (static_cpu_has(X86_FEATURE_CLFLUSH))
+   cpa_flush_range(start, numpages, 1);
+   else
+   cpa_flush_all(1);
+
+   ret = __change_page_attr_set_clr(, 1);
+
+   /*
+* After changing the encryption attribute, we need to flush TLBs
+* again in case any speculative TLB caching occurred (but no need
+* to flush caches again).  We could just use cpa_flush_all(), but
+* in case TLB flushing gets optimized in the cpa_flush_range()
+* path use the same logic as above.
+*/
+   if (static_cpu_has(X86_FEATURE_CLFLUSH))
+   cpa_flush_range(start, numpages, 0);
+   else
+   cpa_flush_all(0);
+
+   return ret;
+}
+
+int set_memory_encrypted(unsigned long addr, int numpages)
+{
+   return __set_memory_enc_dec(addr, numpages, true);
+}
+EXPORT_SYMBOL(set_memory_encrypted);
+
+int set_memory_decrypted(unsigned long addr, int numpages)
+{
+   return __set_memory_enc_dec(addr, numpages, false);
+}
+EXPORT_SYMBOL(set_memory_decrypted);
+
 int set_pages_uc(struct page *page, int numpages)
 {
unsigned long addr = (unsigned long)page_address(page);

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 14/28] Add support to access boot related data in the clear

[Tom Lendacky]

Boot data (such as EFI related data) is not encrypted when the system is
booted and needs to be mapped decrypted.  Add support to apply the proper
attributes to the EFI page tables and to the early_memremap and memremap
APIs to identify the type of data being accessed so that the proper
encryption attribute can be applied.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/io.h  |3 +
 arch/x86/include/asm/setup.h   |8 +++
 arch/x86/kernel/setup.c|   33 
 arch/x86/mm/ioremap.c  |  111 
 arch/x86/platform/efi/efi_64.c |   16 --
 kernel/memremap.c  |   11 
 mm/early_ioremap.c |   18 +-
 7 files changed, 192 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 7afb0e2..833f7cc 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -381,4 +381,7 @@ extern int __must_check arch_phys_wc_add(unsigned long base,
 #define arch_io_reserve_memtype_wc arch_io_reserve_memtype_wc
 #endif
 
+extern bool arch_memremap_do_ram_remap(resource_size_t offset, size_t size);
+#define arch_memremap_do_ram_remap arch_memremap_do_ram_remap
+
 #endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h
index ac1d5da..8d9 100644
--- a/arch/x86/include/asm/setup.h
+++ b/arch/x86/include/asm/setup.h
@@ -63,6 +63,14 @@ static inline void x86_ce4100_early_setup(void) { }
 #include 
 #include 
 
+struct setup_data_attrs {
+   u64 paddr;
+   unsigned long size;
+};
+
+extern struct setup_data_attrs setup_data_list[];
+extern unsigned int setup_data_list_count;
+
 /*
  * This is set up by the setup-routine at boot-time
  */
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index bd5b9a7..d2234bf 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -148,6 +148,9 @@ int default_check_phys_apicid_present(int phys_apicid)
 
 struct boot_params boot_params;
 
+struct setup_data_attrs setup_data_list[32];
+unsigned int setup_data_list_count;
+
 /*
  * Machine setup..
  */
@@ -419,6 +422,32 @@ static void __init reserve_initrd(void)
 }
 #endif /* CONFIG_BLK_DEV_INITRD */
 
+static void __init update_setup_data_list(u64 pa_data, unsigned long size)
+{
+   unsigned int i;
+
+   for (i = 0; i < setup_data_list_count; i++) {
+   if (setup_data_list[i].paddr != pa_data)
+   continue;
+
+   setup_data_list[i].size = size;
+   break;
+   }
+}
+
+static void __init add_to_setup_data_list(u64 pa_data, unsigned long size)
+{
+   if (!sme_active())
+   return;
+
+   if (!WARN(setup_data_list_count == ARRAY_SIZE(setup_data_list),
+ "exceeded maximum setup data list slots")) {
+   setup_data_list[setup_data_list_count].paddr = pa_data;
+   setup_data_list[setup_data_list_count].size = size;
+   setup_data_list_count++;
+   }
+}
+
 static void __init parse_setup_data(void)
 {
struct setup_data *data;
@@ -428,12 +457,16 @@ static void __init parse_setup_data(void)
while (pa_data) {
u32 data_len, data_type;
 
+   add_to_setup_data_list(pa_data, sizeof(*data));
+
data = early_memremap(pa_data, sizeof(*data));
data_len = data->len + sizeof(struct setup_data);
data_type = data->type;
pa_next = data->next;
early_memunmap(data, sizeof(*data));
 
+   update_setup_data_list(pa_data, data_len);
+
switch (data_type) {
case SETUP_E820_EXT:
e820__memory_setup_extended(pa_data, data_len);
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 2385e70..b0ff6bc 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -13,6 +13,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -21,6 +22,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "physaddr.h"
 
@@ -419,6 +421,115 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+/*
+ * Examine the physical address to determine if it is boot data. Check
+ * it against the boot params structure and EFI tables.
+ */
+static bool memremap_is_setup_data(resource_size_t phys_addr,
+  unsigned long size)
+{
+   unsigned int i;
+   u64 paddr;
+
+   for (i = 0; i < setup_data_list_count; i++) {
+   if (phys_addr < setup_data_list[i].paddr)
+   continue;
+
+   if (phys_addr >= (setup_data_list[i].paddr +
+ setup_data_list[i].size))
+   continue;
+
+   /* Address is within setup data range */
+   return true;
+   }
+
+   

[RFC PATCH v4 13/28] efi: Update efi_mem_type() to return defined EFI mem types

[Tom Lendacky]

Update the efi_mem_type() to return EFI_RESERVED_TYPE instead of a
hardcoded 0.

Signed-off-by: Tom Lendacky 
---
 arch/x86/platform/efi/efi.c |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index a15cf81..6407103 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -1037,7 +1037,7 @@ u32 efi_mem_type(unsigned long phys_addr)
efi_memory_desc_t *md;
 
if (!efi_enabled(EFI_MEMMAP))
-   return 0;
+   return EFI_RESERVED_TYPE;
 
for_each_efi_memory_desc(md) {
if ((md->phys_addr <= phys_addr) &&
@@ -1045,7 +1045,7 @@ u32 efi_mem_type(unsigned long phys_addr)
  (md->num_pages << EFI_PAGE_SHIFT
return md->type;
}
-   return 0;
+   return EFI_RESERVED_TYPE;
 }
 
 static int __init arch_parse_efi_cmdline(char *str)

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 12/28] efi: Add an EFI table address match function

[Tom Lendacky]

This patch adds support that will determine if a supplied physical address
matches the address of an EFI table.

Signed-off-by: Tom Lendacky 
---
 drivers/firmware/efi/efi.c |   33 +
 include/linux/efi.h|7 +++
 2 files changed, 40 insertions(+)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index e7d4040..e8dbcdf 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -55,6 +55,25 @@ struct efi __read_mostly efi = {
 };
 EXPORT_SYMBOL(efi);
 
+static unsigned long *efi_tables[] = {
+   ,
+   ,
+   ,
+   ,
+   ,
+   _systab,
+   _info,
+   ,
+   ,
+   _systab,
+   _vendor,
+   ,
+   _table,
+   ,
+   _table,
+   _attr_table,
+};
+
 static bool disable_runtime;
 static int __init setup_noefi(char *arg)
 {
@@ -855,6 +874,20 @@ int efi_status_to_err(efi_status_t status)
return err;
 }
 
+bool efi_table_address_match(unsigned long phys_addr)
+{
+   unsigned int i;
+
+   if (phys_addr == EFI_INVALID_TABLE_ADDR)
+   return false;
+
+   for (i = 0; i < ARRAY_SIZE(efi_tables); i++)
+   if (*(efi_tables[i]) == phys_addr)
+   return true;
+
+   return false;
+}
+
 #ifdef CONFIG_KEXEC
 static int update_efi_random_seed(struct notifier_block *nb,
  unsigned long code, void *unused)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 94d34e0..7694b23 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1079,6 +1079,8 @@ static inline bool efi_enabled(int feature)
return test_bit(feature, ) != 0;
 }
 extern void efi_reboot(enum reboot_mode reboot_mode, const char *__unused);
+
+extern bool efi_table_address_match(unsigned long phys_addr);
 #else
 static inline bool efi_enabled(int feature)
 {
@@ -1092,6 +1094,11 @@ static inline bool efi_enabled(int feature)
 {
return false;
 }
+
+static inline bool efi_table_address_match(unsigned long phys_addr)
+{
+   return false;
+}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 10/28] x86: Insure that boot memory areas are mapped properly

[Tom Lendacky]

The boot data and command line data are present in memory in a decrypted
state and are copied early in the boot process.  The early page fault
support will map these areas as encrypted, so before attempting to copy
them, add decrypted mappings so the data is accessed properly when copied.

For the initrd, encrypt this data in place. Since the future mapping of the
initrd area will be mapped as encrypted the data will be accessed properly.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/mem_encrypt.h |   11 +
 arch/x86/kernel/head64.c   |   34 +++--
 arch/x86/kernel/setup.c|   10 +
 arch/x86/mm/mem_encrypt.c  |   74 
 4 files changed, 126 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index 3c9052c..e2b7364 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -31,6 +31,9 @@ void __init sme_early_encrypt(resource_size_t paddr,
 void __init sme_early_decrypt(resource_size_t paddr,
  unsigned long size);
 
+void __init sme_map_bootdata(char *real_mode_data);
+void __init sme_unmap_bootdata(char *real_mode_data);
+
 void __init sme_early_init(void);
 
 #define __sme_pa(x)(__pa((x)) | sme_me_mask)
@@ -57,6 +60,14 @@ static inline void __init sme_early_decrypt(resource_size_t 
paddr,
 {
 }
 
+static inline void __init sme_map_bootdata(char *real_mode_data)
+{
+}
+
+static inline void __init sme_unmap_bootdata(char *real_mode_data)
+{
+}
+
 static inline void __init sme_early_init(void)
 {
 }
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 182a4c7..03f8e74 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -46,13 +46,18 @@ static void __init reset_early_page_tables(void)
write_cr3(__sme_pa_nodebug(early_level4_pgt));
 }
 
+void __init __early_pgtable_flush(void)
+{
+   write_cr3(__sme_pa_nodebug(early_level4_pgt));
+}
+
 /* Create a new PMD entry */
-int __init early_make_pgtable(unsigned long address)
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd)
 {
unsigned long physaddr = address - __PAGE_OFFSET;
pgdval_t pgd, *pgd_p;
pudval_t pud, *pud_p;
-   pmdval_t pmd, *pmd_p;
+   pmdval_t *pmd_p;
 
/* Invalid address or early pgt is done ?  */
if (physaddr >= MAXMEM || read_cr3() != 
__sme_pa_nodebug(early_level4_pgt))
@@ -94,12 +99,21 @@ int __init early_make_pgtable(unsigned long address)
memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
*pud_p = (pudval_t)pmd_p - __START_KERNEL_map + phys_base + 
_KERNPG_TABLE;
}
-   pmd = (physaddr & PMD_MASK) + early_pmd_flags;
pmd_p[pmd_index(address)] = pmd;
 
return 0;
 }
 
+int __init early_make_pgtable(unsigned long address)
+{
+   unsigned long physaddr = address - __PAGE_OFFSET;
+   pmdval_t pmd;
+
+   pmd = (physaddr & PMD_MASK) + early_pmd_flags;
+
+   return __early_make_pgtable(address, pmd);
+}
+
 /* Don't add a printk in there. printk relies on the PDA which is not 
initialized 
yet. */
 static void __init clear_bss(void)
@@ -122,6 +136,12 @@ static void __init copy_bootdata(char *real_mode_data)
char * command_line;
unsigned long cmd_line_ptr;
 
+   /*
+* If SME is active, this will create decrypted mappings of the
+* boot data in advance of the copy operations.
+*/
+   sme_map_bootdata(real_mode_data);
+
memcpy(_params, real_mode_data, sizeof boot_params);
sanitize_boot_params(_params);
cmd_line_ptr = get_cmd_line_ptr();
@@ -129,6 +149,14 @@ static void __init copy_bootdata(char *real_mode_data)
command_line = __va(cmd_line_ptr);
memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE);
}
+
+   /*
+* The old boot data is no longer needed and won't be reserved,
+* freeing up that memory for use by the system. If SME is active,
+* we need to remove the mappings that were created so that the
+* memory doesn't remain mapped as decrypted.
+*/
+   sme_unmap_bootdata(real_mode_data);
 }
 
 asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index cab13f7..bd5b9a7 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -114,6 +114,7 @@
 #include 
 #include 
 #include 
+#include 
 
 /*
  * max_low_pfn_mapped: highest direct mapped pfn under 4GB
@@ -376,6 +377,15 @@ static void __init reserve_initrd(void)
!ramdisk_image || !ramdisk_size)
return; /* No initrd provided by bootloader */
 
+   /*
+* If SME is active, this memory will be marked encrypted by the
+* kernel when it is accessed (including 

[RFC PATCH v4 09/28] x86: Add support for early encryption/decryption of memory

[Tom Lendacky]

Add support to be able to either encrypt or decrypt data in place during
the early stages of booting the kernel. This does not change the memory
encryption attribute - it is used for ensuring that data present in either
an encrypted or decrypted memory area is in the proper state (for example
the initrd will have been loaded by the boot loader and will not be
encrypted, but the memory that it resides in is marked as encrypted).

The early_memmap support is enhanced to specify encrypted and decrypted
mappings with and without write-protection. The use of write-protection is
necessary when encrypting data "in place". The write-protect attribute is
considered cacheable for loads, but not stores. This implies that the
hardware will never give the core a dirty line with this memtype.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/mem_encrypt.h |   15 +++
 arch/x86/mm/mem_encrypt.c  |   79 
 2 files changed, 94 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index 547989d..3c9052c 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -26,6 +26,11 @@ static inline bool sme_active(void)
return (sme_me_mask) ? true : false;
 }
 
+void __init sme_early_encrypt(resource_size_t paddr,
+ unsigned long size);
+void __init sme_early_decrypt(resource_size_t paddr,
+ unsigned long size);
+
 void __init sme_early_init(void);
 
 #define __sme_pa(x)(__pa((x)) | sme_me_mask)
@@ -42,6 +47,16 @@ static inline bool sme_active(void)
 }
 #endif
 
+static inline void __init sme_early_encrypt(resource_size_t paddr,
+   unsigned long size)
+{
+}
+
+static inline void __init sme_early_decrypt(resource_size_t paddr,
+   unsigned long size)
+{
+}
+
 static inline void __init sme_early_init(void)
 {
 }
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index d71df97..ac3565c 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -14,6 +14,9 @@
 #include 
 #include 
 
+#include 
+#include 
+
 extern pmdval_t early_pmd_flags;
 
 /*
@@ -24,6 +27,82 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+/* Buffer used for early in-place encryption by BSP, no locking needed */
+static char sme_early_buffer[PAGE_SIZE] __aligned(PAGE_SIZE);
+
+/*
+ * This routine does not change the underlying encryption setting of the
+ * page(s) that map this memory. It assumes that eventually the memory is
+ * meant to be accessed as either encrypted or decrypted but the contents
+ * are currently not in the desired stated.
+ *
+ * This routine follows the steps outlined in the AMD64 Architecture
+ * Programmer's Manual Volume 2, Section 7.10.8 Encrypt-in-Place.
+ */
+static void __init __sme_early_enc_dec(resource_size_t paddr,
+  unsigned long size, bool enc)
+{
+   void *src, *dst;
+   size_t len;
+
+   if (!sme_me_mask)
+   return;
+
+   local_flush_tlb();
+   wbinvd();
+
+   /*
+* There are limited number of early mapping slots, so map (at most)
+* one page at time.
+*/
+   while (size) {
+   len = min_t(size_t, sizeof(sme_early_buffer), size);
+
+   /*
+* Create write protected mappings for the current format
+* of the memory.
+*/
+   src = enc ? early_memremap_decrypted_wp(paddr, len) :
+   early_memremap_encrypted_wp(paddr, len);
+
+   /*
+* Create mappings for the desired format of the memory.
+*/
+   dst = enc ? early_memremap_encrypted(paddr, len) :
+   early_memremap_decrypted(paddr, len);
+
+   /*
+* If a mapping can't be obtained to perform the operation,
+* then eventual access of that area will in the desired
+* mode will cause a crash.
+*/
+   BUG_ON(!src || !dst);
+
+   /*
+* Use a temporary buffer, of cache-line multiple size, to
+* avoid data corruption as documented in the APM.
+*/
+   memcpy(sme_early_buffer, src, len);
+   memcpy(dst, sme_early_buffer, len);
+
+   early_memunmap(dst, len);
+   early_memunmap(src, len);
+
+   paddr += len;
+   size -= len;
+   }
+}
+
+void __init sme_early_encrypt(resource_size_t paddr, unsigned long size)
+{
+   __sme_early_enc_dec(paddr, size, true);
+}
+
+void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
+{
+   __sme_early_enc_dec(paddr, size, false);
+}
+
 void __init 

[RFC PATCH v4 11/28] x86: Add support to determine the E820 type of an address

[Tom Lendacky]

This patch adds support to return the E820 type associated with an address
range.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/e820/api.h   |2 ++
 arch/x86/include/asm/e820/types.h |2 ++
 arch/x86/kernel/e820.c|   26 +++---
 3 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/e820/api.h b/arch/x86/include/asm/e820/api.h
index 8e0f8b8..7c1bdc9 100644
--- a/arch/x86/include/asm/e820/api.h
+++ b/arch/x86/include/asm/e820/api.h
@@ -38,6 +38,8 @@
 extern void e820__reallocate_tables(void);
 extern void e820__register_nosave_regions(unsigned long limit_pfn);
 
+extern enum e820_type e820__get_entry_type(u64 start, u64 end);
+
 /*
  * Returns true iff the specified range [start,end) is completely contained 
inside
  * the ISA region.
diff --git a/arch/x86/include/asm/e820/types.h 
b/arch/x86/include/asm/e820/types.h
index 4adeed0..bf49591 100644
--- a/arch/x86/include/asm/e820/types.h
+++ b/arch/x86/include/asm/e820/types.h
@@ -7,6 +7,8 @@
  * These are the E820 types known to the kernel:
  */
 enum e820_type {
+   E820_TYPE_INVALID   = 0,
+
E820_TYPE_RAM   = 1,
E820_TYPE_RESERVED  = 2,
E820_TYPE_ACPI  = 3,
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index 6e9b26f..2ee7ee2 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -84,7 +84,8 @@ bool e820__mapped_any(u64 start, u64 end, enum e820_type type)
  * Note: this function only works correctly once the E820 table is sorted and
  * not-overlapping (at least for the range specified), which is the case 
normally.
  */
-bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+static struct e820_entry *__e820__mapped_all(u64 start, u64 end,
+enum e820_type type)
 {
int i;
 
@@ -110,9 +111,28 @@ bool __init e820__mapped_all(u64 start, u64 end, enum 
e820_type type)
 * coverage of the desired range exists:
 */
if (start >= end)
-   return 1;
+   return entry;
}
-   return 0;
+
+   return NULL;
+}
+
+/*
+ * This function checks if the entire range  is mapped with type.
+ */
+bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+{
+   return __e820__mapped_all(start, end, type) ? 1 : 0;
+}
+
+/*
+ * This function returns the type associated with the range .
+ */
+enum e820_type e820__get_entry_type(u64 start, u64 end)
+{
+   struct e820_entry *entry = __e820__mapped_all(start, end, 0);
+
+   return entry ? entry->type : E820_TYPE_INVALID;
 }
 
 /*

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 04/28] x86: Handle reduction in physical address size with SME

[Tom Lendacky]

When System Memory Encryption (SME) is enabled, the physical address
space is reduced. Adjust the x86_phys_bits value to reflect this
reduction.

Signed-off-by: Tom Lendacky 
---
 arch/x86/kernel/cpu/common.c |   10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index b33bc06..358208d7 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -771,11 +771,15 @@ void get_cpu_cap(struct cpuinfo_x86 *c)
u64 msr;
 
/*
-* For SME, BIOS support is required. If BIOS has not
-* enabled SME don't advertise the feature.
+* For SME, BIOS support is required. If BIOS has
+* enabled SME adjust x86_phys_bits by the SME
+* physical address space reduction value. If BIOS
+* has not enabled SME don't advertise the feature.
 */
rdmsrl(MSR_K8_SYSCFG, msr);
-   if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+   if (msr & MSR_K8_SYSCFG_MEM_ENCRYPT)
+   c->x86_phys_bits -= (ebx >> 6) & 0x3f;
+   else
eax &= ~0x01;
}
 

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v4 05/28] x86: Add Secure Memory Encryption (SME) support

[Tom Lendacky]

Add support for Secure Memory Encryption (SME). This initial support
provides a Kconfig entry to build the SME support into the kernel and
defines the memory encryption mask that will be used in subsequent
patches to mark pages as encrypted.

Signed-off-by: Tom Lendacky 
---
 arch/x86/Kconfig   |   22 +++
 arch/x86/include/asm/mem_encrypt.h |   42 
 arch/x86/mm/Makefile   |1 +
 arch/x86/mm/mem_encrypt.c  |   21 ++
 include/linux/mem_encrypt.h|   37 
 5 files changed, 123 insertions(+)
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 include/linux/mem_encrypt.h

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index f8fbfc5..a3b8c71 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1395,6 +1395,28 @@ config X86_DIRECT_GBPAGES
  supports them), so don't confuse the user by printing
  that we have them enabled.
 
+config AMD_MEM_ENCRYPT
+   bool "AMD Secure Memory Encryption (SME) support"
+   depends on X86_64 && CPU_SUP_AMD
+   ---help---
+ Say yes to enable support for the encryption of system memory.
+ This requires an AMD processor that supports Secure Memory
+ Encryption (SME).
+
+config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
+   bool "Activate AMD Secure Memory Encryption (SME) by default"
+   default y
+   depends on AMD_MEM_ENCRYPT
+   ---help---
+ Say yes to have system memory encrypted by default if running on
+ an AMD processor that supports Secure Memory Encryption (SME).
+
+ If set to Y, then the encryption of system memory can be
+ deactivated with the mem_encrypt=off command line option.
+
+ If set to N, then the encryption of system memory can be
+ activated with the mem_encrypt=on command line option.
+
 # Common NUMA Features
 config NUMA
bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
new file mode 100644
index 000..ccc53b0
--- /dev/null
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -0,0 +1,42 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __X86_MEM_ENCRYPT_H__
+#define __X86_MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+extern unsigned long sme_me_mask;
+
+static inline bool sme_active(void)
+{
+   return (sme_me_mask) ? true : false;
+}
+
+#else  /* !CONFIG_AMD_MEM_ENCRYPT */
+
+#ifndef sme_me_mask
+#define sme_me_mask0UL
+
+static inline bool sme_active(void)
+{
+   return false;
+}
+#endif
+
+#endif /* CONFIG_AMD_MEM_ENCRYPT */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 96d2b84..44d4d21 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -39,3 +39,4 @@ obj-$(CONFIG_X86_INTEL_MPX)   += mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
 
+obj-$(CONFIG_AMD_MEM_ENCRYPT)  += mem_encrypt.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
new file mode 100644
index 000..b99d469
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt.c
@@ -0,0 +1,21 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+
+/*
+ * Since SME related variables are set early in the boot process they must
+ * reside in the .data section so as not to be zeroed out when the .bss
+ * section is later cleared.
+ */
+unsigned long sme_me_mask __section(.data) = 0;
+EXPORT_SYMBOL_GPL(sme_me_mask);
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
new file mode 100644
index 000..14a7b9f
--- /dev/null
+++ b/include/linux/mem_encrypt.h
@@ -0,0 +1,37 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __MEM_ENCRYPT_H__
+#define __MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+#include 
+
+#else  

[RFC PATCH v4 03/28] x86: Add the Secure Memory Encryption CPU feature

[Tom Lendacky]

Update the CPU features to include identifying and reporting on the
Secure Memory Encryption (SME) feature.  SME is identified by CPUID
0x801f, but requires BIOS support to enable it (set bit 23 of
SYS_CFG MSR).  Only show the SME feature as available if reported by
CPUID and enabled by BIOS.

Signed-off-by: Tom Lendacky 
---
 arch/x86/include/asm/cpufeature.h|7 +--
 arch/x86/include/asm/cpufeatures.h   |5 -
 arch/x86/include/asm/disabled-features.h |3 ++-
 arch/x86/include/asm/msr-index.h |2 ++
 arch/x86/include/asm/required-features.h |3 ++-
 arch/x86/kernel/cpu/common.c |   19 +++
 6 files changed, 34 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/cpufeature.h 
b/arch/x86/include/asm/cpufeature.h
index d59c15c..ea2de6a 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -28,6 +28,7 @@ enum cpuid_leafs
CPUID_8000_000A_EDX,
CPUID_7_ECX,
CPUID_8000_0007_EBX,
+   CPUID_8000_001F_EAX,
 };
 
 #ifdef CONFIG_X86_FEATURE_NAMES
@@ -78,8 +79,9 @@ enum cpuid_leafs
   CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 15, feature_bit) ||\
   CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 16, feature_bit) ||\
   CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 17, feature_bit) ||\
+  CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 18, feature_bit) ||\
   REQUIRED_MASK_CHECK||\
-  BUILD_BUG_ON_ZERO(NCAPINTS != 18))
+  BUILD_BUG_ON_ZERO(NCAPINTS != 19))
 
 #define DISABLED_MASK_BIT_SET(feature_bit) \
 ( CHECK_BIT_IN_MASK_WORD(DISABLED_MASK,  0, feature_bit) ||\
@@ -100,8 +102,9 @@ enum cpuid_leafs
   CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 15, feature_bit) ||\
   CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 16, feature_bit) ||\
   CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 17, feature_bit) ||\
+  CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 18, feature_bit) ||\
   DISABLED_MASK_CHECK||\
-  BUILD_BUG_ON_ZERO(NCAPINTS != 18))
+  BUILD_BUG_ON_ZERO(NCAPINTS != 19))
 
 #define cpu_has(c, bit)
\
(__builtin_constant_p(bit) && REQUIRED_MASK_BIT_SET(bit) ? 1 :  \
diff --git a/arch/x86/include/asm/cpufeatures.h 
b/arch/x86/include/asm/cpufeatures.h
index d45ab4b..331fb81 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -12,7 +12,7 @@
 /*
  * Defines x86 CPU feature bits
  */
-#define NCAPINTS   18  /* N 32-bit words worth of info */
+#define NCAPINTS   19  /* N 32-bit words worth of info */
 #define NBUGINTS   1   /* N 32-bit bug flags */
 
 /*
@@ -296,6 +296,9 @@
 #define X86_FEATURE_SUCCOR (17*32+1) /* Uncorrectable error containment 
and recovery */
 #define X86_FEATURE_SMCA   (17*32+3) /* Scalable MCA */
 
+/* AMD-defined CPU features, CPUID level 0x801f (eax), word 18 */
+#define X86_FEATURE_SME(18*32+0) /* Secure Memory Encryption */
+
 /*
  * BUG word(s)
  */
diff --git a/arch/x86/include/asm/disabled-features.h 
b/arch/x86/include/asm/disabled-features.h
index 85599ad..8b45e08 100644
--- a/arch/x86/include/asm/disabled-features.h
+++ b/arch/x86/include/asm/disabled-features.h
@@ -57,6 +57,7 @@
 #define DISABLED_MASK150
 #define DISABLED_MASK16(DISABLE_PKU|DISABLE_OSPKE)
 #define DISABLED_MASK170
-#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 18)
+#define DISABLED_MASK180
+#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19)
 
 #endif /* _ASM_X86_DISABLED_FEATURES_H */
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 00293a9..e2d0503 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -339,6 +339,8 @@
 #define MSR_K8_TOP_MEM10xc001001a
 #define MSR_K8_TOP_MEM20xc001001d
 #define MSR_K8_SYSCFG  0xc0010010
+#define MSR_K8_SYSCFG_MEM_ENCRYPT_BIT  23
+#define MSR_K8_SYSCFG_MEM_ENCRYPT  BIT_ULL(MSR_K8_SYSCFG_MEM_ENCRYPT_BIT)
 #define MSR_K8_INT_PENDING_MSG 0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK0x1800
diff --git a/arch/x86/include/asm/required-features.h 
b/arch/x86/include/asm/required-features.h
index fac9a5c..6847d85 100644
--- a/arch/x86/include/asm/required-features.h
+++ b/arch/x86/include/asm/required-features.h
@@ -100,6 +100,7 @@
 #define REQUIRED_MASK150
 #define REQUIRED_MASK160
 #define REQUIRED_MASK170
-#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 18)
+#define REQUIRED_MASK180
+#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19)
 
 #endif /* 

[RFC PATCH v4 08/28] x86: Extend the early_memremap support with additional attrs

[Tom Lendacky]

Add to the early_memremap support to be able to specify encrypted and
decrypted mappings with and without write-protection. The use of
write-protection is necessary when encrypting data "in place". The
write-protect attribute is considered cacheable for loads, but not
stores. This implies that the hardware will never give the core a
dirty line with this memtype.

Signed-off-by: Tom Lendacky 
---
 arch/x86/Kconfig |4 +++
 arch/x86/include/asm/fixmap.h|   13 ++
 arch/x86/include/asm/pgtable_types.h |8 ++
 arch/x86/mm/ioremap.c|   44 ++
 include/asm-generic/early_ioremap.h  |2 ++
 mm/early_ioremap.c   |   10 
 6 files changed, 81 insertions(+)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index a3b8c71..581eae4 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1417,6 +1417,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
  If set to N, then the encryption of system memory can be
  activated with the mem_encrypt=on command line option.
 
+config ARCH_USE_MEMREMAP_PROT
+   def_bool y
+   depends on AMD_MEM_ENCRYPT
+
 # Common NUMA Features
 config NUMA
bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index 83e91f0..8233373 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -160,6 +160,19 @@ static inline void __set_fixmap(enum fixed_addresses idx,
  */
 #define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
 
+/*
+ * Early memremap routines used for in-place encryption. The mappings created
+ * by these routines are intended to be used as temporary mappings.
+ */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+ unsigned long size);
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+unsigned long size);
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+ unsigned long size);
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+unsigned long size);
+
 #include 
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/pgtable_types.h 
b/arch/x86/include/asm/pgtable_types.h
index 500fc60..f00e70f 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -161,6 +161,7 @@ enum page_cache_mode {
 
 #define _PAGE_CACHE_MASK   (_PAGE_PAT | _PAGE_PCD | _PAGE_PWT)
 #define _PAGE_NOCACHE  (cachemode2protval(_PAGE_CACHE_MODE_UC))
+#define _PAGE_CACHE_WP (cachemode2protval(_PAGE_CACHE_MODE_WP))
 
 #define PAGE_NONE  __pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)
 #define PAGE_SHARED__pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \
@@ -189,6 +190,7 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_VVAR (__PAGE_KERNEL_RO | _PAGE_USER)
 #define __PAGE_KERNEL_LARGE(__PAGE_KERNEL | _PAGE_PSE)
 #define __PAGE_KERNEL_LARGE_EXEC   (__PAGE_KERNEL_EXEC | _PAGE_PSE)
+#define __PAGE_KERNEL_WP   (__PAGE_KERNEL | _PAGE_CACHE_WP)
 
 #define __PAGE_KERNEL_IO   (__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE   (__PAGE_KERNEL_NOCACHE)
@@ -202,6 +204,12 @@ enum page_cache_mode {
 #define _KERNPG_TABLE  (_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |\
 _PAGE_DIRTY | _PAGE_ENC)
 
+#define __PAGE_KERNEL_ENC  (__PAGE_KERNEL | _PAGE_ENC)
+#define __PAGE_KERNEL_ENC_WP   (__PAGE_KERNEL_WP | _PAGE_ENC)
+
+#define __PAGE_KERNEL_NOENC(__PAGE_KERNEL)
+#define __PAGE_KERNEL_NOENC_WP (__PAGE_KERNEL_WP)
+
 #define PAGE_KERNEL__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO __pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC   __pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index c43b6b3..2385e70 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -419,6 +419,50 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+/* Remap memory with encryption */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+ unsigned long size)
+{
+   return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC);
+}
+
+/*
+ * Remap memory with encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+unsigned long size)
+{
+   /* Be sure the write-protect PAT entry is set for write-protect */
+   if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != 

[RFC PATCH v4 07/28] x86: Provide general kernel support for memory encryption

[Tom Lendacky]

Adding general kernel support for memory encryption includes:
- Modify and create some page table macros to include the Secure Memory
  Encryption (SME) memory encryption mask
- Modify and create some macros for calculating physical and virtual
  memory addresses
- Provide an SME initialization routine to update the protection map with
  the memory encryption mask so that it is used by default
- #undef CONFIG_AMD_MEM_ENCRYPT in the compressed boot path

Signed-off-by: Tom Lendacky 
---
 arch/x86/boot/compressed/pagetable.c |7 +
 arch/x86/include/asm/fixmap.h|7 +
 arch/x86/include/asm/mem_encrypt.h   |   14 +++
 arch/x86/include/asm/page.h  |4 ++-
 arch/x86/include/asm/pgtable.h   |   26 ++--
 arch/x86/include/asm/pgtable_types.h |   45 ++
 arch/x86/include/asm/processor.h |3 ++
 arch/x86/kernel/espfix_64.c  |2 +-
 arch/x86/kernel/head64.c |   12 -
 arch/x86/kernel/head_64.S|   18 +++---
 arch/x86/mm/kasan_init_64.c  |4 ++-
 arch/x86/mm/mem_encrypt.c|   20 +++
 arch/x86/mm/pageattr.c   |3 ++
 include/asm-generic/pgtable.h|8 ++
 14 files changed, 133 insertions(+), 40 deletions(-)

diff --git a/arch/x86/boot/compressed/pagetable.c 
b/arch/x86/boot/compressed/pagetable.c
index 56589d0..411c443 100644
--- a/arch/x86/boot/compressed/pagetable.c
+++ b/arch/x86/boot/compressed/pagetable.c
@@ -15,6 +15,13 @@
 #define __pa(x)  ((unsigned long)(x))
 #define __va(x)  ((void *)((unsigned long)(x)))
 
+/*
+ * The pgtable.h and mm/ident_map.c includes make use of the SME related
+ * information which is not used in the compressed image support. Un-define
+ * the SME support to avoid any compile and link errors.
+ */
+#undef CONFIG_AMD_MEM_ENCRYPT
+
 #include "misc.h"
 
 /* These actually do the work of building the kernel identity maps. */
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index 8554f96..83e91f0 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -153,6 +153,13 @@ static inline void __set_fixmap(enum fixed_addresses idx,
 }
 #endif
 
+/*
+ * Fixmap settings used with memory encryption
+ *   - FIXMAP_PAGE_NOCACHE is used for MMIO so make sure the memory
+ * encryption mask is not part of the page attributes
+ */
+#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
+
 #include 
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index ccc53b0..547989d 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -15,6 +15,8 @@
 
 #ifndef __ASSEMBLY__
 
+#include 
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 extern unsigned long sme_me_mask;
@@ -24,6 +26,11 @@ static inline bool sme_active(void)
return (sme_me_mask) ? true : false;
 }
 
+void __init sme_early_init(void);
+
+#define __sme_pa(x)(__pa((x)) | sme_me_mask)
+#define __sme_pa_nodebug(x)(__pa_nodebug((x)) | sme_me_mask)
+
 #else  /* !CONFIG_AMD_MEM_ENCRYPT */
 
 #ifndef sme_me_mask
@@ -35,6 +42,13 @@ static inline bool sme_active(void)
 }
 #endif
 
+static inline void __init sme_early_init(void)
+{
+}
+
+#define __sme_pa   __pa
+#define __sme_pa_nodebug   __pa_nodebug
+
 #endif /* CONFIG_AMD_MEM_ENCRYPT */
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h
index cf8f619..b1f7bf6 100644
--- a/arch/x86/include/asm/page.h
+++ b/arch/x86/include/asm/page.h
@@ -15,6 +15,8 @@
 
 #ifndef __ASSEMBLY__
 
+#include 
+
 struct page;
 
 #include 
@@ -55,7 +57,7 @@ static inline void copy_user_page(void *to, void *from, 
unsigned long vaddr,
__phys_addr_symbol(__phys_reloc_hide((unsigned long)(x)))
 
 #ifndef __va
-#define __va(x)((void *)((unsigned 
long)(x)+PAGE_OFFSET))
+#define __va(x)((void *)(((unsigned long)(x) & 
~sme_me_mask) + PAGE_OFFSET))
 #endif
 
 #define __boot_va(x)   __va(x)
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 2d81161..b41caab 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -3,6 +3,7 @@
 
 #include 
 #include 
+#include 
 
 /*
  * Macro to mark a page protection value as UC-
@@ -13,6 +14,12 @@
 cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))) \
 : (prot))
 
+/*
+ * Macros to add or remove encryption attribute
+ */
+#define pgprot_encrypted(prot) __pgprot(pgprot_val(prot) | sme_me_mask)
+#define pgprot_decrypted(prot) __pgprot(pgprot_val(prot) & ~sme_me_mask)
+
 #ifndef __ASSEMBLY__
 #include 
 
@@ -153,17 +160,22 @@ static inline int pte_special(pte_t pte)
 
 static inline unsigned long pte_pfn(pte_t pte)
 {
-   return (pte_val(pte) & 

[RFC PATCH v4 06/28] x86: Add support to enable SME during early boot processing

[Tom Lendacky]

This patch adds support to the early boot code to use Secure Memory
Encryption (SME).  Support is added to update the early pagetables with
the memory encryption mask and to encrypt the kernel in place.

The routines to set the encryption mask and perform the encryption are
stub routines for now with full function to be added in a later patch.

A new file, arch/x86/kernel/mem_encrypt_init.c, is introduced to avoid
adding #ifdefs within arch/x86/kernel/head_64.S and allow
arch/x86/mm/mem_encrypt.c to be removed from the build if SME is not
configured. The mem_encrypt_init.c file will contain the necessary #ifdefs
to allow head_64.S to successfully build and call the SME routines.

Signed-off-by: Tom Lendacky 
---
 arch/x86/kernel/Makefile   |2 +
 arch/x86/kernel/head_64.S  |   46 -
 arch/x86/kernel/mem_encrypt_init.c |   50 
 3 files changed, 96 insertions(+), 2 deletions(-)
 create mode 100644 arch/x86/kernel/mem_encrypt_init.c

diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index bdcdb3b..33af80a 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -140,4 +140,6 @@ ifeq ($(CONFIG_X86_64),y)
 
obj-$(CONFIG_PCI_MMCONFIG)  += mmconf-fam10h_64.o
obj-y   += vsmp_64.o
+
+   obj-y   += mem_encrypt_init.o
 endif
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index b467b14..4f8201b 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -91,6 +91,23 @@ startup_64:
jnz bad_address
 
/*
+* Enable Secure Memory Encryption (SME), if supported and enabled.
+* The real_mode_data address is in %rsi and that register can be
+* clobbered by the called function so be sure to save it.
+* Save the returned mask in %r12 for later use.
+*/
+   push%rsi
+   callsme_enable
+   pop %rsi
+   movq%rax, %r12
+
+   /*
+* Add the memory encryption mask to %rbp to include it in the page
+* table fixups.
+*/
+   addq%r12, %rbp
+
+   /*
 * Fixup the physical addresses in the page table
 */
addq%rbp, early_level4_pgt + (L4_START_KERNEL*8)(%rip)
@@ -113,6 +130,7 @@ startup_64:
shrq$PGDIR_SHIFT, %rax
 
leaq(PAGE_SIZE + _KERNPG_TABLE)(%rbx), %rdx
+   addq%r12, %rdx
movq%rdx, 0(%rbx,%rax,8)
movq%rdx, 8(%rbx,%rax,8)
 
@@ -129,6 +147,7 @@ startup_64:
movq%rdi, %rax
shrq$PMD_SHIFT, %rdi
addq$(__PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL), %rax
+   addq%r12, %rax
leaq(_end - 1)(%rip), %rcx
shrq$PMD_SHIFT, %rcx
subq%rdi, %rcx
@@ -162,11 +181,25 @@ startup_64:
cmp %r8, %rdi
jne 1b
 
-   /* Fixup phys_base */
+   /*
+* Fixup phys_base - remove the memory encryption mask from %rbp
+* to obtain the true physical address.
+*/
+   subq%r12, %rbp
addq%rbp, phys_base(%rip)
 
+   /*
+* Encrypt the kernel if SME is active.
+* The real_mode_data address is in %rsi and that register can be
+* clobbered by the called function so be sure to save it.
+*/
+   push%rsi
+   callsme_encrypt_kernel
+   pop %rsi
+
 .Lskip_fixup:
movq$(early_level4_pgt - __START_KERNEL_map), %rax
+   addq%r12, %rax
jmp 1f
 ENTRY(secondary_startup_64)
/*
@@ -186,7 +219,16 @@ ENTRY(secondary_startup_64)
/* Sanitize CPU configuration */
call verify_cpu
 
-   movq$(init_level4_pgt - __START_KERNEL_map), %rax
+   /*
+* Get the SME encryption mask.
+* The real_mode_data address is in %rsi and that register can be
+* clobbered by the called function so be sure to save it.
+*/
+   push%rsi
+   callsme_get_me_mask
+   pop %rsi
+
+   addq$(init_level4_pgt - __START_KERNEL_map), %rax
 1:
 
/* Enable PAE mode and PGE */
diff --git a/arch/x86/kernel/mem_encrypt_init.c 
b/arch/x86/kernel/mem_encrypt_init.c
new file mode 100644
index 000..25af15d
--- /dev/null
+++ b/arch/x86/kernel/mem_encrypt_init.c
@@ -0,0 +1,50 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+#include 
+
+void __init sme_encrypt_kernel(void)
+{
+}
+
+unsigned long __init sme_get_me_mask(void)
+{
+   return sme_me_mask;
+}
+
+unsigned long __init 

[RFC PATCH v4 01/28] x86: Documentation for AMD Secure Memory Encryption (SME)

[Tom Lendacky]

This patch adds a Documenation entry to decribe the AMD Secure Memory
Encryption (SME) feature.

Signed-off-by: Tom Lendacky 
---
 Documentation/admin-guide/kernel-parameters.txt |   11 
 Documentation/x86/amd-memory-encryption.txt |   57 +++
 2 files changed, 68 insertions(+)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt

diff --git a/Documentation/admin-guide/kernel-parameters.txt 
b/Documentation/admin-guide/kernel-parameters.txt
index 110745e..91c40fa 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2145,6 +2145,17 @@
memory contents and reserves bad memory
regions that are detected.
 
+   mem_encrypt=[X86-64] AMD Secure Memory Encryption (SME) control
+   Valid arguments: on, off
+   Default (depends on kernel configuration option):
+ on  (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y)
+ off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n)
+   mem_encrypt=on: Activate SME
+   mem_encrypt=off:Do not activate SME
+
+   Refer to the SME documentation for details on when
+   memory encryption can be activated.
+
mem_sleep_default=  [SUSPEND] Default system suspend mode:
s2idle  - Suspend-To-Idle
shallow - Power-On Suspend or equivalent (if supported)
diff --git a/Documentation/x86/amd-memory-encryption.txt 
b/Documentation/x86/amd-memory-encryption.txt
new file mode 100644
index 000..0938e89
--- /dev/null
+++ b/Documentation/x86/amd-memory-encryption.txt
@@ -0,0 +1,57 @@
+Secure Memory Encryption (SME) is a feature found on AMD processors.
+
+SME provides the ability to mark individual pages of memory as encrypted using
+the standard x86 page tables.  A page that is marked encrypted will be
+automatically decrypted when read from DRAM and encrypted when written to
+DRAM.  SME can therefore be used to protect the contents of DRAM from physical
+attacks on the system.
+
+A page is encrypted when a page table entry has the encryption bit set (see
+below how to determine the position of the bit).  The encryption bit can be
+specified in the cr3 register, allowing the PGD table to be encrypted. Each
+successive level of page tables can also be encrypted.
+
+Support for SME can be determined through the CPUID instruction. The CPUID
+function 0x801f reports information related to SME:
+
+   0x801f[eax]:
+   Bit[0] indicates support for SME
+   0x801f[ebx]:
+   Bit[5:0]  pagetable bit number used to activate memory
+ encryption
+   Bit[11:6] reduction in physical address space, in bits, when
+ memory encryption is enabled (this only affects system
+ physical addresses, not guest physical addresses)
+
+If support for SME is present, MSR 0xc00100010 (SYS_CFG) can be used to
+determine if SME is enabled and/or to enable memory encryption:
+
+   0xc0010010:
+   Bit[23]   0 = memory encryption features are disabled
+ 1 = memory encryption features are enabled
+
+Linux relies on BIOS to set this bit if BIOS has determined that the reduction
+in the physical address space as a result of enabling memory encryption (see
+CPUID information above) will not conflict with the address space resource
+requirements for the system.  If this bit is not set upon Linux startup then
+Linux itself will not set it and memory encryption will not be possible.
+
+The state of SME in the Linux kernel can be documented as follows:
+   - Supported:
+ The CPU supports SME (determined through CPUID instruction).
+
+   - Enabled:
+ Supported and bit 23 of the SYS_CFG MSR is set.
+
+   - Active:
+ Supported, Enabled and the Linux kernel is actively applying
+ the encryption bit to page table entries (the SME mask in the
+ kernel is non-zero).
+
+SME can also be enabled and activated in the BIOS. If SME is enabled and
+activated in the BIOS, then all memory accesses will be encrypted and it will
+not be necessary to activate the Linux memory encryption support.  If the BIOS
+merely enables SME (sets bit 23 of the SYS_CFG MSR), then Linux can activate
+memory encryption.  However, if BIOS does not enable SME, then Linux will not
+attempt to activate memory encryption, even if configured to do so by default
+or the mem_encrypt=on command line parameter is specified.

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [RFC PATCH v1] iommu/io-pgtable-arm-v7s: Check for leaf entry right after finding it

[Oleksandr Tyshchenko]

Hi, Robin.

It would be greatly appreciated if you could test this patch if nobody
has objections to the patch itself.

On Thu, Feb 16, 2017 at 3:52 PM, Oleksandr Tyshchenko
 wrote:
> From: Oleksandr Tyshchenko 
>
> Do a check for already installed leaf entry at the current level before
> performing any actions when trying to map.
>
> This check is already present in arm_v7s_init_pte(), i.e. before
> installing new leaf entry at the current level if conditions to do so
> are met (num_entries > 0).
>
> But, this might be insufficient in case when we have already
> installed block mapping at this level and it is not time to
> install new leaf entry (num_entries == 0).
> In that case we continue walking the page table down with wrong pointer
> to the next level.
>
> So, move check from arm_v7s_init_pte() to __arm_v7s_map() in order to
> avoid all cases.
>
> Signed-off-by: Oleksandr Tyshchenko 
> ---
> This patch does similar things as the patch I have pushed a week ago
> for long-descriptor format [1].
> The reason why this patch is RFC is because I am not sure I did the right
> thing and I even didn't test it.
>
> [1] 
> https://lists.linuxfoundation.org/pipermail/iommu/2017-February/020411.html
> ---
> ---
>  drivers/iommu/io-pgtable-arm-v7s.c | 15 ++-
>  1 file changed, 10 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/iommu/io-pgtable-arm-v7s.c 
> b/drivers/iommu/io-pgtable-arm-v7s.c
> index f50e51c..7f7594b 100644
> --- a/drivers/iommu/io-pgtable-arm-v7s.c
> +++ b/drivers/iommu/io-pgtable-arm-v7s.c
> @@ -364,10 +364,6 @@ static int arm_v7s_init_pte(struct arm_v7s_io_pgtable 
> *data,
> if (WARN_ON(__arm_v7s_unmap(data, iova + i * sz,
> sz, lvl, tblp) != sz))
> return -EINVAL;
> -   } else if (ptep[i]) {
> -   /* We require an unmap first */
> -   WARN_ON(!selftest_running);
> -   return -EEXIST;
> }
>
> pte |= ARM_V7S_PTE_TYPE_PAGE;
> @@ -392,11 +388,20 @@ static int __arm_v7s_map(struct arm_v7s_io_pgtable 
> *data, unsigned long iova,
>  {
> struct io_pgtable_cfg *cfg = >iop.cfg;
> arm_v7s_iopte pte, *cptep;
> -   int num_entries = size >> ARM_V7S_LVL_SHIFT(lvl);
> +   int i = 0, num_entries = size >> ARM_V7S_LVL_SHIFT(lvl);
>
> /* Find our entry at the current level */
> ptep += ARM_V7S_LVL_IDX(iova, lvl);
>
> +   /* Check for already installed leaf entry */
> +   do {
> +   if (ptep[i] && !ARM_V7S_PTE_IS_TABLE(ptep[i], lvl)) {
> +   /* We require an unmap first */
> +   WARN_ON(!selftest_running);
> +   return -EEXIST;
> +   }
> +   } while (++i < num_entries);
> +
> /* If we can install a leaf entry at this level, then do so */
> if (num_entries)
> return arm_v7s_init_pte(data, iova, paddr, prot,
> --
> 2.7.4
>



-- 
Regards,

Oleksandr Tyshchenko
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[RFC PATCH v1] iommu/io-pgtable-arm-v7s: Check for leaf entry right after finding it

[Oleksandr Tyshchenko]

From: Oleksandr Tyshchenko 

Do a check for already installed leaf entry at the current level before
performing any actions when trying to map.

This check is already present in arm_v7s_init_pte(), i.e. before
installing new leaf entry at the current level if conditions to do so
are met (num_entries > 0).

But, this might be insufficient in case when we have already
installed block mapping at this level and it is not time to
install new leaf entry (num_entries == 0).
In that case we continue walking the page table down with wrong pointer
to the next level.

So, move check from arm_v7s_init_pte() to __arm_v7s_map() in order to
avoid all cases.

Signed-off-by: Oleksandr Tyshchenko 
---
This patch does similar things as the patch I have pushed a week ago
for long-descriptor format [1].
The reason why this patch is RFC is because I am not sure I did the right
thing and I even didn't test it.

[1] https://lists.linuxfoundation.org/pipermail/iommu/2017-February/020411.html
---
---
 drivers/iommu/io-pgtable-arm-v7s.c | 15 ++-
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/io-pgtable-arm-v7s.c 
b/drivers/iommu/io-pgtable-arm-v7s.c
index f50e51c..7f7594b 100644
--- a/drivers/iommu/io-pgtable-arm-v7s.c
+++ b/drivers/iommu/io-pgtable-arm-v7s.c
@@ -364,10 +364,6 @@ static int arm_v7s_init_pte(struct arm_v7s_io_pgtable 
*data,
if (WARN_ON(__arm_v7s_unmap(data, iova + i * sz,
sz, lvl, tblp) != sz))
return -EINVAL;
-   } else if (ptep[i]) {
-   /* We require an unmap first */
-   WARN_ON(!selftest_running);
-   return -EEXIST;
}
 
pte |= ARM_V7S_PTE_TYPE_PAGE;
@@ -392,11 +388,20 @@ static int __arm_v7s_map(struct arm_v7s_io_pgtable *data, 
unsigned long iova,
 {
struct io_pgtable_cfg *cfg = >iop.cfg;
arm_v7s_iopte pte, *cptep;
-   int num_entries = size >> ARM_V7S_LVL_SHIFT(lvl);
+   int i = 0, num_entries = size >> ARM_V7S_LVL_SHIFT(lvl);
 
/* Find our entry at the current level */
ptep += ARM_V7S_LVL_IDX(iova, lvl);
 
+   /* Check for already installed leaf entry */
+   do {
+   if (ptep[i] && !ARM_V7S_PTE_IS_TABLE(ptep[i], lvl)) {
+   /* We require an unmap first */
+   WARN_ON(!selftest_running);
+   return -EEXIST;
+   }
+   } while (++i < num_entries);
+
/* If we can install a leaf entry at this level, then do so */
if (num_entries)
return arm_v7s_init_pte(data, iova, paddr, prot,
-- 
2.7.4

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v10 1/9] perf/amd/iommu: Declare pr_fmt and remove unnecessary pr_debug

[Joe Perches]

On Wed, 2017-02-15 at 14:56 -0600, Suravee Suthikulpanit wrote:
> Declare pr_fmt for perf/amd_iommu and remove unnecessary pr_debug.
[]
> diff --git a/arch/x86/events/amd/iommu.c b/arch/x86/events/amd/iommu.c
[]
> @@ -11,6 +11,8 @@
>   * published by the Free Software Foundation.
>   */
>  
> +#define pr_fmt(fmt)  "perf/amd_iommu: " fmt
> +
>  #include 
>  #include 
>  #include 
[]
> @@ -444,24 +440,24 @@ static __init int _init_perf_amd_iommu(
>  
>   raw_spin_lock_init(_iommu->lock);
>  
> - /* Init format attributes */
>   perf_iommu->format_group = _iommu_format_group;
>  
>   /* Init cpumask attributes to only core 0 */
>   cpumask_set_cpu(0, _cpumask);
>   perf_iommu->cpumask_group = _iommu_cpumask_group;
>  
> - /* Init events attributes */
> - if (_init_events_attrs(perf_iommu) != 0)
> - pr_err("perf: amd_iommu: Only support raw events.\n");
> + ret = _init_events_attrs(perf_iommu);
> + if (ret) {
> + pr_err("Error initializing AMD IOMMU perf events.\n");
> + return ret;
> + }
>  
> - /* Init null attributes */
>   perf_iommu->null_group = NULL;
>   perf_iommu->pmu.attr_groups = perf_iommu->attr_groups;
>  
>   ret = perf_pmu_register(_iommu->pmu, name, -1);
>   if (ret) {
> - pr_err("perf: amd_iommu: Failed to initialized.\n");
> + pr_err("Error initializing AMD IOMMU perf counters.\n");
>   amd_iommu_pc_exit();
>   } else {
>   pr_info("perf: amd_iommu: Detected. (%d banks, %d 
> counters/bank)\n",

You should remove the now unnecessary internal prefixes
in the other pr_ uses like the pr_info above.
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu