Re: [PATCH v2 00/17] arm64 SSBD (aka Spectre-v4) mitigation
Hi Marc, On Tue, May 29, 2018 at 01:11:04PM +0100, Marc Zyngier wrote: > This patch series implements the Linux kernel side of the "Spectre-v4" > (CVE-2018-3639) mitigation known as "Speculative Store Bypass Disable" > (SSBD). > > More information can be found at: > > https://bugs.chromium.org/p/project-zero/issues/detail?id=1528 > > https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability > > For all released Arm Cortex-A CPUs that are affected by this issue, then > the preferred mitigation is simply to set a chicken bit in the firmware > during CPU initialisation and therefore no change to Linux is required. > Other CPUs may require the chicken bit to be toggled dynamically (for > example, when switching between user-mode and kernel-mode) and this is > achieved by calling into EL3 via an SMC which has been published as part > of the latest SMCCC specification: > > > https://developer.arm.com/cache-speculation-vulnerability-firmware-specification > > as well as an ATF update for the released ARM cores affected by SSBD: > > https://github.com/ARM-software/arm-trusted-firmware/pull/1392 > > These patches provide the following: > > 1. Safe probing of firmware to establish which CPUs in the system > require calling into EL3 as part of the mitigation. > > 2. For CPUs that require it, call into EL3 on exception entry/exit > from EL0 to apply the SSBD mitigation when running at EL1. > > 3. A command-line option to force the SSBD mitigation to be always on, > always off, or dymamically toggled (default) for CPUs that require > the EL3 call. > > 4. An initial implementation of a prctl() backend for arm64 that allows > userspace tasks to opt-in to the mitigation explicitly. This is > intended to match the interface provided by x86, and so we rely on > their core changes here. The seccomp interface is provided as an > extra set of patches, which I'd like *not* to see merged. The main > reason is that it is invasive, has ugly/unclear semantics, and could > probably be left to the existing prctl interface. I agree with you here. For patches 1-10, then: Acked-by: Will Deacon but I'd prefer to leave the seccomp stuff alone for the moment because I don't think the implicit enabling is necessarily the right thing to do there and supporting it comes at a cost. Will ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
[PATCH v2 0/6] KVM/arm64: Cache maintenance relaxations
This small series makes use of features recently introduced in the
ARMv8 architecture to relax the cache maintenance operations on CPUs
that implement these features.
FWB is the most important one. It allows stage-2 to enforce the
cacheability of memory, no matter what the guest says. It also
mandates that the whole machine is cache coherent (no non-coherent
I/O), meaning we can drop a whole class of cache maintenance
operations.
FWB, when combined with CTL_EL0.{IDC,DIC} allows the removal of all
cache maintenance and tracking of pages being executed.
We also take the opportunity to drop a few useless CMOs that were
applied to the HYP page tables, but that were never necessary. This
ended up requiring quite a few changes in out page table accessors so
that they get consistent barriers. These barriers are a bit on the
heavy side, and could be further optimized, although that's probably
for a different patch series
* From v1:
- Reordered the series in a slightly more consistent order
- Added patches refactoring the PT accessors before dropping
the CMOs
- Removed final dsb(ishst) from the last patch, as all PT accessors
now have their own barriers
- Collected Acks from Catalin
Marc Zyngier (6):
arm64: KVM: Add support for Stage-2 control of memory types and
cacheability
arm64: KVM: Handle Set/Way CMOs as NOPs if FWB is present
arm64: KVM: Avoid marking pages as XN in Stage-2 if CTR_EL0.DIC is set
KVM: arm/arm64: Consolidate page-table accessors
KVM: arm/arm64: Stop using {pmd,pud,pgd}_populate
KVM: arm/arm64: Remove unnecessary CMOs when creating HYP page tables
arch/arm/include/asm/kvm_mmu.h| 14 ++---
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_emulate.h | 2 ++
arch/arm64/include/asm/kvm_mmu.h | 32 +--
arch/arm64/include/asm/memory.h | 7 +
arch/arm64/include/asm/pgtable-prot.h | 24 --
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/kernel/cpufeature.c| 20
arch/arm64/kvm/sys_regs.c | 8 -
virt/kvm/arm/mmu.c| 45 ++-
11 files changed, 125 insertions(+), 32 deletions(-)
--
2.17.1
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
[PATCH v2 1/6] arm64: KVM: Add support for Stage-2 control of memory types and cacheability
Up to ARMv8.3, the combinaison of Stage-1 and Stage-2 attributes
results in the strongest attribute of the two stages. This means
that the hypervisor has to perform quite a lot of cache maintenance
just in case the guest has some non-cacheable mappings around.
ARMv8.4 solves this problem by offering a different mode (FWB) where
Stage-2 has total control over the memory attribute (this is limited
to systems where both I/O and instruction caches are coherent with
the dcache). This is achieved by having a different set of memory
attributes in the page tables, and a new bit set in HCR_EL2.
On such a system, we can then safely sidestep any form of dcache
management.
Acked-by: Catalin Marinas
Signed-off-by: Marc Zyngier
---
arch/arm64/include/asm/cpucaps.h | 3 ++-
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_emulate.h | 2 ++
arch/arm64/include/asm/kvm_mmu.h | 24 +---
arch/arm64/include/asm/memory.h | 7 +++
arch/arm64/include/asm/pgtable-prot.h | 14 --
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/kernel/cpufeature.c| 20
virt/kvm/arm/mmu.c| 4
9 files changed, 66 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index bc51b72fafd4..90e06a49f3e1 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -48,7 +48,8 @@
#define ARM64_HAS_CACHE_IDC27
#define ARM64_HAS_CACHE_DIC28
#define ARM64_HW_DBM 29
+#define ARM64_HAS_STAGE2_FWB 30
-#define ARM64_NCAPS30
+#define ARM64_NCAPS31
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 6dd285e979c9..aa45df752a16 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -23,6 +23,7 @@
#include
/* Hyp Configuration Register (HCR) bits */
+#define HCR_FWB(UL(1) << 46)
#define HCR_TEA(UL(1) << 37)
#define HCR_TERR (UL(1) << 36)
#define HCR_TLOR (UL(1) << 35)
diff --git a/arch/arm64/include/asm/kvm_emulate.h
b/arch/arm64/include/asm/kvm_emulate.h
index 1dab3a984608..dd98fdf33d99 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -63,6 +63,8 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
/* trap error record accesses */
vcpu->arch.hcr_el2 |= HCR_TERR;
}
+ if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB))
+ vcpu->arch.hcr_el2 |= HCR_FWB;
if (test_bit(KVM_ARM_VCPU_EL1_32BIT, vcpu->arch.features))
vcpu->arch.hcr_el2 &= ~HCR_RW;
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 082110993647..9dbca5355029 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -258,6 +258,7 @@ static inline bool kvm_page_empty(void *ptr)
struct kvm;
#define kvm_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l))
+#define kvm_flush_dcache_to_pou(a,l) __clean_dcache_area_pou((a), (l))
static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
{
@@ -268,7 +269,10 @@ static inline void __clean_dcache_guest_page(kvm_pfn_t
pfn, unsigned long size)
{
void *va = page_address(pfn_to_page(pfn));
- kvm_flush_dcache_to_poc(va, size);
+ if (!cpus_have_const_cap(ARM64_HAS_STAGE2_FWB))
+ kvm_flush_dcache_to_poc(va, size);
+ else
+ kvm_flush_dcache_to_pou(va, size);
}
static inline void __invalidate_icache_guest_page(kvm_pfn_t pfn,
@@ -288,20 +292,26 @@ static inline void
__invalidate_icache_guest_page(kvm_pfn_t pfn,
static inline void __kvm_flush_dcache_pte(pte_t pte)
{
- struct page *page = pte_page(pte);
- kvm_flush_dcache_to_poc(page_address(page), PAGE_SIZE);
+ if (!cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) {
+ struct page *page = pte_page(pte);
+ kvm_flush_dcache_to_poc(page_address(page), PAGE_SIZE);
+ }
}
static inline void __kvm_flush_dcache_pmd(pmd_t pmd)
{
- struct page *page = pmd_page(pmd);
- kvm_flush_dcache_to_poc(page_address(page), PMD_SIZE);
+ if (!cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) {
+ struct page *page = pmd_page(pmd);
+ kvm_flush_dcache_to_poc(page_address(page), PMD_SIZE);
+ }
}
static inline void __kvm_flush_dcache_pud(pud_t pud)
{
- struct page *page = pud_page(pud);
- kvm_flush_dcache_to_poc(page_address(page), PUD_SIZE);
+ if (!cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) {
+ struct page *page = pud_page(pud);
+ kvm_flush_dcache_to_poc(page_address(page), PUD_SIZE);
+ }
}
#define
[PATCH v2 2/6] arm64: KVM: Handle Set/Way CMOs as NOPs if FWB is present
Set/Way handling is one of the ugliest corners of KVM. We shouldn't have to handle that, but better safe than sorry. Thankfully, FWB fixes this for us by not requiering any maintenance whatsoever, which means we don't have to emulate S/W CMOs, and don't have to track VM ops either. We still have to trap S/W though, if only to prevent the guest from doing something bad. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/sys_regs.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 6e3b969391fd..9a740f159245 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -195,7 +195,13 @@ static bool access_dcsw(struct kvm_vcpu *vcpu, if (!p->is_write) return read_from_write_only(vcpu, p, r); - kvm_set_way_flush(vcpu); + /* +* Only track S/W ops if we don't have FWB. It still indicates +* that the guest is a bit broken... +*/ + if (!cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) + kvm_set_way_flush(vcpu); + return true; } -- 2.17.1 ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
[PATCH v2 4/6] KVM: arm/arm64: Consolidate page-table accessors
The arm and arm64 KVM page tables accessors are pointlessly different
between the two architectures, and likely both wrong one way or another:
arm64 lacks a dsb(), and arm doesn't use WRITE_ONCE.
Let's unify them.
Signed-off-by: Marc Zyngier
---
arch/arm/include/asm/kvm_mmu.h | 12 ---
arch/arm64/include/asm/kvm_mmu.h | 3 ---
virt/kvm/arm/mmu.c | 35
3 files changed, 31 insertions(+), 19 deletions(-)
diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 707a1f06dc5d..468ff945efa0 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -75,18 +75,6 @@ phys_addr_t kvm_get_idmap_vector(void);
int kvm_mmu_init(void);
void kvm_clear_hyp_idmap(void);
-static inline void kvm_set_pmd(pmd_t *pmd, pmd_t new_pmd)
-{
- *pmd = new_pmd;
- dsb(ishst);
-}
-
-static inline void kvm_set_pte(pte_t *pte, pte_t new_pte)
-{
- *pte = new_pte;
- dsb(ishst);
-}
-
static inline pte_t kvm_s2pte_mkwrite(pte_t pte)
{
pte_val(pte) |= L_PTE_S2_RDWR;
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 9dbca5355029..26c89b63f604 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -170,9 +170,6 @@ phys_addr_t kvm_get_idmap_vector(void);
int kvm_mmu_init(void);
void kvm_clear_hyp_idmap(void);
-#definekvm_set_pte(ptep, pte) set_pte(ptep, pte)
-#definekvm_set_pmd(pmdp, pmd) set_pmd(pmdp, pmd)
-
static inline pte_t kvm_s2pte_mkwrite(pte_t pte)
{
pte_val(pte) |= PTE_S2_RDWR;
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index ba66bf7ae299..c9ed239c0840 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -177,6 +177,33 @@ static void clear_stage2_pmd_entry(struct kvm *kvm, pmd_t
*pmd, phys_addr_t addr
put_page(virt_to_page(pmd));
}
+static inline void kvm_set_pte(pte_t *ptep, pte_t new_pte)
+{
+ WRITE_ONCE(*ptep, new_pte);
+ dsb(ishst);
+}
+
+static inline void kvm_set_pmd(pmd_t *pmdp, pmd_t new_pmd)
+{
+ WRITE_ONCE(*pmdp, new_pmd);
+ dsb(ishst);
+}
+
+static inline void kvm_pmd_populate(pmd_t *pmdp, pte_t *ptep)
+{
+ pmd_populate_kernel(NULL, pmdp, ptep);
+}
+
+static inline void kvm_pud_populate(pud_t *pudp, pmd_t *pmdp)
+{
+ pud_populate(NULL, pudp, pmdp);
+}
+
+static inline void kvm_pgd_populate(pgd_t *pgdp, pud_t *pudp)
+{
+ pgd_populate(NULL, pgdp, pudp);
+}
+
/*
* Unmapping vs dcache management:
*
@@ -603,7 +630,7 @@ static int create_hyp_pmd_mappings(pud_t *pud, unsigned
long start,
kvm_err("Cannot allocate Hyp pte\n");
return -ENOMEM;
}
- pmd_populate_kernel(NULL, pmd, pte);
+ kvm_pmd_populate(pmd, pte);
get_page(virt_to_page(pmd));
kvm_flush_dcache_to_poc(pmd, sizeof(*pmd));
}
@@ -636,7 +663,7 @@ static int create_hyp_pud_mappings(pgd_t *pgd, unsigned
long start,
kvm_err("Cannot allocate Hyp pmd\n");
return -ENOMEM;
}
- pud_populate(NULL, pud, pmd);
+ kvm_pud_populate(pud, pmd);
get_page(virt_to_page(pud));
kvm_flush_dcache_to_poc(pud, sizeof(*pud));
}
@@ -673,7 +700,7 @@ static int __create_hyp_mappings(pgd_t *pgdp, unsigned long
ptrs_per_pgd,
err = -ENOMEM;
goto out;
}
- pgd_populate(NULL, pgd, pud);
+ kvm_pgd_populate(pgd, pud);
get_page(virt_to_page(pgd));
kvm_flush_dcache_to_poc(pgd, sizeof(*pgd));
}
@@ -1092,7 +1119,7 @@ static int stage2_set_pte(struct kvm *kvm, struct
kvm_mmu_memory_cache *cache,
if (!cache)
return 0; /* ignore calls from kvm_set_spte_hva */
pte = mmu_memory_cache_alloc(cache);
- pmd_populate_kernel(NULL, pmd, pte);
+ kvm_pmd_populate(pmd, pte);
get_page(virt_to_page(pmd));
}
--
2.17.1
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
[PATCH v2 5/6] KVM: arm/arm64: Stop using {pmd,pud,pgd}_populate
The {pmd,pud,pgd}_populate accessors usage in the kernel have always
been a bit weird in KVM. We don't have a struct mm to pass (and
neither does the kernel most of the time, but still...), and
the 32bit code has all kind of cache maintenance that doesn't make
sense on ARMv7+ when MP extensions are mandatory (which is the
case when the VEs are present).
Let's bite the bullet and provide our own implementations. The
only bit of architectural code left has to do with building the table
entry itself (arm64 having up to 52bit PA, arm lacking PUD level).
Signed-off-by: Marc Zyngier
---
arch/arm/include/asm/kvm_mmu.h | 4
arch/arm64/include/asm/kvm_mmu.h | 7 +++
virt/kvm/arm/mmu.c | 8 +---
3 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 468ff945efa0..a94ef9833bd3 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -75,6 +75,10 @@ phys_addr_t kvm_get_idmap_vector(void);
int kvm_mmu_init(void);
void kvm_clear_hyp_idmap(void);
+#define kvm_mk_pmd(ptep) __pmd(__pa(ptep) | PMD_TYPE_TABLE)
+#define kvm_mk_pud(pmdp) __pud(__pa(pmdp) | PMD_TYPE_TABLE)
+#define kvm_mk_pgd(pudp) ({ BUILD_BUG(); 0; })
+
static inline pte_t kvm_s2pte_mkwrite(pte_t pte)
{
pte_val(pte) |= L_PTE_S2_RDWR;
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 26c89b63f604..22c9f7cfdf93 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -170,6 +170,13 @@ phys_addr_t kvm_get_idmap_vector(void);
int kvm_mmu_init(void);
void kvm_clear_hyp_idmap(void);
+#define kvm_mk_pmd(ptep) \
+ __pmd(__phys_to_pmd_val(__pa(ptep) | PMD_TYPE_TABLE))
+#define kvm_mk_pud(pmdp) \
+ __pud(__phys_to_pud_val(__pa(pmdp) | PMD_TYPE_TABLE))
+#define kvm_mk_pgd(pudp) \
+ __pgd(__phys_to_pgd_val(__pa(pudp) | PUD_TYPE_TABLE))
+
static inline pte_t kvm_s2pte_mkwrite(pte_t pte)
{
pte_val(pte) |= PTE_S2_RDWR;
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index c9ed239c0840..ad1980d2118a 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -191,17 +191,19 @@ static inline void kvm_set_pmd(pmd_t *pmdp, pmd_t new_pmd)
static inline void kvm_pmd_populate(pmd_t *pmdp, pte_t *ptep)
{
- pmd_populate_kernel(NULL, pmdp, ptep);
+ kvm_set_pmd(pmdp, kvm_mk_pmd(ptep));
}
static inline void kvm_pud_populate(pud_t *pudp, pmd_t *pmdp)
{
- pud_populate(NULL, pudp, pmdp);
+ WRITE_ONCE(*pudp, kvm_mk_pud(pmdp));
+ dsb(ishst);
}
static inline void kvm_pgd_populate(pgd_t *pgdp, pud_t *pudp)
{
- pgd_populate(NULL, pgdp, pudp);
+ WRITE_ONCE(*pgdp, kvm_mk_pgd(pudp));
+ dsb(ishst);
}
/*
--
2.17.1
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
