Re: [PATCH] crypto: powerpc: remove unneeded semicolon
On Tue, Feb 02, 2021 at 11:17:30AM +0800, Yang Li wrote: > Eliminate the following coccicheck warning: > ./arch/powerpc/crypto/sha256-spe-glue.c:132:2-3: Unneeded > semicolon > > Reported-by: Abaci Robot > Signed-off-by: Yang Li > --- > arch/powerpc/crypto/sha256-spe-glue.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Patch applied. Thanks. -- Email: Herbert Xu Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Re: [PATCH 00/20] Rid W=1 warnings in Crypto
On Thu, Feb 04, 2021 at 11:09:40AM +, Lee Jones wrote: > This set is part of a larger effort attempting to clean-up W=1 > kernel builds, which are currently overwhelmingly riddled with > niggly little warnings. > > This is set 1 of 2 sets required to fully clean Crypto. > > Lee Jones (20): > crypto: hisilicon: sec_drv: Supply missing description for > 'sec_queue_empty()'s 'queue' param > crypto: bcm: util: Repair a couple of documentation formatting issues > crypto: chelsio: chcr_core: File headers are not good candidates for > kernel-doc > crypto: ux500: hash: hash_core: Fix worthy kernel-doc headers and > remove others > crypto: bcm: spu: Fix formatting and misspelling issues > crypto: keembay: ocs-hcu: Fix incorrectly named functions/structs > crypto: bcm: spu2: Fix a whole host of kernel-doc misdemeanours > crypto: ux500: cryp: Demote some conformant non-kernel headers fix > another > crypto: ux500: cryp_irq: File headers are not good kernel-doc > candidates > crypto: chelsio: chcr_algo: Fix a couple of kernel-doc issues caused > by doc-rot > crypto: ux500: cryp_core: Fix formatting issue and add description for > 'session_id' > crypto: atmel-ecc: Struct headers need to start with keyword 'struct' > crypto: bcm: cipher: Provide description for 'req' and fix formatting > issues > crypto: caam: caampkc: Provide the name of the function > crypto: caam: caamalg_qi2: Supply a couple of 'fallback' related > descriptions > crypto: vmx: Source headers are not good kernel-doc candidates > crypto: nx: nx-aes-cbc: Headers comments should not be kernel-doc > crypto: nx: nx_debugfs: Header comments should not be kernel-doc > crypto: nx: Demote header comment and add description for 'nbytes' > crypto: cavium: nitrox_isr: Demote non-compliant kernel-doc headers Thanks for doing this. But please don't split the patches at the file level. Instead split them at the driver level. For example, all of your bcm changes should be one patch. -- Email: Herbert Xu Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
[powerpc:next] BUILD SUCCESS e7eb919057c3450cdd9d335e4a23a4da8da58db4
defconfig nios2allyesconfig cskydefconfig xtensa allyesconfig h8300allyesconfig arc defconfig parisc defconfig s390 allyesconfig s390 allmodconfig parisc allyesconfig s390defconfig i386 allyesconfig sparcallyesconfig sparc defconfig i386 tinyconfig i386defconfig mips allyesconfig mips allmodconfig powerpc allyesconfig powerpc allmodconfig powerpc allnoconfig x86_64 randconfig-a006-20210209 x86_64 randconfig-a001-20210209 x86_64 randconfig-a005-20210209 x86_64 randconfig-a004-20210209 x86_64 randconfig-a002-20210209 x86_64 randconfig-a003-20210209 i386 randconfig-a001-20210209 i386 randconfig-a005-20210209 i386 randconfig-a003-20210209 i386 randconfig-a002-20210209 i386 randconfig-a006-20210209 i386 randconfig-a004-20210209 i386 randconfig-a016-20210209 i386 randconfig-a013-20210209 i386 randconfig-a012-20210209 i386 randconfig-a014-20210209 i386 randconfig-a011-20210209 i386 randconfig-a015-20210209 riscvallyesconfig riscvnommu_virt_defconfig riscv allnoconfig riscv defconfig riscv rv32_defconfig riscvallmodconfig x86_64 rhel x86_64 allyesconfig x86_64rhel-7.6-kselftests x86_64 defconfig x86_64 rhel-8.3 x86_64 rhel-8.3-kbuiltin x86_64 kexec clang tested configs: x86_64 randconfig-a013-20210209 x86_64 randconfig-a014-20210209 x86_64 randconfig-a015-20210209 x86_64 randconfig-a012-20210209 x86_64 randconfig-a016-20210209 x86_64 randconfig-a011-20210209 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-...@lists.01.org
[powerpc:next-test] BUILD REGRESSION 5811244192fc4e18c001c69300044c2acf30bd91
tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next-test branch HEAD: 5811244192fc4e18c001c69300044c2acf30bd91 powerpc/64s: power4 nap fixup in C Error/Warning reports: https://lore.kernel.org/linuxppc-dev/202102100438.uvrtpnen-...@intel.com https://lore.kernel.org/linuxppc-dev/202102100601.eltcmofo-...@intel.com https://lore.kernel.org/linuxppc-dev/202102101057.kqissfvf-...@intel.com Error/Warning in current branch: arch/powerpc/mm/book3s64/radix_tlb.c:646:6: warning: no previous prototype for function 'exit_lazy_flush_tlb' [-Wmissing-prototypes] arch/powerpc/platforms/83xx/km83xx.c:183:19: error: 'mpc83xx_setup_pci' undeclared here (not in a function); did you mean 'mpc83xx_setup_arch'? arch/powerpc/platforms/amigaone/setup.c:73:13: error: no previous prototype for 'amigaone_discover_phbs' [-Werror=missing-prototypes] possible Error/Warning in current branch: arch/powerpc/mm/book3s64/radix_tlb.c:646:6: error: no previous prototype for 'exit_lazy_flush_tlb' [-Werror=missing-prototypes] Error/Warning ids grouped by kconfigs: gcc_recent_errors |-- powerpc-cell_defconfig | `-- arch-powerpc-mm-book3s64-radix_tlb.c:error:no-previous-prototype-for-exit_lazy_flush_tlb |-- powerpc-kmeter1_defconfig | `-- arch-powerpc-platforms-83xx-km83xx.c:error:mpc83xx_setup_pci-undeclared-here-(not-in-a-function) |-- powerpc-pasemi_defconfig | `-- arch-powerpc-mm-book3s64-radix_tlb.c:error:no-previous-prototype-for-exit_lazy_flush_tlb |-- powerpc-ppc64_defconfig | `-- arch-powerpc-mm-book3s64-radix_tlb.c:error:no-previous-prototype-for-exit_lazy_flush_tlb `-- powerpc64-randconfig-r036-20210209 `-- arch-powerpc-platforms-amigaone-setup.c:error:no-previous-prototype-for-amigaone_discover_phbs clang_recent_errors `-- powerpc-randconfig-r026-20210209 `-- arch-powerpc-mm-book3s64-radix_tlb.c:warning:no-previous-prototype-for-function-exit_lazy_flush_tlb elapsed time: 720m configs tested: 116 configs skipped: 2 gcc tested configs: arm defconfig arm64allyesconfig arm64 defconfig arm allyesconfig arm allmodconfig powerpc pasemi_defconfig arm mxs_defconfig arc alldefconfig mips ath79_defconfig c6xevmc6474_defconfig arm pxa3xx_defconfig powerpcsocrates_defconfig xtensasmp_lx200_defconfig mipsjmr3927_defconfig powerpc ppc64_defconfig c6x allyesconfig xtensa audio_kc705_defconfig sh allmodconfig armvt8500_v6_v7_defconfig arm pxa_defconfig mips xway_defconfig arm netwinder_defconfig mipsgpr_defconfig arcvdk_hs38_defconfig mips rs90_defconfig powerpc sequoia_defconfig powerpc taishan_defconfig alpha defconfig mips maltaaprp_defconfig arc allyesconfig m68k m5475evb_defconfig arm stm32_defconfig mips malta_defconfig m68kmvme147_defconfig arm cns3420vb_defconfig alphaallyesconfig sh shx3_defconfig arm ixp4xx_defconfig xtensa nommu_kc705_defconfig armmvebu_v7_defconfig arm s3c2410_defconfig powerpccell_defconfig sh rts7751r2dplus_defconfig arm hackkit_defconfig sh rsk7201_defconfig arm imx_v4_v5_defconfig arm am200epdkit_defconfig microblaze mmu_defconfig sh sh7785lcr_32bit_defconfig ia64 allmodconfig ia64defconfig ia64 allyesconfig m68k allmodconfig m68kdefconfig m68k allyesconfig nios2 defconfig nds32 allnoconfig nds32 defconfig nios2allyesconfig cskydefconfig xtensa allyesconfig h8300allyesconfig arc defconfig parisc defconfig s390
[powerpc:fixes-test] BUILD SUCCESS 8c511eff1827239f24ded212b1bcda7ca5b16203
smp_lx200_defconfig mipsjmr3927_defconfig powerpc ppc64_defconfig c6x allyesconfig mips rm200_defconfig arm aspeed_g5_defconfig powerpc bamboo_defconfig powerpc mpc85xx_cds_defconfig xtensa audio_kc705_defconfig sh allmodconfig m68k alldefconfig powerpcadder875_defconfig shmigor_defconfig mips sb1250_swarm_defconfig armzeus_defconfig powerpc pq2fads_defconfig mipse55_defconfig powerpc mpc837x_rdb_defconfig armvt8500_v6_v7_defconfig arm pxa_defconfig mips xway_defconfig arm netwinder_defconfig mipsgpr_defconfig arcvdk_hs38_defconfig powerpc sequoia_defconfig powerpc taishan_defconfig alpha defconfig mips maltaaprp_defconfig m68k m5475evb_defconfig mips malta_defconfig m68kmvme147_defconfig arm cns3420vb_defconfig alphaallyesconfig xtensa nommu_kc705_defconfig armmvebu_v7_defconfig arm s3c2410_defconfig powerpccell_defconfig sh rts7751r2dplus_defconfig arm hackkit_defconfig sh rsk7201_defconfig m68k allyesconfig arc axs101_defconfig openrisc simple_smp_defconfig powerpc mpc8540_ads_defconfig m68kmac_defconfig shdreamcast_defconfig mips decstation_r4k_defconfig arm imx_v4_v5_defconfig microblaze mmu_defconfig sh sh7785lcr_32bit_defconfig nios2 3c120_defconfig powerpc powernv_defconfig sh sh7710voipgw_defconfig h8300 edosk2674_defconfig powerpc ppc40x_defconfig m68k multi_defconfig arm socfpga_defconfig um x86_64_defconfig armlart_defconfig armkeystone_defconfig ia64 allmodconfig ia64defconfig ia64 allyesconfig m68k allmodconfig m68kdefconfig nds32 defconfig nios2allyesconfig cskydefconfig xtensa allyesconfig h8300allyesconfig arc defconfig parisc defconfig s390 allyesconfig s390 allmodconfig parisc allyesconfig s390defconfig i386 allyesconfig sparcallyesconfig sparc defconfig i386 tinyconfig i386defconfig mips allyesconfig mips allmodconfig powerpc allyesconfig powerpc allmodconfig powerpc allnoconfig x86_64 randconfig-a006-20210209 x86_64 randconfig-a001-20210209 x86_64 randconfig-a005-20210209 x86_64 randconfig-a004-20210209 x86_64 randconfig-a002-20210209 x86_64 randconfig-a003-20210209 i386 randconfig-a001-20210209 i386 randconfig-a005-20210209 i386 randconfig-a003-20210209 i386 randconfig-a002-20210209 i386 randconfig-a006-20210209 i386 randconfig-a004-20210209 i386 randconfig-a001-20210206 i386 randconfig-a005-20210206 i386 randconfig-a003-20210206 i386 randconfig-a006-20210206 i386 randconfig-a002-20210206 i386 randconfig-a004-20210206 x86_64 randconfig-a013-20210206 x86_64 randconfig-a014-20210206 x86_64 randconfig-a015-20210206 x86_64 randconfig-a011-20210206 x86_64 randconfig-a016-20210206 x86_64
[powerpc:merge] BUILD SUCCESS 393ff0ee1405c44af2720c953d1090b9bb8d0226
tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git merge branch HEAD: 393ff0ee1405c44af2720c953d1090b9bb8d0226 Automatic merge of 'master' into merge (2021-02-07 21:53) elapsed time: 1361m configs tested: 125 configs skipped: 2 The following configs have been built successfully. More configs may be tested in the coming days. gcc tested configs: arm defconfig arm64allyesconfig arm64 defconfig arm allyesconfig arm allmodconfig m68k m5208evb_defconfig shedosk7760_defconfig sparcalldefconfig arm palmz72_defconfig h8300h8300h-sim_defconfig powerpc ep88xc_defconfig arm zx_defconfig x86_64 alldefconfig mips rb532_defconfig powerpc mpc836x_mds_defconfig sh rts7751r2d1_defconfig powerpcklondike_defconfig arm sunxi_defconfig powerpcsam440ep_defconfig nios2 defconfig m68km5307c3_defconfig m68k m5275evb_defconfig xtensa common_defconfig powerpc mpc8315_rdb_defconfig arm versatile_defconfig powerpc stx_gp3_defconfig shsh7785lcr_defconfig riscvnommu_k210_defconfig mips rm200_defconfig arm aspeed_g5_defconfig powerpc bamboo_defconfig powerpc mpc85xx_cds_defconfig armzeus_defconfig powerpc pq2fads_defconfig mipse55_defconfig powerpc mpc837x_rdb_defconfig arcvdk_hs38_defconfig mips rs90_defconfig powerpc sequoia_defconfig powerpc taishan_defconfig mips maltaaprp_defconfig arm cns3420vb_defconfig alphaallyesconfig sh shx3_defconfig arm ixp4xx_defconfig xtensa nommu_kc705_defconfig arm hackkit_defconfig m68k m5475evb_defconfig arm stm32_defconfig sh rsk7201_defconfig h8300 edosk2674_defconfig powerpcsocrates_defconfig powerpc mpc832x_rdb_defconfig powerpc mpc8313_rdb_defconfig powerpc ppc40x_defconfig m68k multi_defconfig ia64 allmodconfig ia64defconfig ia64 allyesconfig m68k allmodconfig m68kdefconfig m68k allyesconfig arc allyesconfig nds32 allnoconfig c6x allyesconfig nds32 defconfig nios2allyesconfig cskydefconfig alpha defconfig xtensa allyesconfig h8300allyesconfig arc defconfig sh allmodconfig parisc defconfig s390 allyesconfig s390 allmodconfig parisc allyesconfig s390defconfig i386 allyesconfig sparcallyesconfig sparc defconfig i386 tinyconfig i386defconfig mips allyesconfig mips allmodconfig powerpc allyesconfig powerpc allmodconfig powerpc allnoconfig x86_64 randconfig-a006-20210209 x86_64 randconfig-a001-20210209 x86_64 randconfig-a005-20210209 x86_64 randconfig-a004-20210209 x86_64 randconfig-a002-20210209 x86_64 randconfig-a003-20210209 i386 randconfig-a001-20210209 i386 randconfig-a005-20210209 i386 randconfig-a003-20210209 i386 randconfig-a002-20210209 i386 randconfig-a006-20210209 i386 randconfig-a004-20210209 i386
[PATCH] selftests/powerpc: Fix L1D flushing tests for Power10
The rfi_flush and entry_flush selftests work by using the PM_LD_MISS_L1
perf event to count L1D misses. The value of this event has changed
over time:
- Power7 uses 0x400f0
- Power8 and Power9 use both 0x400f0 and 0x3e054
- Power10 uses only 0x3e054
Update these selftests to use the value 0x3e054 on P10 and later,
fixing the tests from finding 0 events.
Signed-off-by: Russell Currey
---
tools/testing/selftests/powerpc/security/entry_flush.c | 4 +++-
tools/testing/selftests/powerpc/security/flush_utils.c | 9 +
tools/testing/selftests/powerpc/security/flush_utils.h | 9 -
tools/testing/selftests/powerpc/security/rfi_flush.c | 4 +++-
4 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/powerpc/security/entry_flush.c
b/tools/testing/selftests/powerpc/security/entry_flush.c
index 78cf914fa321..ffcc93be7df1 100644
--- a/tools/testing/selftests/powerpc/security/entry_flush.c
+++ b/tools/testing/selftests/powerpc/security/entry_flush.c
@@ -26,6 +26,7 @@ int entry_flush_test(void)
__u64 l1d_misses_total = 0;
unsigned long iterations = 10, zero_size = 24 * 1024;
unsigned long l1d_misses_expected;
+ unsigned long perf_l1d_miss_event;
int rfi_flush_orig;
int entry_flush, entry_flush_orig;
@@ -53,7 +54,8 @@ int entry_flush_test(void)
entry_flush = entry_flush_orig;
- fd = perf_event_open_counter(PERF_TYPE_RAW, /* L1d miss */ 0x400f0, -1);
+ perf_l1d_miss_event = get_perf_l1d_miss_event();
+ fd = perf_event_open_counter(PERF_TYPE_RAW, perf_l1d_miss_event, -1);
FAIL_IF(fd < 0);
p = (char *)memalign(zero_size, CACHELINE_SIZE);
diff --git a/tools/testing/selftests/powerpc/security/flush_utils.c
b/tools/testing/selftests/powerpc/security/flush_utils.c
index 0c3c4c40c7fb..7a5ef1a7a228 100644
--- a/tools/testing/selftests/powerpc/security/flush_utils.c
+++ b/tools/testing/selftests/powerpc/security/flush_utils.c
@@ -68,3 +68,12 @@ void set_dscr(unsigned long val)
asm volatile("mtspr %1,%0" : : "r" (val), "i" (SPRN_DSCR));
}
+
+unsigned long get_perf_l1d_miss_event(void)
+{
+ bool is_p10_or_later = ((mfspr(SPRN_PVR) >> 16) & 0x) >= 0x80;
+
+ if (is_p10_or_later)
+ return PERF_L1D_MISS_P10;
+ return PERF_L1D_MISS_P7;
+}
diff --git a/tools/testing/selftests/powerpc/security/flush_utils.h
b/tools/testing/selftests/powerpc/security/flush_utils.h
index 07a5eb301466..c60d15f3eb4b 100644
--- a/tools/testing/selftests/powerpc/security/flush_utils.h
+++ b/tools/testing/selftests/powerpc/security/flush_utils.h
@@ -7,11 +7,18 @@
#ifndef _SELFTESTS_POWERPC_SECURITY_FLUSH_UTILS_H
#define _SELFTESTS_POWERPC_SECURITY_FLUSH_UTILS_H
-#define CACHELINE_SIZE 128
+#define CACHELINE_SIZE 128
+
+#define SPRN_PVR 287
+
+#define PERF_L1D_MISS_P7 0x400f0
+#define PERF_L1D_MISS_P10 0x3e054
void syscall_loop(char *p, unsigned long iterations,
unsigned long zero_size);
void set_dscr(unsigned long val);
+unsigned long get_perf_l1d_miss_event(void);
+
#endif /* _SELFTESTS_POWERPC_SECURITY_FLUSH_UTILS_H */
diff --git a/tools/testing/selftests/powerpc/security/rfi_flush.c
b/tools/testing/selftests/powerpc/security/rfi_flush.c
index 7565fd786640..edf67c91ef79 100644
--- a/tools/testing/selftests/powerpc/security/rfi_flush.c
+++ b/tools/testing/selftests/powerpc/security/rfi_flush.c
@@ -26,6 +26,7 @@ int rfi_flush_test(void)
__u64 l1d_misses_total = 0;
unsigned long iterations = 10, zero_size = 24 * 1024;
unsigned long l1d_misses_expected;
+ unsigned long perf_l1d_miss_event;
int rfi_flush_orig, rfi_flush;
int have_entry_flush, entry_flush_orig;
@@ -54,7 +55,8 @@ int rfi_flush_test(void)
rfi_flush = rfi_flush_orig;
- fd = perf_event_open_counter(PERF_TYPE_RAW, /* L1d miss */ 0x400f0, -1);
+ perf_l1d_miss_event = get_perf_l1d_miss_event();
+ fd = perf_event_open_counter(PERF_TYPE_RAW, perf_l1d_miss_event, -1);
FAIL_IF(fd < 0);
p = (char *)memalign(zero_size, CACHELINE_SIZE);
--
2.30.1
Re: [PATCH] mm/pmem: Avoid inserting hugepage PTE entry with fsdax if hugepage support is disabled
> Differentiate between hardware not supporting hugepages and user disabling THP
> via 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'
>
> For the devdax namespace, the kernel handles the above via the
> supported_alignment attribute and failing to initialize the namespace
> if the namespace align value is not supported on the platform.
>
> For the fsdax namespace, the kernel will continue to initialize
> the namespace. This can result in the kernel creating a huge pte
> entry even though the hardware don't support the same.
>
> We do want hugepage support with pmem even if the end-user disabled THP
> via sysfs file (/sys/kernel/mm/transparent_hugepage/enabled). Hence
> differentiate between hardware/firmware lacking support vs user-controlled
> disable of THP and prevent a huge fault if the hardware lacks hugepage
> support.
>
> Signed-off-by: Aneesh Kumar K.V
> ---
> include/linux/huge_mm.h | 15 +--
> mm/huge_memory.c| 6 +-
> 2 files changed, 14 insertions(+), 7 deletions(-)
>
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index 6a19f35f836b..ba973efcd369 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -78,6 +78,7 @@ static inline vm_fault_t vmf_insert_pfn_pud(struct vm_fault
> *vmf, pfn_t pfn,
> }
>
> enum transparent_hugepage_flag {
> + TRANSPARENT_HUGEPAGE_NEVER_DAX,
> TRANSPARENT_HUGEPAGE_FLAG,
> TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG,
> TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG,
> @@ -123,6 +124,13 @@ extern unsigned long transparent_hugepage_flags;
> */
> static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma)
> {
> +
> + /*
> +* If the hardware/firmware marked hugepage support disabled.
> +*/
> + if (transparent_hugepage_flags & (1 <<
> TRANSPARENT_HUGEPAGE_NEVER_DAX))
> + return false;
> +
> if (vma->vm_flags & VM_NOHUGEPAGE)
> return false;
>
> @@ -134,12 +142,7 @@ static inline bool __transparent_hugepage_enabled(struct
> vm_area_struct *vma)
>
> if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG))
> return true;
> - /*
> -* For dax vmas, try to always use hugepage mappings. If the kernel
> does
> -* not support hugepages, fsdax mappings will fallback to PAGE_SIZE
> -* mappings, and device-dax namespaces, that try to guarantee a given
> -* mapping size, will fail to enable
> -*/
> +
> if (vma_is_dax(vma))
> return true;
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 9237976abe72..d698b7e27447 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -386,7 +386,11 @@ static int __init hugepage_init(void)
> struct kobject *hugepage_kobj;
>
> if (!has_transparent_hugepage()) {
> - transparent_hugepage_flags = 0;
> + /*
> +* Hardware doesn't support hugepages, hence disable
> +* DAX PMD support.
> +*/
> + transparent_hugepage_flags = 1 <<
> TRANSPARENT_HUGEPAGE_NEVER_DAX;
> return -EINVAL;
> }
Reviewed-by: Pankaj Gupta
Re: [PATCH v5 03/10] powerpc/signal64: Move non-inline functions out of setup_sigcontext()
On Sun Feb 7, 2021 at 10:44 PM CST, Daniel Axtens wrote:
> Hi Chris,
>
> These two paragraphs are a little confusing and they seem slightly
> repetitive. But I get the general idea. Two specific comments below:
Umm... yeah only one of those was supposed to be sent. I will reword
this for the next spin and address the comment below about how it is
not entirely clear that the inline functions are being moved out.
>
> > There are non-inline functions which get called in setup_sigcontext() to
> > save register state to the thread struct. Move these functions into a
> > separate prepare_setup_sigcontext() function so that
> > setup_sigcontext() can be refactored later into an "unsafe" version
> > which assumes an open uaccess window. Non-inline functions should be
> > avoided when uaccess is open.
>
> Why do we want to avoid non-inline functions? We came up with:
>
> - we want KUAP protection for as much of the kernel as possible: each
> extra bit of code run with the window open is another piece of attack
> surface.
>
> - non-inline functions default to traceable, which means we could end
> up ftracing while uaccess is enabled. That's a pretty big hole in the
> defences that KUAP provides.
>
> I think we've also had problems with the window being opened or closed
> unexpectedly by various bits of code? So the less code runs in uaccess
> context the less likely that is to occur.
That is my understanding as well.
>
> > The majority of setup_sigcontext() can be refactored to execute in an
> > "unsafe" context (uaccess window is opened) except for some non-inline
> > functions. Move these out into a separate prepare_setup_sigcontext()
> > function which must be called first and before opening up a uaccess
> > window. A follow-up commit converts setup_sigcontext() to be "unsafe".
>
> This was a bit confusing until we realise that you're moving the _calls_
> to the non-inline functions out, not the non-inline functions
> themselves.
>
> > Signed-off-by: Christopher M. Riedl
> > ---
> > arch/powerpc/kernel/signal_64.c | 32 +---
> > 1 file changed, 21 insertions(+), 11 deletions(-)
> >
> > diff --git a/arch/powerpc/kernel/signal_64.c
> > b/arch/powerpc/kernel/signal_64.c
> > index f9e4a1ac440f..b211a8ea4f6e 100644
> > --- a/arch/powerpc/kernel/signal_64.c
> > +++ b/arch/powerpc/kernel/signal_64.c
> > @@ -79,6 +79,24 @@ static elf_vrreg_t __user *sigcontext_vmx_regs(struct
> > sigcontext __user *sc)
> > }
> > #endif
> >
> > +static void prepare_setup_sigcontext(struct task_struct *tsk, int
> > ctx_has_vsx_region)
>
> ctx_has_vsx_region should probably be a bool? Although setup_sigcontext
> also has it as an int so I guess that's arguable, and maybe it's better
> to stick with this for constency.
I've been told not to introduce unrelated changes in my patches before
so chose to keep this as an int for consistency.
>
> > +{
> > +#ifdef CONFIG_ALTIVEC
> > + /* save altivec registers */
> > + if (tsk->thread.used_vr)
> > + flush_altivec_to_thread(tsk);
> > + if (cpu_has_feature(CPU_FTR_ALTIVEC))
> > + tsk->thread.vrsave = mfspr(SPRN_VRSAVE);
> > +#endif /* CONFIG_ALTIVEC */
> > +
> > + flush_fp_to_thread(tsk);
> > +
> > +#ifdef CONFIG_VSX
> > + if (tsk->thread.used_vsr && ctx_has_vsx_region)
> > + flush_vsx_to_thread(tsk);
> > +#endif /* CONFIG_VSX */
>
> Alternatively, given that this is the only use of ctx_has_vsx_region,
> mpe suggested that perhaps we could drop it entirely and always
> flush_vsx if used_vsr. The function is only ever called with either
> `current` or wth ctx_has_vsx_region set to 1, so in either case I think
> that's safe? I'm not sure if it would have performance implications.
I think that could work as long as we can guarantee that the context
passed to swapcontext will always be sufficiently sized if used_vsr,
which I think *has* to be the case?
>
> Should we move this and the altivec ifdef to IS_ENABLED(CONFIG_VSX) etc?
> I'm not sure if that runs into any problems with things like 'used_vsr'
> only being defined if CONFIG_VSX is set, but I thought I'd ask.
That's why I didn't use IS_ENABLED(CONFIG_...) here - all of these
field (used_vr, vrsave, used_vsr) declarations are guarded by #ifdefs :/
>
>
> > +}
> > +
> > /*
> > * Set up the sigcontext for the signal frame.
> > */
> > @@ -97,7 +115,6 @@ static long setup_sigcontext(struct sigcontext __user
> > *sc,
> > */
> > #ifdef CONFIG_ALTIVEC
> > elf_vrreg_t __user *v_regs = sigcontext_vmx_regs(sc);
> > - unsigned long vrsave;
> > #endif
> > struct pt_regs *regs = tsk->thread.regs;
> > unsigned long msr = regs->msr;
> > @@ -112,7 +129,6 @@ static long setup_sigcontext(struct sigcontext __user
> > *sc,
> >
> > /* save altivec registers */
> > if (tsk->thread.used_vr) {
> > - flush_altivec_to_thread(tsk);
> > /* Copy 33 vec registers (vr0..31 and vscr) to the stack */
> > err |=
Re: [PATCH v5 10/10] powerpc/signal64: Use __get_user() to copy sigset_t
On Tue Feb 9, 2021 at 3:45 PM CST, Christophe Leroy wrote:
> "Christopher M. Riedl" a écrit :
>
> > Usually sigset_t is exactly 8B which is a "trivial" size and does not
> > warrant using __copy_from_user(). Use __get_user() directly in
> > anticipation of future work to remove the trivial size optimizations
> > from __copy_from_user(). Calling __get_user() also results in a small
> > boost to signal handling throughput here.
> >
> > Signed-off-by: Christopher M. Riedl
> > ---
> > arch/powerpc/kernel/signal_64.c | 14 --
> > 1 file changed, 12 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/powerpc/kernel/signal_64.c
> > b/arch/powerpc/kernel/signal_64.c
> > index 817b64e1e409..42fdc4a7ff72 100644
> > --- a/arch/powerpc/kernel/signal_64.c
> > +++ b/arch/powerpc/kernel/signal_64.c
> > @@ -97,6 +97,14 @@ static void prepare_setup_sigcontext(struct
> > task_struct *tsk, int ctx_has_vsx_re
> > #endif /* CONFIG_VSX */
> > }
> >
> > +static inline int get_user_sigset(sigset_t *dst, const sigset_t *src)
>
> Should be called __get_user_sigset() as it is a helper for __get_user()
Ok makes sense.
>
> > +{
> > + if (sizeof(sigset_t) <= 8)
>
> We should always use __get_user(), see below.
>
> > + return __get_user(dst->sig[0], >sig[0]);
>
> I think the above will not work on ppc32, it will only copy 4 bytes.
> You must cast the source to u64*
Well this is signal_64.c :) Looks like ppc32 needs the same thing so
I'll just move this into signal.h and use it for both.
The only exception would be the COMPAT case in signal_32.c which ends up
calling the common get_compat_sigset(). Updating that is probably
outside the scope of this series.
>
> > + else
> > + return __copy_from_user(dst, src, sizeof(sigset_t));
>
> I see no point in keeping this alternative. Today sigset_ t is fixed.
> If you fear one day someone might change it to something different
> than a u64, just add a BUILD_BUG_ON(sizeof(sigset_t) != sizeof(u64));
Ah yes that is much better - thanks for the suggestion.
>
> > +}
> > +
> > /*
> > * Set up the sigcontext for the signal frame.
> > */
> > @@ -701,8 +709,9 @@ SYSCALL_DEFINE3(swapcontext, struct ucontext
> > __user *, old_ctx,
> > * We kill the task with a SIGSEGV in this situation.
> > */
> >
> > - if (__copy_from_user(, _ctx->uc_sigmask, sizeof(set)))
> > + if (get_user_sigset(, _ctx->uc_sigmask))
> > do_exit(SIGSEGV);
> > +
>
> This white space is not part of the change, keep patches to the
> minimum, avoid cosmetic
Just a (bad?) habit on my part that I missed - I'll remove this one and
the one further below.
>
> > set_current_blocked();
> >
> > if (!user_read_access_begin(new_ctx, ctx_size))
> > @@ -740,8 +749,9 @@ SYSCALL_DEFINE0(rt_sigreturn)
> > if (!access_ok(uc, sizeof(*uc)))
> > goto badframe;
> >
> > - if (__copy_from_user(, >uc_sigmask, sizeof(set)))
> > + if (get_user_sigset(, >uc_sigmask))
> > goto badframe;
> > +
>
> Same
>
> > set_current_blocked();
> >
> > #ifdef CONFIG_PPC_TRANSACTIONAL_MEM
> > --
> > 2.26.1
Re: [PATCH 2/4] KVM: PPC: Book3S HV: Fix radix guest SLB side channel
Excerpts from Paul Mackerras's message of February 10, 2021 11:28 am: > On Mon, Jan 18, 2021 at 04:28:07PM +1000, Nicholas Piggin wrote: >> The slbmte instruction is legal in radix mode, including radix guest >> mode. This means radix guests can load the SLB with arbitrary data. >> >> KVM host does not clear the SLB when exiting a guest if it was a >> radix guest, which would allow a rogue radix guest to use the SLB as >> a side channel to communicate with other guests. > > No, because the code currently clears the SLB when entering a radix > guest, Not AFAIKS. > which you remove in the next patch. The next patch avoids clearing host SLB entries when a hash guest is entered from a radix host, it doesn't apply to radix guests. Not sure where the changelog for it went but it should have "HPT guests" in the title at least, I guess. > I'm OK with moving the SLB > clearing from guest entry to guest exit, I guess, but I don't see that > you are in fact fixing anything by doing so. I can set slb entries in a radix guest in simulator and observe they stay around over host<->guest transitions, and they don't after this patch. Thanks, Nick
[powerpc:next-test 68/159] arch/powerpc/platforms/amigaone/setup.c:73:13: error: no previous prototype for 'amigaone_discover_phbs'
tree: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git
next-test
head: 5811244192fc4e18c001c69300044c2acf30bd91
commit: 053d58c870298d62b9c5154672ef2f1684c4ea43 [68/159] powerpc/amigaone:
Move PHB discovery
config: powerpc64-randconfig-r036-20210209 (attached as .config)
compiler: powerpc-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O
~/bin/make.cross
chmod +x ~/bin/make.cross
#
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=053d58c870298d62b9c5154672ef2f1684c4ea43
git remote add powerpc
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git
git fetch --no-tags powerpc next-test
git checkout 053d58c870298d62b9c5154672ef2f1684c4ea43
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross
ARCH=powerpc64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot
All errors (new ones prefixed by >>):
arch/powerpc/platforms/amigaone/setup.c:27:6: error: no previous prototype
for 'amigaone_show_cpuinfo' [-Werror=missing-prototypes]
27 | void amigaone_show_cpuinfo(struct seq_file *m)
| ^
arch/powerpc/platforms/amigaone/setup.c:67:13: error: no previous prototype
for 'amigaone_setup_arch' [-Werror=missing-prototypes]
67 | void __init amigaone_setup_arch(void)
| ^~~
>> arch/powerpc/platforms/amigaone/setup.c:73:13: error: no previous prototype
>> for 'amigaone_discover_phbs' [-Werror=missing-prototypes]
73 | void __init amigaone_discover_phbs(void)
| ^~
arch/powerpc/platforms/amigaone/setup.c:85:13: error: no previous prototype
for 'amigaone_init_IRQ' [-Werror=missing-prototypes]
85 | void __init amigaone_init_IRQ(void)
| ^
arch/powerpc/platforms/amigaone/setup.c:125:17: error: no previous prototype
for 'amigaone_restart' [-Werror=missing-prototypes]
125 | void __noreturn amigaone_restart(char *cmd)
| ^~~~
cc1: all warnings being treated as errors
vim +/amigaone_discover_phbs +73 arch/powerpc/platforms/amigaone/setup.c
72
> 73 void __init amigaone_discover_phbs(void)
74 {
75 struct device_node *np;
76 int phb = -ENODEV;
77
78 /* Lookup PCI host bridges. */
79 for_each_compatible_node(np, "pci", "mai-logic,articia-s")
80 phb = amigaone_add_bridge(np);
81
82 BUG_ON(phb != 0);
83 }
84
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-...@lists.01.org
.config.gz
Description: application/gzip
Re: [PATCH v5 20/22] powerpc/syscall: Avoid storing 'current' in another pointer
Excerpts from Christophe Leroy's message of February 10, 2021 3:03 am:
>
>
> Le 09/02/2021 à 15:31, David Laight a écrit :
>> From: Segher Boessenkool
>>> Sent: 09 February 2021 13:51
>>>
>>> On Tue, Feb 09, 2021 at 12:36:20PM +1000, Nicholas Piggin wrote:
What if you did this?
>>>
+static inline struct task_struct *get_current(void)
+{
+ register struct task_struct *task asm ("r2");
+
+ return task;
+}
>>>
>>> Local register asm variables are *only* guaranteed to live in that
>>> register as operands to an asm. See
>>>
>>> https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables
>>> ("The only supported use" etc.)
>>>
>>> You can do something like
>>>
>>> static inline struct task_struct *get_current(void)
>>> {
>>> register struct task_struct *task asm ("r2");
>>>
>>> asm("" : "+r"(task));
>>>
>>> return task;
>>> }
>>>
>>> which makes sure that "task" actually is in r2 at the point of that asm.
>>
>> If "r2" always contains current (and is never assigned by the compiler)
>> why not use a global register variable for it?
>>
>
>
> The change proposed by Nick doesn't solve the issue.
It seemed to change code generation in a simple test case, oh well.
>
> The problem is that at the begining of the function we have:
>
> unsigned long *ti_flagsp = _thread_info()->flags;
>
> When the function uses ti_flagsp for the first time, it does use 112(r2)
>
> Then the function calls some other functions.
>
> Most likely because the function could update 'current', GCC copies r2 into
> r30, so that if r2 get
> changed by the called function, ti_flagsp is still based on the previous
> value of current.
>
> Allthough we know r2 wont change, GCC doesn't know it. And in order to save
> r2 into r30, it needs to
> save r30 in the stack.
>
>
> By using _thread_info()->flags directly instead of this intermediaite
> ti_flagsp pointer, GCC
> uses r2 instead instead of doing a copy.
>
>
> Nick, I don't understand the reason why you need that 'ti_flagsp' local var.
Just to save typing, I don't mind your patch I was just wondering if
current could be improved in general.
Thanks,
Nick
Re: [PATCH v5 18/22] powerpc/syscall: Remove FULL_REGS verification in system_call_exception
Excerpts from Christophe Leroy's message of February 10, 2021 12:31 am: > > > Le 09/02/2021 à 03:02, Nicholas Piggin a écrit : >> Excerpts from Christophe Leroy's message of February 9, 2021 1:10 am: >>> For book3s/64, FULL_REGS() is 'true' at all time, so the test voids. >>> For others, non volatile registers are saved inconditionally. >>> >>> So the verification is pointless. >>> >>> Should one fail to do it, it would anyway be caught by the >>> CHECK_FULL_REGS() in copy_thread() as we have removed the >>> special versions ppc_fork() and friends. >>> >>> null_syscall benchmark reduction 4 cycles (332 => 328 cycles) >> >> I wonder if we rather make a CONFIG option for a bunch of these simpler >> debug checks here (and also in interrupt exit, wrappers, etc) rather >> than remove them entirely. > > We can drop this patch if you prefer. Anyway, like book3s/64, once ppc32 also > do interrupt > entry/exit in C, FULL_REGS() will already return true. Sure let's do that. Thanks, Nick
Re: [PATCH v5 16/22] powerpc/syscall: Avoid stack frame in likely part of system_call_exception()
Excerpts from Christophe Leroy's message of February 10, 2021 2:13 am: > > > Le 09/02/2021 à 02:55, Nicholas Piggin a écrit : >> Excerpts from Christophe Leroy's message of February 9, 2021 1:10 am: >>> When r3 is not modified, reload it from regs->orig_r3 to free >>> volatile registers. This avoids a stack frame for the likely part >>> of system_call_exception() >> >> This doesn't on my 64s build, but it does reduce one non volatile >> register save/restore. With quite a bit more register pressure >> reduction 64s can avoid the stack frame as well. > > The stack frame is not due to the registers because on PPC64 you have the > redzone that you don't > have on PPC32. > > As far as I can see, this is due to a call to .arch_local_irq_restore(). > > On ppc32 arch_local_irq_restore() is just a write to MSR. Oh you're right there. We can actually inline fast paths of that I have a patch somewhere, but not sure if it's worthwhile. >> It's a cool trick but quite code and compiler specific so I don't know >> how worthwhile it is to keep considering we're calling out into random >> kernel C code after this. >> >> Maybe just keep it PPC32 specific for the moment, will have to do more >> tuning for 64 and we have other stuff to do there first. >> >> If you are happy to make it 32-bit only then > > I think we can leave without this, that's only one or two cycles won. Okay for this round let's drop it for now. Thanks, Nick
Re: [PATCH 2/4] KVM: PPC: Book3S HV: Fix radix guest SLB side channel
On Mon, Jan 18, 2021 at 04:28:07PM +1000, Nicholas Piggin wrote: > The slbmte instruction is legal in radix mode, including radix guest > mode. This means radix guests can load the SLB with arbitrary data. > > KVM host does not clear the SLB when exiting a guest if it was a > radix guest, which would allow a rogue radix guest to use the SLB as > a side channel to communicate with other guests. No, because the code currently clears the SLB when entering a radix guest, which you remove in the next patch. I'm OK with moving the SLB clearing from guest entry to guest exit, I guess, but I don't see that you are in fact fixing anything by doing so. Paul.
Re: [PATCH 1/3] spi: mpc52xx: Avoid using get_tbl()
On Tue, Feb 09, 2021 at 10:26:21AM +, Christophe Leroy wrote: > get_tbl() is confusing as it returns the content TBL register > on PPC32 but the concatenation of TBL and TBU on PPC64. > > Use mftb() instead. > > This will allow the removal of get_tbl() in a following patch. Acked-by: Mark Brown signature.asc Description: PGP signature
[powerpc:next-test 67/159] arch/powerpc/platforms/83xx/km83xx.c:183:19: error: 'mpc83xx_setup_pci' undeclared here (not in a function); did you mean
tree: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git
next-test
head: 5811244192fc4e18c001c69300044c2acf30bd91
commit: 83f84041ff1cf6c23fc38861218af2d4ca2d9b38 [67/159] powerpc/83xx: Move
PHB discovery
config: powerpc-kmeter1_defconfig (attached as .config)
compiler: powerpc-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O
~/bin/make.cross
chmod +x ~/bin/make.cross
#
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=83f84041ff1cf6c23fc38861218af2d4ca2d9b38
git remote add powerpc
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git
git fetch --no-tags powerpc next-test
git checkout 83f84041ff1cf6c23fc38861218af2d4ca2d9b38
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross
ARCH=powerpc
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot
All errors (new ones prefixed by >>):
>> arch/powerpc/platforms/83xx/km83xx.c:183:19: error: 'mpc83xx_setup_pci'
>> undeclared here (not in a function); did you mean 'mpc83xx_setup_arch'?
183 | .discover_phbs = mpc83xx_setup_pci,
| ^
| mpc83xx_setup_arch
vim +183 arch/powerpc/platforms/83xx/km83xx.c
178
179 define_machine(mpc83xx_km) {
180 .name = "mpc83xx-km-platform",
181 .probe = mpc83xx_km_probe,
182 .setup_arch = mpc83xx_km_setup_arch,
> 183 .discover_phbs = mpc83xx_setup_pci,
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-...@lists.01.org
.config.gz
Description: application/gzip
Re: [PATCH v5 10/10] powerpc/signal64: Use __get_user() to copy sigset_t
"Christopher M. Riedl" a écrit :
Usually sigset_t is exactly 8B which is a "trivial" size and does not
warrant using __copy_from_user(). Use __get_user() directly in
anticipation of future work to remove the trivial size optimizations
from __copy_from_user(). Calling __get_user() also results in a small
boost to signal handling throughput here.
Signed-off-by: Christopher M. Riedl
---
arch/powerpc/kernel/signal_64.c | 14 --
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/signal_64.c
b/arch/powerpc/kernel/signal_64.c
index 817b64e1e409..42fdc4a7ff72 100644
--- a/arch/powerpc/kernel/signal_64.c
+++ b/arch/powerpc/kernel/signal_64.c
@@ -97,6 +97,14 @@ static void prepare_setup_sigcontext(struct
task_struct *tsk, int ctx_has_vsx_re
#endif /* CONFIG_VSX */
}
+static inline int get_user_sigset(sigset_t *dst, const sigset_t *src)
Should be called __get_user_sigset() as it is a helper for __get_user()
+{
+ if (sizeof(sigset_t) <= 8)
We should always use __get_user(), see below.
+ return __get_user(dst->sig[0], >sig[0]);
I think the above will not work on ppc32, it will only copy 4 bytes.
You must cast the source to u64*
+ else
+ return __copy_from_user(dst, src, sizeof(sigset_t));
I see no point in keeping this alternative. Today sigset_ t is fixed.
If you fear one day someone might change it to something different
than a u64, just add a BUILD_BUG_ON(sizeof(sigset_t) != sizeof(u64));
+}
+
/*
* Set up the sigcontext for the signal frame.
*/
@@ -701,8 +709,9 @@ SYSCALL_DEFINE3(swapcontext, struct ucontext
__user *, old_ctx,
* We kill the task with a SIGSEGV in this situation.
*/
- if (__copy_from_user(, _ctx->uc_sigmask, sizeof(set)))
+ if (get_user_sigset(, _ctx->uc_sigmask))
do_exit(SIGSEGV);
+
This white space is not part of the change, keep patches to the
minimum, avoid cosmetic
set_current_blocked();
if (!user_read_access_begin(new_ctx, ctx_size))
@@ -740,8 +749,9 @@ SYSCALL_DEFINE0(rt_sigreturn)
if (!access_ok(uc, sizeof(*uc)))
goto badframe;
- if (__copy_from_user(, >uc_sigmask, sizeof(set)))
+ if (get_user_sigset(, >uc_sigmask))
goto badframe;
+
Same
set_current_blocked();
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
--
2.26.1
[powerpc:next-test 129/159] arch/powerpc/mm/book3s64/radix_tlb.c:646:6: warning: no previous prototype for function 'exit_lazy_flush_tlb'
tree: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git
next-test
head: 5811244192fc4e18c001c69300044c2acf30bd91
commit: 032b7f08932c9b212952d6d585e45b2941b3e8be [129/159] powerpc/64s/radix:
serialize_against_pte_lookup IPIs trim mm_cpumask
config: powerpc-randconfig-r026-20210209 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project
c9439ca36342fb6013187d0a69aef92736951476)
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O
~/bin/make.cross
chmod +x ~/bin/make.cross
# install powerpc cross compiling tool for clang build
# apt-get install binutils-powerpc-linux-gnu
#
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=032b7f08932c9b212952d6d585e45b2941b3e8be
git remote add powerpc
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git
git fetch --no-tags powerpc next-test
git checkout 032b7f08932c9b212952d6d585e45b2941b3e8be
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=powerpc
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot
All warnings (new ones prefixed by >>):
>> arch/powerpc/mm/book3s64/radix_tlb.c:646:6: warning: no previous prototype
>> for function 'exit_lazy_flush_tlb' [-Wmissing-prototypes]
void exit_lazy_flush_tlb(struct mm_struct *mm, bool always_flush)
^
arch/powerpc/mm/book3s64/radix_tlb.c:646:1: note: declare 'static' if the
function is not intended to be used outside of this translation unit
void exit_lazy_flush_tlb(struct mm_struct *mm, bool always_flush)
^
static
1 warning generated.
vim +/exit_lazy_flush_tlb +646 arch/powerpc/mm/book3s64/radix_tlb.c
641
642 /*
643 * If always_flush is true, then flush even if this CPU can't be removed
644 * from mm_cpumask.
645 */
> 646 void exit_lazy_flush_tlb(struct mm_struct *mm, bool always_flush)
647 {
648 unsigned long pid = mm->context.id;
649 int cpu = smp_processor_id();
650
651 /*
652 * A kthread could have done a mmget_not_zero() after the
flushing CPU
653 * checked mm_cpumask, and be in the process of kthread_use_mm
when
654 * interrupted here. In that case, current->mm will be set to
mm,
655 * because kthread_use_mm() setting ->mm and switching to the
mm is
656 * done with interrupts off.
657 */
658 if (current->mm == mm)
659 goto out;
660
661 if (current->active_mm == mm) {
662 WARN_ON_ONCE(current->mm != NULL);
663 /* Is a kernel thread and is using mm as the lazy tlb */
664 mmgrab(_mm);
665 current->active_mm = _mm;
666 switch_mm_irqs_off(mm, _mm, current);
667 mmdrop(mm);
668 }
669
670 /*
671 * This IPI may be initiated from any source including those not
672 * running the mm, so there may be a racing IPI that comes after
673 * this one which finds the cpumask already clear. Check and
avoid
674 * underflowing the active_cpus count in that case. The race
should
675 * not otherwise be a problem, but the TLB must be flushed
because
676 * that's what the caller expects.
677 */
678 if (cpumask_test_cpu(cpu, mm_cpumask(mm))) {
679 atomic_dec(>context.active_cpus);
680 cpumask_clear_cpu(cpu, mm_cpumask(mm));
681 always_flush = true;
682 }
683
684 out:
685 if (always_flush)
686 _tlbiel_pid(pid, RIC_FLUSH_ALL);
687 }
688
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-...@lists.01.org
.config.gz
Description: application/gzip
[PATCH v6 1/2] powerpc/syscall: Do not check unsupported scv vector on PPC32
Only book3s/64 has scv. No need to check the 0x7ff0 trap on 32 or 64e.
For that, add a helper trap_is_unsupported_scv() similar to
trap_is_scv().
And ignore the scv parameter in syscall_exit_prepare (Save 14 cycles
346 => 332 cycles)
Signed-off-by: Christophe Leroy
---
v5: Added a helper trap_is_unsupported_scv()
v6: Still set r5 when calling syscall_exit_prepare() and introduce a local bool
'is_not_scv'
---
arch/powerpc/include/asm/ptrace.h | 5 +
arch/powerpc/kernel/interrupt.c | 9 +
2 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/ptrace.h
b/arch/powerpc/include/asm/ptrace.h
index 8236c5e749e4..975ba260006a 100644
--- a/arch/powerpc/include/asm/ptrace.h
+++ b/arch/powerpc/include/asm/ptrace.h
@@ -232,6 +232,11 @@ static inline bool trap_is_scv(struct pt_regs *regs)
return (IS_ENABLED(CONFIG_PPC_BOOK3S_64) && TRAP(regs) == 0x3000);
}
+static inline bool trap_is_unsupported_scv(struct pt_regs *regs)
+{
+ return IS_ENABLED(CONFIG_PPC_BOOK3S_64) && TRAP(regs) == 0x7ff0;
+}
+
static inline bool trap_is_syscall(struct pt_regs *regs)
{
return (trap_is_scv(regs) || TRAP(regs) == 0xc00);
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index 107ec39f05cb..75d657b63332 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -88,7 +88,7 @@ notrace long system_call_exception(long r3, long r4, long r5,
local_irq_enable();
if (unlikely(current_thread_info()->flags & _TIF_SYSCALL_DOTRACE)) {
- if (unlikely(regs->trap == 0x7ff0)) {
+ if (unlikely(trap_is_unsupported_scv(regs))) {
/* Unsupported scv vector */
_exception(SIGILL, regs, ILL_ILLOPC, regs->nip);
return regs->gpr[3];
@@ -111,7 +111,7 @@ notrace long system_call_exception(long r3, long r4, long
r5,
r8 = regs->gpr[8];
} else if (unlikely(r0 >= NR_syscalls)) {
- if (unlikely(regs->trap == 0x7ff0)) {
+ if (unlikely(trap_is_unsupported_scv(regs))) {
/* Unsupported scv vector */
_exception(SIGILL, regs, ILL_ILLOPC, regs->nip);
return regs->gpr[3];
@@ -220,6 +220,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
unsigned long *ti_flagsp = _thread_info()->flags;
unsigned long ti_flags;
unsigned long ret = 0;
+ bool is_not_scv = !IS_ENABLED(CONFIG_PPC_BOOK3S_64) || !scv;
CT_WARN_ON(ct_state() == CONTEXT_USER);
@@ -234,7 +235,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
ti_flags = *ti_flagsp;
- if (unlikely(r3 >= (unsigned long)-MAX_ERRNO) && !scv) {
+ if (unlikely(r3 >= (unsigned long)-MAX_ERRNO) && is_not_scv) {
if (likely(!(ti_flags & (_TIF_NOERROR | _TIF_RESTOREALL {
r3 = -r3;
regs->ccr |= 0x1000; /* Set SO bit in CR */
@@ -305,7 +306,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
user_enter_irqoff();
/* scv need not set RI=0 because SRRs are not used */
- if (unlikely(!__prep_irq_for_enabled_exit(!scv))) {
+ if (unlikely(!__prep_irq_for_enabled_exit(is_not_scv))) {
user_exit_irqoff();
local_irq_enable();
local_irq_disable();
--
2.25.0
[PATCH v6 2/2] powerpc/32: Handle bookE debugging in C in syscall entry/exit
The handling of SPRN_DBCR0 and other registers can easily
be done in C instead of ASM.
For that, create booke_load_dbcr0() and booke_restore_dbcr0().
Signed-off-by: Christophe Leroy
---
v5: New
v6: Refactor into helpers
---
arch/powerpc/include/asm/interrupt.h | 12
arch/powerpc/include/asm/reg_booke.h | 3 ++
arch/powerpc/kernel/entry_32.S | 7 -
arch/powerpc/kernel/head_32.h| 15 --
arch/powerpc/kernel/head_booke.h | 19 -
arch/powerpc/kernel/interrupt.c | 41 ++--
6 files changed, 42 insertions(+), 55 deletions(-)
diff --git a/arch/powerpc/include/asm/interrupt.h
b/arch/powerpc/include/asm/interrupt.h
index 4badb3e51c19..e62c37915bbe 100644
--- a/arch/powerpc/include/asm/interrupt.h
+++ b/arch/powerpc/include/asm/interrupt.h
@@ -14,6 +14,18 @@ struct interrupt_state {
#endif
};
+static inline void booke_restore_dbcr0(void)
+{
+#ifdef CONFIG_PPC_ADV_DEBUG_REGS
+ unsigned long dbcr0 = current->thread.debug.dbcr0;
+
+ if (IS_ENABLED(CONFIG_PPC32) && unlikely(dbcr0 & DBCR0_IDM)) {
+ mtspr(SPRN_DBSR, -1);
+ mtspr(SPRN_DBCR0, global_dbcr0[smp_processor_id()]);
+ }
+#endif
+}
+
static inline void interrupt_enter_prepare(struct pt_regs *regs, struct
interrupt_state *state)
{
/*
diff --git a/arch/powerpc/include/asm/reg_booke.h
b/arch/powerpc/include/asm/reg_booke.h
index 262782f08fd4..17b8dcd9a40d 100644
--- a/arch/powerpc/include/asm/reg_booke.h
+++ b/arch/powerpc/include/asm/reg_booke.h
@@ -691,6 +691,9 @@
#define mttmr(rn, v) asm volatile(MTTMR(rn, %0) : \
: "r" ((unsigned long)(v)) \
: "memory")
+
+extern unsigned long global_dbcr0[];
+
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_POWERPC_REG_BOOKE_H__ */
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 9dd90be9f8a5..78c430b7f9d9 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -343,13 +343,6 @@ ret_from_syscall:
addir4,r1,STACK_FRAME_OVERHEAD
li r5,0
bl syscall_exit_prepare
-#if defined(CONFIG_4xx) || defined(CONFIG_BOOKE)
- /* If the process has its own DBCR0 value, load it up. The internal
- debug mode bit tells us that dbcr0 should be loaded. */
- lwz r0,THREAD+THREAD_DBCR0(r2)
- andis. r10,r0,DBCR0_IDM@h
- bnel- load_dbcr0
-#endif
#ifdef CONFIG_PPC_47x
lis r4,icache_44x_need_flush@ha
lwz r5,icache_44x_need_flush@l(r4)
diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h
index 1afad7bc3395..5d4706c14572 100644
--- a/arch/powerpc/kernel/head_32.h
+++ b/arch/powerpc/kernel/head_32.h
@@ -153,21 +153,6 @@
SAVE_4GPRS(3, r11)
SAVE_2GPRS(7, r11)
addir2,r12,-THREAD
-#if defined(CONFIG_40x)
- /* Check to see if the dbcr0 register is set up to debug. Use the
- internal debug mode bit to do this. */
- lwz r12,THREAD_DBCR0(r12)
- andis. r12,r12,DBCR0_IDM@h
- beq+3f
- /* From user and task is ptraced - load up global dbcr0 */
- li r12,-1 /* clear all pending debug events */
- mtspr SPRN_DBSR,r12
- lis r11,global_dbcr0@ha
- addir11,r11,global_dbcr0@l
- lwz r12,0(r11)
- mtspr SPRN_DBCR0,r12
-3:
-#endif
b transfer_to_syscall /* jump to handler */
.endm
diff --git a/arch/powerpc/kernel/head_booke.h b/arch/powerpc/kernel/head_booke.h
index 5f565232b99d..47857795f50a 100644
--- a/arch/powerpc/kernel/head_booke.h
+++ b/arch/powerpc/kernel/head_booke.h
@@ -130,25 +130,6 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV)
SAVE_2GPRS(7, r11)
addir2,r10,-THREAD
- /* Check to see if the dbcr0 register is set up to debug. Use the
- internal debug mode bit to do this. */
- lwz r12,THREAD_DBCR0(r10)
- andis. r12,r12,DBCR0_IDM@h
- beq+3f
- /* From user and task is ptraced - load up global dbcr0 */
- li r12,-1 /* clear all pending debug events */
- mtspr SPRN_DBSR,r12
- lis r11,global_dbcr0@ha
- addir11,r11,global_dbcr0@l
-#ifdef CONFIG_SMP
- lwz r10, TASK_CPU(r2)
- slwir10, r10, 2
- add r11, r11, r10
-#endif
- lwz r12,0(r11)
- mtspr SPRN_DBCR0,r12
-
-3:
b transfer_to_syscall /* jump to handler */
.endm
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index 75d657b63332..f93664ad4a5e 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -73,6 +73,8 @@ notrace long system_call_exception(long r3, long r4, long r5,
kuap_check_amr();
#endif
+ booke_restore_dbcr0();
+
account_cpu_user_entry();
[PATCH v6 0/2] powerpc/32: Implement C syscall entry/exit (complement)
This series implements C syscall entry/exit for PPC32. It reuses the work already done for PPC64. This series is based on today's next-test (f538b53fd47a) where main patchs from v5 are merged in. The first patch is important for performance. v6: - Forced scv param to 0 on syscall_exit_prepare(), and added is_not_scv bool. - Reworked the last patch to avoid code duplication. Christophe Leroy (2): powerpc/syscall: Do not check unsupported scv vector on PPC32 powerpc/32: Handle bookE debugging in C in syscall entry/exit arch/powerpc/include/asm/interrupt.h | 12 +++ arch/powerpc/include/asm/ptrace.h| 5 +++ arch/powerpc/include/asm/reg_booke.h | 3 ++ arch/powerpc/kernel/entry_32.S | 7 arch/powerpc/kernel/head_32.h| 15 - arch/powerpc/kernel/head_booke.h | 19 --- arch/powerpc/kernel/interrupt.c | 50 ++-- 7 files changed, 52 insertions(+), 59 deletions(-) -- 2.25.0
[PATCH v17 10/10] arm64: Enable passing IMA log to next kernel on kexec
Update CONFIG_KEXEC_FILE to select CONFIG_HAVE_IMA_KEXEC, if CONFIG_IMA is enabled, to indicate that the IMA measurement log information is present in the device tree for ARM64. Co-developed-by: Prakhar Srivastava Signed-off-by: Prakhar Srivastava Signed-off-by: Lakshmi Ramasubramanian Suggested-by: Thiago Jung Bauermann --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 05e17351e4f3..8a93573cebb6 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1093,6 +1093,7 @@ config KEXEC config KEXEC_FILE bool "kexec file based system call" select KEXEC_CORE + select HAVE_IMA_KEXEC if IMA help This is new version of kexec system call. This system call is file based and takes file descriptors as system call argument -- 2.30.0
[PATCH v17 09/10] powerpc: Delete unused function delete_fdt_mem_rsv()
delete_fdt_mem_rsv() defined in "arch/powerpc/kexec/file_load.c"
has been renamed to fdt_find_and_del_mem_rsv(), and moved to
"drivers/of/kexec.c".
Remove delete_fdt_mem_rsv() in "arch/powerpc/kexec/file_load.c".
Co-developed-by: Prakhar Srivastava
Signed-off-by: Prakhar Srivastava
Signed-off-by: Lakshmi Ramasubramanian
---
arch/powerpc/include/asm/kexec.h | 1 -
arch/powerpc/kexec/file_load.c | 32
2 files changed, 33 deletions(-)
diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
index 2b87993f6e66..e543593da1e1 100644
--- a/arch/powerpc/include/asm/kexec.h
+++ b/arch/powerpc/include/asm/kexec.h
@@ -119,7 +119,6 @@ char *setup_kdump_cmdline(struct kimage *image, char
*cmdline,
int setup_purgatory(struct kimage *image, const void *slave_code,
const void *fdt, unsigned long kernel_load_addr,
unsigned long fdt_load_addr);
-int delete_fdt_mem_rsv(void *fdt, unsigned long start, unsigned long size);
#ifdef CONFIG_PPC64
struct kexec_buf;
diff --git a/arch/powerpc/kexec/file_load.c b/arch/powerpc/kexec/file_load.c
index bd8b956aafc3..6f75a45f14c5 100644
--- a/arch/powerpc/kexec/file_load.c
+++ b/arch/powerpc/kexec/file_load.c
@@ -107,35 +107,3 @@ int setup_purgatory(struct kimage *image, const void
*slave_code,
return 0;
}
-
-/**
- * delete_fdt_mem_rsv - delete memory reservation with given address and size
- *
- * Return: 0 on success, or negative errno on error.
- */
-int delete_fdt_mem_rsv(void *fdt, unsigned long start, unsigned long size)
-{
- int i, ret, num_rsvs = fdt_num_mem_rsv(fdt);
-
- for (i = 0; i < num_rsvs; i++) {
- uint64_t rsv_start, rsv_size;
-
- ret = fdt_get_mem_rsv(fdt, i, _start, _size);
- if (ret) {
- pr_err("Malformed device tree.\n");
- return -EINVAL;
- }
-
- if (rsv_start == start && rsv_size == size) {
- ret = fdt_del_mem_rsv(fdt, i);
- if (ret) {
- pr_err("Error deleting device tree
reservation.\n");
- return -EINVAL;
- }
-
- return 0;
- }
- }
-
- return -ENOENT;
-}
--
2.30.0
[PATCH v17 08/10] kexec: Use fdt_appendprop_addrrange() to add ima buffer to FDT
fdt_appendprop_addrrange() function adds a property, with the given name,
to the device tree at the given node offset, and also sets the address
and size of the property. This function should be used to add
"linux,ima-kexec-buffer" property to the device tree and set the address
and size of the IMA measurement buffer, instead of using custom function.
Use fdt_appendprop_addrrange() to add "linux,ima-kexec-buffer" property
to the device tree. This property holds the address and size of
the IMA measurement buffer that needs to be passed from the current
kernel to the next kernel across kexec system call.
Remove custom code that is used in setup_ima_buffer() to add
"linux,ima-kexec-buffer" property to the device tree.
Co-developed-by: Prakhar Srivastava
Signed-off-by: Prakhar Srivastava
Signed-off-by: Lakshmi Ramasubramanian
Reviewed-by: Thiago Jung Bauermann
---
drivers/of/kexec.c | 57 --
1 file changed, 5 insertions(+), 52 deletions(-)
diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c
index c601b5af4a88..c53746b9b168 100644
--- a/drivers/of/kexec.c
+++ b/drivers/of/kexec.c
@@ -232,36 +232,6 @@ int of_ima_add_kexec_buffer(struct kimage *image,
return 0;
}
-/**
- * write_number - Convert number to big-endian format
- *
- * @p: Buffer to write the number to
- * @value: Number to convert
- * @cells: Number of cells
- *
- * Return: 0 on success, or negative errno on error.
- */
-static int write_number(void *p, u64 value, int cells)
-{
- if (cells == 1) {
- u32 tmp;
-
- if (value > U32_MAX)
- return -EINVAL;
-
- tmp = cpu_to_be32(value);
- memcpy(p, , sizeof(tmp));
- } else if (cells == 2) {
- u64 tmp;
-
- tmp = cpu_to_be64(value);
- memcpy(p, , sizeof(tmp));
- } else
- return -EINVAL;
-
- return 0;
-}
-
/**
* setup_ima_buffer - add IMA buffer information to the fdt
* @image: kexec image being loaded.
@@ -273,32 +243,15 @@ static int write_number(void *p, u64 value, int cells)
static int setup_ima_buffer(const struct kimage *image, void *fdt,
int chosen_node)
{
- int ret, addr_cells, size_cells, entry_size;
- u8 value[16];
+ int ret;
if (!image->ima_buffer_size)
return 0;
- ret = get_addr_size_cells(_cells, _cells);
- if (ret)
- return ret;
-
- entry_size = 4 * (addr_cells + size_cells);
-
- if (entry_size > sizeof(value))
- return -EINVAL;
-
- ret = write_number(value, image->ima_buffer_addr, addr_cells);
- if (ret)
- return ret;
-
- ret = write_number(value + 4 * addr_cells, image->ima_buffer_size,
- size_cells);
- if (ret)
- return ret;
-
- ret = fdt_setprop(fdt, chosen_node, "linux,ima-kexec-buffer", value,
- entry_size);
+ ret = fdt_appendprop_addrrange(fdt, 0, chosen_node,
+ "linux,ima-kexec-buffer",
+ image->ima_buffer_addr,
+ image->ima_buffer_size);
if (ret < 0)
return -EINVAL;
--
2.30.0
[PATCH v17 07/10] powerpc: Move arch independent ima kexec functions to drivers/of/kexec.c
The functions defined in "arch/powerpc/kexec/ima.c" handle setting up
and freeing the resources required to carry over the IMA measurement
list from the current kernel to the next kernel across kexec system call.
These functions do not have architecture specific code, but are
currently limited to powerpc.
Move remove_ima_buffer() and setup_ima_buffer() calls into
of_kexec_alloc_and_setup_fdt() defined in "drivers/of/kexec.c".
Move the remaining architecture independent functions from
"arch/powerpc/kexec/ima.c" to "drivers/of/kexec.c".
Delete "arch/powerpc/kexec/ima.c" and "arch/powerpc/include/asm/ima.h".
Remove references to the deleted files and functions in powerpc and
in ima.
Co-developed-by: Prakhar Srivastava
Signed-off-by: Prakhar Srivastava
Signed-off-by: Lakshmi Ramasubramanian
---
arch/powerpc/include/asm/ima.h| 27
arch/powerpc/include/asm/kexec.h | 3 -
arch/powerpc/kexec/Makefile | 7 -
arch/powerpc/kexec/file_load.c| 25
arch/powerpc/kexec/file_load_64.c | 4 -
arch/powerpc/kexec/ima.c | 202 -
drivers/of/kexec.c| 239 ++
include/linux/of.h| 2 +
security/integrity/ima/ima.h | 4 -
9 files changed, 241 insertions(+), 272 deletions(-)
delete mode 100644 arch/powerpc/include/asm/ima.h
delete mode 100644 arch/powerpc/kexec/ima.c
diff --git a/arch/powerpc/include/asm/ima.h b/arch/powerpc/include/asm/ima.h
deleted file mode 100644
index 51f64fd06c19..
--- a/arch/powerpc/include/asm/ima.h
+++ /dev/null
@@ -1,27 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_POWERPC_IMA_H
-#define _ASM_POWERPC_IMA_H
-
-struct kimage;
-
-int ima_get_kexec_buffer(void **addr, size_t *size);
-int ima_free_kexec_buffer(void);
-
-#ifdef CONFIG_IMA
-void remove_ima_buffer(void *fdt, int chosen_node);
-#else
-static inline void remove_ima_buffer(void *fdt, int chosen_node) {}
-#endif
-
-#ifdef CONFIG_IMA_KEXEC
-int setup_ima_buffer(const struct kimage *image, void *fdt, int chosen_node);
-#else
-static inline int setup_ima_buffer(const struct kimage *image, void *fdt,
- int chosen_node)
-{
- remove_ima_buffer(fdt, chosen_node);
- return 0;
-}
-#endif /* CONFIG_IMA_KEXEC */
-
-#endif /* _ASM_POWERPC_IMA_H */
diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
index ecf88533d6b4..2b87993f6e66 100644
--- a/arch/powerpc/include/asm/kexec.h
+++ b/arch/powerpc/include/asm/kexec.h
@@ -119,9 +119,6 @@ char *setup_kdump_cmdline(struct kimage *image, char
*cmdline,
int setup_purgatory(struct kimage *image, const void *slave_code,
const void *fdt, unsigned long kernel_load_addr,
unsigned long fdt_load_addr);
-int setup_new_fdt(const struct kimage *image, void *fdt,
- unsigned long initrd_load_addr, unsigned long initrd_len,
- const char *cmdline);
int delete_fdt_mem_rsv(void *fdt, unsigned long start, unsigned long size);
#ifdef CONFIG_PPC64
diff --git a/arch/powerpc/kexec/Makefile b/arch/powerpc/kexec/Makefile
index 4aff6846c772..b6c52608cb49 100644
--- a/arch/powerpc/kexec/Makefile
+++ b/arch/powerpc/kexec/Makefile
@@ -9,13 +9,6 @@ obj-$(CONFIG_PPC32)+= relocate_32.o
obj-$(CONFIG_KEXEC_FILE) += file_load.o ranges.o file_load_$(BITS).o
elf_$(BITS).o
-ifdef CONFIG_HAVE_IMA_KEXEC
-ifdef CONFIG_IMA
-obj-y += ima.o
-endif
-endif
-
-
# Disable GCOV, KCOV & sanitizers in odd or sensitive code
GCOV_PROFILE_core_$(BITS).o := n
KCOV_INSTRUMENT_core_$(BITS).o := n
diff --git a/arch/powerpc/kexec/file_load.c b/arch/powerpc/kexec/file_load.c
index d23e2969395c..bd8b956aafc3 100644
--- a/arch/powerpc/kexec/file_load.c
+++ b/arch/powerpc/kexec/file_load.c
@@ -19,7 +19,6 @@
#include
#include
#include
-#include
#define SLAVE_CODE_SIZE256 /* First 0x100 bytes */
@@ -140,27 +139,3 @@ int delete_fdt_mem_rsv(void *fdt, unsigned long start,
unsigned long size)
return -ENOENT;
}
-
-/*
- * setup_new_fdt - modify /chosen and memory reservation for the next kernel
- * @image: kexec image being loaded.
- * @fdt: Flattened device tree for the next kernel.
- * @initrd_load_addr: Address where the next initrd will be loaded.
- * @initrd_len:Size of the next initrd, or 0 if there will be
none.
- * @cmdline: Command line for the next kernel, or NULL if there will
- * be none.
- *
- * Return: 0 on success, or negative errno on error.
- */
-int setup_new_fdt(const struct kimage *image, void *fdt,
- unsigned long initrd_load_addr, unsigned long initrd_len,
- const char *cmdline)
-{
- int ret;
-
- ret = setup_ima_buffer(image, fdt, fdt_path_offset(fdt, "/chosen"));
- if (ret)
- pr_err("Error setting up the new device
[PATCH v17 06/10] powerpc: Enable passing IMA log to next kernel on kexec
CONFIG_HAVE_IMA_KEXEC is enabled to indicate that the IMA measurement log information is present in the device tree. This should be selected only if CONFIG_IMA is enabled. Update CONFIG_KEXEC_FILE to select CONFIG_HAVE_IMA_KEXEC, if CONFIG_IMA is enabled, to indicate that the IMA measurement log information is present in the device tree for powerpc. Signed-off-by: Lakshmi Ramasubramanian Suggested-by: Thiago Jung Bauermann --- arch/powerpc/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 107bb4319e0e..d6e593ad270e 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -554,7 +554,7 @@ config KEXEC config KEXEC_FILE bool "kexec file based system call" select KEXEC_CORE - select HAVE_IMA_KEXEC + select HAVE_IMA_KEXEC if IMA select BUILD_BIN2C select KEXEC_ELF depends on PPC64 -- 2.30.0
[PATCH v17 05/10] powerpc: Move ima buffer fields to struct kimage
The fields ima_buffer_addr and ima_buffer_size in "struct kimage_arch"
for powerpc are used to carry forward the IMA measurement list across
kexec system call. These fields are not architecture specific, but are
currently limited to powerpc.
arch_ima_add_kexec_buffer() defined in "arch/powerpc/kexec/ima.c"
sets ima_buffer_addr and ima_buffer_size for the kexec system call.
This function does not have architecture specific code, but is
currently limited to powerpc.
Move ima_buffer_addr and ima_buffer_size to "struct kimage".
Rename arch_ima_add_kexec_buffer() to of_ima_add_kexec_buffer()
and move it in drivers/of/kexec.c.
Co-developed-by: Prakhar Srivastava
Signed-off-by: Prakhar Srivastava
Signed-off-by: Lakshmi Ramasubramanian
Suggested-by: Will Deacon
---
arch/powerpc/include/asm/ima.h | 3 ---
arch/powerpc/include/asm/kexec.h | 5 -
arch/powerpc/kexec/ima.c | 29 ++---
drivers/of/kexec.c | 23 +++
include/linux/kexec.h | 3 +++
include/linux/of.h | 5 +
security/integrity/ima/ima_kexec.c | 3 ++-
7 files changed, 39 insertions(+), 32 deletions(-)
diff --git a/arch/powerpc/include/asm/ima.h b/arch/powerpc/include/asm/ima.h
index ead488cf3981..51f64fd06c19 100644
--- a/arch/powerpc/include/asm/ima.h
+++ b/arch/powerpc/include/asm/ima.h
@@ -14,9 +14,6 @@ static inline void remove_ima_buffer(void *fdt, int
chosen_node) {}
#endif
#ifdef CONFIG_IMA_KEXEC
-int arch_ima_add_kexec_buffer(struct kimage *image, unsigned long load_addr,
- size_t size);
-
int setup_ima_buffer(const struct kimage *image, void *fdt, int chosen_node);
#else
static inline int setup_ima_buffer(const struct kimage *image, void *fdt,
diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
index bdd0ddb9ac4d..ecf88533d6b4 100644
--- a/arch/powerpc/include/asm/kexec.h
+++ b/arch/powerpc/include/asm/kexec.h
@@ -112,11 +112,6 @@ struct kimage_arch {
unsigned long elf_headers_sz;
void *elf_headers;
void *fdt;
-
-#ifdef CONFIG_IMA_KEXEC
- phys_addr_t ima_buffer_addr;
- size_t ima_buffer_size;
-#endif
};
char *setup_kdump_cmdline(struct kimage *image, char *cmdline,
diff --git a/arch/powerpc/kexec/ima.c b/arch/powerpc/kexec/ima.c
index 720e50e490b6..ed38125e2f87 100644
--- a/arch/powerpc/kexec/ima.c
+++ b/arch/powerpc/kexec/ima.c
@@ -128,23 +128,6 @@ void remove_ima_buffer(void *fdt, int chosen_node)
}
#ifdef CONFIG_IMA_KEXEC
-/**
- * arch_ima_add_kexec_buffer - do arch-specific steps to add the IMA buffer
- *
- * Architectures should use this function to pass on the IMA buffer
- * information to the next kernel.
- *
- * Return: 0 on success, negative errno on error.
- */
-int arch_ima_add_kexec_buffer(struct kimage *image, unsigned long load_addr,
- size_t size)
-{
- image->arch.ima_buffer_addr = load_addr;
- image->arch.ima_buffer_size = size;
-
- return 0;
-}
-
static int write_number(void *p, u64 value, int cells)
{
if (cells == 1) {
@@ -180,7 +163,7 @@ int setup_ima_buffer(const struct kimage *image, void *fdt,
int chosen_node)
u8 value[16];
remove_ima_buffer(fdt, chosen_node);
- if (!image->arch.ima_buffer_size)
+ if (!image->ima_buffer_size)
return 0;
ret = get_addr_size_cells(_cells, _cells);
@@ -192,11 +175,11 @@ int setup_ima_buffer(const struct kimage *image, void
*fdt, int chosen_node)
if (entry_size > sizeof(value))
return -EINVAL;
- ret = write_number(value, image->arch.ima_buffer_addr, addr_cells);
+ ret = write_number(value, image->ima_buffer_addr, addr_cells);
if (ret)
return ret;
- ret = write_number(value + 4 * addr_cells, image->arch.ima_buffer_size,
+ ret = write_number(value + 4 * addr_cells, image->ima_buffer_size,
size_cells);
if (ret)
return ret;
@@ -206,13 +189,13 @@ int setup_ima_buffer(const struct kimage *image, void
*fdt, int chosen_node)
if (ret < 0)
return -EINVAL;
- ret = fdt_add_mem_rsv(fdt, image->arch.ima_buffer_addr,
- image->arch.ima_buffer_size);
+ ret = fdt_add_mem_rsv(fdt, image->ima_buffer_addr,
+ image->ima_buffer_size);
if (ret)
return -EINVAL;
pr_debug("IMA buffer at 0x%llx, size = 0x%zx\n",
-image->arch.ima_buffer_addr, image->arch.ima_buffer_size);
+image->ima_buffer_addr, image->ima_buffer_size);
return 0;
}
diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c
index 469e09613cdd..9f33d215b9f2 100644
--- a/drivers/of/kexec.c
+++ b/drivers/of/kexec.c
@@ -63,6 +63,29 @@ static int fdt_find_and_del_mem_rsv(void *fdt, unsigned long
start, unsigned lon
return
[PATCH v17 04/10] powerpc: Use common of_kexec_alloc_and_setup_fdt()
From: Rob Herring
The code for setting up the /chosen node in the device tree
and updating the memory reservation for the next kernel has been
moved to of_kexec_alloc_and_setup_fdt() defined in "drivers/of/kexec.c".
Use the common of_kexec_alloc_and_setup_fdt() to setup the device tree
and update the memory reservation for kexec for powerpc.
Signed-off-by: Rob Herring
Signed-off-by: Lakshmi Ramasubramanian
---
arch/powerpc/include/asm/kexec.h | 1 +
arch/powerpc/kexec/elf_64.c | 29 ---
arch/powerpc/kexec/file_load.c| 132 +-
arch/powerpc/kexec/file_load_64.c | 3 +
4 files changed, 25 insertions(+), 140 deletions(-)
diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
index dbf09d2f36d0..bdd0ddb9ac4d 100644
--- a/arch/powerpc/include/asm/kexec.h
+++ b/arch/powerpc/include/asm/kexec.h
@@ -111,6 +111,7 @@ struct kimage_arch {
unsigned long elf_headers_mem;
unsigned long elf_headers_sz;
void *elf_headers;
+ void *fdt;
#ifdef CONFIG_IMA_KEXEC
phys_addr_t ima_buffer_addr;
diff --git a/arch/powerpc/kexec/elf_64.c b/arch/powerpc/kexec/elf_64.c
index d0e459bb2f05..bfabd06f99b1 100644
--- a/arch/powerpc/kexec/elf_64.c
+++ b/arch/powerpc/kexec/elf_64.c
@@ -19,6 +19,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -29,7 +30,6 @@ static void *elf64_load(struct kimage *image, char
*kernel_buf,
unsigned long cmdline_len)
{
int ret;
- unsigned int fdt_size;
unsigned long kernel_load_addr;
unsigned long initrd_load_addr = 0, fdt_load_addr;
void *fdt;
@@ -102,19 +102,13 @@ static void *elf64_load(struct kimage *image, char
*kernel_buf,
pr_debug("Loaded initrd at 0x%lx\n", initrd_load_addr);
}
- fdt_size = fdt_totalsize(initial_boot_params) * 2;
- fdt = kmalloc(fdt_size, GFP_KERNEL);
+ fdt = of_kexec_alloc_and_setup_fdt(image, initrd_load_addr,
+ initrd_len, cmdline);
if (!fdt) {
pr_err("Not enough memory for the device tree.\n");
ret = -ENOMEM;
goto out;
}
- ret = fdt_open_into(initial_boot_params, fdt, fdt_size);
- if (ret < 0) {
- pr_err("Error setting up the new device tree.\n");
- ret = -EINVAL;
- goto out;
- }
ret = setup_new_fdt_ppc64(image, fdt, initrd_load_addr,
initrd_len, cmdline);
@@ -124,13 +118,17 @@ static void *elf64_load(struct kimage *image, char
*kernel_buf,
fdt_pack(fdt);
kbuf.buffer = fdt;
- kbuf.bufsz = kbuf.memsz = fdt_size;
+ kbuf.bufsz = kbuf.memsz = fdt_totalsize(fdt);
kbuf.buf_align = PAGE_SIZE;
kbuf.top_down = true;
kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
ret = kexec_add_buffer();
if (ret)
goto out;
+
+ /* FDT will be freed in arch_kimage_file_post_load_cleanup */
+ image->arch.fdt = fdt;
+
fdt_load_addr = kbuf.mem;
pr_debug("Loaded device tree at 0x%lx\n", fdt_load_addr);
@@ -145,8 +143,15 @@ static void *elf64_load(struct kimage *image, char
*kernel_buf,
kfree(modified_cmdline);
kexec_free_elf_info(_info);
- /* Make kimage_file_post_load_cleanup free the fdt buffer for us. */
- return ret ? ERR_PTR(ret) : fdt;
+ /*
+* Once FDT buffer has been successfully passed to kexec_add_buffer(),
+* the FDT buffer address is saved in image->arch.fdt. In that case,
+* the memory cannot be freed here in case of any other error.
+*/
+ if (ret && !image->arch.fdt)
+ kvfree(fdt);
+
+ return ret ? ERR_PTR(ret) : NULL;
}
const struct kexec_file_ops kexec_elf64_ops = {
diff --git a/arch/powerpc/kexec/file_load.c b/arch/powerpc/kexec/file_load.c
index e452b11df631..d23e2969395c 100644
--- a/arch/powerpc/kexec/file_load.c
+++ b/arch/powerpc/kexec/file_load.c
@@ -156,135 +156,11 @@ int setup_new_fdt(const struct kimage *image, void *fdt,
unsigned long initrd_load_addr, unsigned long initrd_len,
const char *cmdline)
{
- int ret, chosen_node;
- const void *prop;
-
- /* Remove memory reservation for the current device tree. */
- ret = delete_fdt_mem_rsv(fdt, __pa(initial_boot_params),
-fdt_totalsize(initial_boot_params));
- if (ret == 0)
- pr_debug("Removed old device tree reservation.\n");
- else if (ret != -ENOENT)
- return ret;
-
- chosen_node = fdt_path_offset(fdt, "/chosen");
- if (chosen_node == -FDT_ERR_NOTFOUND) {
- chosen_node = fdt_add_subnode(fdt, fdt_path_offset(fdt, "/"),
- "chosen");
- if (chosen_node < 0) {
-
[PATCH v17 03/10] arm64: Use common of_kexec_alloc_and_setup_fdt()
From: Rob Herring
The code for setting up the /chosen node in the device tree
and updating the memory reservation for the next kernel has been
moved to of_kexec_alloc_and_setup_fdt() defined in "drivers/of/kexec.c".
Use the common of_kexec_alloc_and_setup_fdt() to setup the device tree
and update the memory reservation for kexec for arm64.
Signed-off-by: Rob Herring
Signed-off-by: Lakshmi Ramasubramanian
---
arch/arm64/kernel/machine_kexec_file.c | 180 ++---
1 file changed, 8 insertions(+), 172 deletions(-)
diff --git a/arch/arm64/kernel/machine_kexec_file.c
b/arch/arm64/kernel/machine_kexec_file.c
index 03210f644790..5a203f4f31fd 100644
--- a/arch/arm64/kernel/machine_kexec_file.c
+++ b/arch/arm64/kernel/machine_kexec_file.c
@@ -15,23 +15,12 @@
#include
#include
#include
+#include
#include
-#include
#include
#include
#include
#include
-#include
-
-/* relevant device tree properties */
-#define FDT_PROP_KEXEC_ELFHDR "linux,elfcorehdr"
-#define FDT_PROP_MEM_RANGE "linux,usable-memory-range"
-#define FDT_PROP_INITRD_START "linux,initrd-start"
-#define FDT_PROP_INITRD_END"linux,initrd-end"
-#define FDT_PROP_BOOTARGS "bootargs"
-#define FDT_PROP_KASLR_SEED"kaslr-seed"
-#define FDT_PROP_RNG_SEED "rng-seed"
-#define RNG_SEED_SIZE 128
const struct kexec_file_ops * const kexec_file_loaders[] = {
_image_ops,
@@ -40,7 +29,7 @@ const struct kexec_file_ops * const kexec_file_loaders[] = {
int arch_kimage_file_post_load_cleanup(struct kimage *image)
{
- vfree(image->arch.dtb);
+ kvfree(image->arch.dtb);
image->arch.dtb = NULL;
vfree(image->arch.elf_headers);
@@ -50,162 +39,6 @@ int arch_kimage_file_post_load_cleanup(struct kimage *image)
return kexec_image_post_load_cleanup_default(image);
}
-static int setup_dtb(struct kimage *image,
-unsigned long initrd_load_addr, unsigned long initrd_len,
-char *cmdline, void *dtb)
-{
- int off, ret;
-
- ret = fdt_path_offset(dtb, "/chosen");
- if (ret < 0)
- goto out;
-
- off = ret;
-
- ret = fdt_delprop(dtb, off, FDT_PROP_KEXEC_ELFHDR);
- if (ret && ret != -FDT_ERR_NOTFOUND)
- goto out;
- ret = fdt_delprop(dtb, off, FDT_PROP_MEM_RANGE);
- if (ret && ret != -FDT_ERR_NOTFOUND)
- goto out;
-
- if (image->type == KEXEC_TYPE_CRASH) {
- /* add linux,elfcorehdr */
- ret = fdt_appendprop_addrrange(dtb, 0, off,
- FDT_PROP_KEXEC_ELFHDR,
- image->arch.elf_headers_mem,
- image->arch.elf_headers_sz);
- if (ret)
- return (ret == -FDT_ERR_NOSPACE ? -ENOMEM : -EINVAL);
-
- /* add linux,usable-memory-range */
- ret = fdt_appendprop_addrrange(dtb, 0, off,
- FDT_PROP_MEM_RANGE,
- crashk_res.start,
- crashk_res.end - crashk_res.start + 1);
- if (ret)
- return (ret == -FDT_ERR_NOSPACE ? -ENOMEM : -EINVAL);
- }
-
- /* add bootargs */
- if (cmdline) {
- ret = fdt_setprop_string(dtb, off, FDT_PROP_BOOTARGS, cmdline);
- if (ret)
- goto out;
- } else {
- ret = fdt_delprop(dtb, off, FDT_PROP_BOOTARGS);
- if (ret && (ret != -FDT_ERR_NOTFOUND))
- goto out;
- }
-
- /* add initrd-* */
- if (initrd_load_addr) {
- ret = fdt_setprop_u64(dtb, off, FDT_PROP_INITRD_START,
- initrd_load_addr);
- if (ret)
- goto out;
-
- ret = fdt_setprop_u64(dtb, off, FDT_PROP_INITRD_END,
- initrd_load_addr + initrd_len);
- if (ret)
- goto out;
- } else {
- ret = fdt_delprop(dtb, off, FDT_PROP_INITRD_START);
- if (ret && (ret != -FDT_ERR_NOTFOUND))
- goto out;
-
- ret = fdt_delprop(dtb, off, FDT_PROP_INITRD_END);
- if (ret && (ret != -FDT_ERR_NOTFOUND))
- goto out;
- }
-
- /* add kaslr-seed */
- ret = fdt_delprop(dtb, off, FDT_PROP_KASLR_SEED);
- if (ret == -FDT_ERR_NOTFOUND)
- ret = 0;
- else if (ret)
- goto out;
-
- if (rng_is_initialized()) {
- u64 seed = get_random_u64();
- ret = fdt_setprop_u64(dtb, off, FDT_PROP_KASLR_SEED, seed);
- if (ret)
- goto out;
- } else {
- pr_notice("RNG is not initialised: omitting \"%s\" property\n",
- FDT_PROP_KASLR_SEED);
- }
-
-
[PATCH v17 01/10] powerpc: Rename kexec elfcorehdr_addr to elf_headers_mem
From: Rob Herring
The architecture specific field, elfcorehdr_addr in struct kimage_arch,
that holds the address of the buffer in memory for ELF core header for
powerpc has a different name than the one used for arm64. This makes
it hard to have a common code for setting up the device tree for
kexec system call.
Rename elfcorehdr_addr to elf_headers_mem to align with arm64 name so
common code can use it.
Signed-off-by: Rob Herring
Reviewed-by: Thiago Jung Bauermann
Reviewed-by: Lakshmi Ramasubramanian
---
arch/powerpc/include/asm/kexec.h | 2 +-
arch/powerpc/kexec/file_load.c| 4 ++--
arch/powerpc/kexec/file_load_64.c | 4 ++--
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
index 55d6ede30c19..dbf09d2f36d0 100644
--- a/arch/powerpc/include/asm/kexec.h
+++ b/arch/powerpc/include/asm/kexec.h
@@ -108,7 +108,7 @@ struct kimage_arch {
unsigned long backup_start;
void *backup_buf;
- unsigned long elfcorehdr_addr;
+ unsigned long elf_headers_mem;
unsigned long elf_headers_sz;
void *elf_headers;
diff --git a/arch/powerpc/kexec/file_load.c b/arch/powerpc/kexec/file_load.c
index 9a232bc36c8f..e452b11df631 100644
--- a/arch/powerpc/kexec/file_load.c
+++ b/arch/powerpc/kexec/file_load.c
@@ -45,7 +45,7 @@ char *setup_kdump_cmdline(struct kimage *image, char *cmdline,
return NULL;
elfcorehdr_strlen = sprintf(cmdline_ptr, "elfcorehdr=0x%lx ",
- image->arch.elfcorehdr_addr);
+ image->arch.elf_headers_mem);
if (elfcorehdr_strlen + cmdline_len > COMMAND_LINE_SIZE) {
pr_err("Appending elfcorehdr= exceeds cmdline size\n");
@@ -263,7 +263,7 @@ int setup_new_fdt(const struct kimage *image, void *fdt,
* Avoid elfcorehdr from being stomped on in kdump kernel by
* setting up memory reserve map.
*/
- ret = fdt_add_mem_rsv(fdt, image->arch.elfcorehdr_addr,
+ ret = fdt_add_mem_rsv(fdt, image->arch.elf_headers_mem,
image->arch.elf_headers_sz);
if (ret) {
pr_err("Error reserving elfcorehdr memory: %s\n",
diff --git a/arch/powerpc/kexec/file_load_64.c
b/arch/powerpc/kexec/file_load_64.c
index c69bcf9b547a..a05c19b3cc60 100644
--- a/arch/powerpc/kexec/file_load_64.c
+++ b/arch/powerpc/kexec/file_load_64.c
@@ -815,7 +815,7 @@ static int load_elfcorehdr_segment(struct kimage *image,
struct kexec_buf *kbuf)
goto out;
}
- image->arch.elfcorehdr_addr = kbuf->mem;
+ image->arch.elf_headers_mem = kbuf->mem;
image->arch.elf_headers_sz = headers_sz;
image->arch.elf_headers = headers;
out:
@@ -851,7 +851,7 @@ int load_crashdump_segments_ppc64(struct kimage *image,
return ret;
}
pr_debug("Loaded elf core header at 0x%lx, bufsz=0x%lx memsz=0x%lx\n",
-image->arch.elfcorehdr_addr, kbuf->bufsz, kbuf->memsz);
+image->arch.elf_headers_mem, kbuf->bufsz, kbuf->memsz);
return 0;
}
--
2.30.0
[PATCH v17 02/10] of: Add a common kexec FDT setup function
From: Rob Herring
Both arm64 and powerpc do essentially the same FDT /chosen setup for
kexec. The differences are either omissions that arm64 should have
or additional properties that will be ignored. The setup code can be
combined and shared by both powerpc and arm64.
The differences relative to the arm64 version:
- If /chosen doesn't exist, it will be created (should never happen).
- Any old dtb and initrd reserved memory will be released.
- The new initrd and elfcorehdr are marked reserved.
- "linux,booted-from-kexec" is set.
The differences relative to the powerpc version:
- "kaslr-seed" and "rng-seed" may be set.
- "linux,elfcorehdr" is set.
- Any existing "linux,usable-memory-range" is removed.
Combine the code for setting up the /chosen node in the FDT and updating
the memory reservation for kexec, for powerpc and arm64, in
of_kexec_alloc_and_setup_fdt() and move it to "drivers/of/kexec.c".
Signed-off-by: Rob Herring
Signed-off-by: Lakshmi Ramasubramanian
---
drivers/of/Makefile | 6 ++
drivers/of/kexec.c | 258
include/linux/of.h | 13 +++
3 files changed, 277 insertions(+)
create mode 100644 drivers/of/kexec.c
diff --git a/drivers/of/Makefile b/drivers/of/Makefile
index 6e1e5212f058..c13b982084a3 100644
--- a/drivers/of/Makefile
+++ b/drivers/of/Makefile
@@ -14,4 +14,10 @@ obj-$(CONFIG_OF_RESOLVE) += resolver.o
obj-$(CONFIG_OF_OVERLAY) += overlay.o
obj-$(CONFIG_OF_NUMA) += of_numa.o
+ifdef CONFIG_KEXEC_FILE
+ifdef CONFIG_OF_FLATTREE
+obj-y += kexec.o
+endif
+endif
+
obj-$(CONFIG_OF_UNITTEST) += unittest-data/
diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c
new file mode 100644
index ..469e09613cdd
--- /dev/null
+++ b/drivers/of/kexec.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2020 Arm Limited
+ *
+ * Based on arch/arm64/kernel/machine_kexec_file.c:
+ * Copyright (C) 2018 Linaro Limited
+ *
+ * And arch/powerpc/kexec/file_load.c:
+ * Copyright (C) 2016 IBM Corporation
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+/* relevant device tree properties */
+#define FDT_PROP_KEXEC_ELFHDR "linux,elfcorehdr"
+#define FDT_PROP_MEM_RANGE "linux,usable-memory-range"
+#define FDT_PROP_INITRD_START "linux,initrd-start"
+#define FDT_PROP_INITRD_END"linux,initrd-end"
+#define FDT_PROP_BOOTARGS "bootargs"
+#define FDT_PROP_KASLR_SEED"kaslr-seed"
+#define FDT_PROP_RNG_SEED "rng-seed"
+#define RNG_SEED_SIZE 128
+
+/**
+ * fdt_find_and_del_mem_rsv - delete memory reservation with given address and
size
+ *
+ * @fdt: Flattened device tree for the current kernel.
+ * @start: Starting address of the reserved memory.
+ * @size: Size of the reserved memory.
+ *
+ * Return: 0 on success, or negative errno on error.
+ */
+static int fdt_find_and_del_mem_rsv(void *fdt, unsigned long start, unsigned
long size)
+{
+ int i, ret, num_rsvs = fdt_num_mem_rsv(fdt);
+
+ for (i = 0; i < num_rsvs; i++) {
+ u64 rsv_start, rsv_size;
+
+ ret = fdt_get_mem_rsv(fdt, i, _start, _size);
+ if (ret) {
+ pr_err("Malformed device tree.\n");
+ return -EINVAL;
+ }
+
+ if (rsv_start == start && rsv_size == size) {
+ ret = fdt_del_mem_rsv(fdt, i);
+ if (ret) {
+ pr_err("Error deleting device tree
reservation.\n");
+ return -EINVAL;
+ }
+
+ return 0;
+ }
+ }
+
+ return -ENOENT;
+}
+
+/*
+ * of_kexec_alloc_and_setup_fdt - Alloc and setup a new Flattened Device Tree
+ *
+ * @image: kexec image being loaded.
+ * @initrd_load_addr: Address where the next initrd will be loaded.
+ * @initrd_len:Size of the next initrd, or 0 if there will be
none.
+ * @cmdline: Command line for the next kernel, or NULL if there will
+ * be none.
+ *
+ * Return: fdt on success, or NULL errno on error.
+ */
+void *of_kexec_alloc_and_setup_fdt(const struct kimage *image,
+ unsigned long initrd_load_addr,
+ unsigned long initrd_len,
+ const char *cmdline)
+{
+ void *fdt;
+ int ret, chosen_node;
+ const void *prop;
+ unsigned long fdt_size;
+
+ fdt_size = fdt_totalsize(initial_boot_params) +
+ (cmdline ? strlen(cmdline) : 0) +
+ FDT_EXTRA_SPACE;
+
+ fdt = kvmalloc(fdt_size, GFP_KERNEL);
+ if (!fdt)
+ return NULL;
+
+ ret = fdt_open_into(initial_boot_params, fdt, fdt_size);
+ if (ret < 0) {
+ pr_err("Error %d setting up the new device tree.\n", ret);
+ goto out;
+ }
+
+
[PATCH v17 00/10] Carry forward IMA measurement log on kexec on ARM64
On kexec file load Integrity Measurement Architecture (IMA) subsystem
may verify the IMA signature of the kernel and initramfs, and measure
it. The command line parameters passed to the kernel in the kexec call
may also be measured by IMA. A remote attestation service can verify
a TPM quote based on the TPM event log, the IMA measurement list, and
the TPM PCR data. This can be achieved only if the IMA measurement log
is carried over from the current kernel to the next kernel across
the kexec call.
powerpc already supports carrying forward the IMA measurement log on
kexec. This patch set adds support for carrying forward the IMA
measurement log on kexec on ARM64.
This patch set moves the platform independent code defined for powerpc
such that it can be reused for other platforms as well. A chosen node
"linux,ima-kexec-buffer" is added to the DTB for ARM64 to hold
the address and the size of the memory reserved to carry
the IMA measurement log.
This patch set has been tested for ARM64 platform using QEMU.
I would like help from the community for testing this change on powerpc.
Thanks.
This patch set is based on
commit 96acc833dec8 ("ima: Free IMA measurement buffer after kexec syscall")
in https://git.kernel.org/pub/scm/linux/kernel/git/zohar/linux-integrity.git
"next-integrity" branch.
Changelog:
v17
- Renamed of_kexec_setup_new_fdt() to of_kexec_alloc_and_setup_fdt(),
and moved memory allocation for the new FDT to this function.
v16
- Defined functions to allocate and free buffer for FDT for powerpc
and arm64.
- Moved ima_buffer_addr and ima_buffer_size fields from
"struct kimage_arch" in powerpc to "struct kimage"
v15
- Included Rob's patches in the patch set, and rebased
the changes to "next-integrity" branch.
- Allocate memory for DTB, on arm64, using kmalloc() instead of
vmalloc() to keep it consistent with powerpc implementation.
- Call of_kexec_setup_new_fdt() from setup_new_fdt_ppc64() and
remove setup_new_fdt() in the same patch to keep it bisect safe.
v14
- Select CONFIG_HAVE_IMA_KEXEC for CONFIG_KEXEC_FILE, for powerpc
and arm64, if CONFIG_IMA is enabled.
- Use IS_ENABLED() macro instead of "#ifdef" in remove_ima_buffer(),
ima_get_kexec_buffer(), and ima_free_kexec_buffer().
- Call of_kexec_setup_new_fdt() from setup_new_fdt_ppc64() and
remove setup_new_fdt() in "arch/powerpc/kexec/file_load.c".
v13
- Moved the arch independent functions to drivers/of/kexec.c
and then refactored the code.
- Moved arch_ima_add_kexec_buffer() to
security/integrity/ima/ima_kexec.c
v12
- Use fdt_appendprop_addrrange() in setup_ima_buffer()
to setup the IMA measurement list property in
the device tree.
- Moved architecture independent functions from
"arch/powerpc/kexec/ima.c" to "drivers/of/kexec."
- Deleted "arch/powerpc/kexec/ima.c" and
"arch/powerpc/include/asm/ima.h".
v11
- Rebased the changes on the kexec code refactoring done by
Rob Herring in his "dt/kexec" branch
- Removed "extern" keyword in function declarations
- Removed unnecessary header files included in C files
- Updated patch descriptions per Thiago's comments
v10
- Moved delete_fdt_mem_rsv(), remove_ima_buffer(),
get_ima_kexec_buffer, and get_root_addr_size_cells()
to drivers/of/kexec.c
- Moved arch_ima_add_kexec_buffer() to
security/integrity/ima/ima_kexec.c
- Conditionally define IMA buffer fields in struct kimage_arch
v9
- Moved delete_fdt_mem_rsv() to drivers/of/kexec_fdt.c
- Defined a new function get_ima_kexec_buffer() in
drivers/of/ima_kexec.c to replace do_get_kexec_buffer()
- Changed remove_ima_kexec_buffer() to the original function name
remove_ima_buffer()
- Moved remove_ima_buffer() to drivers/of/ima_kexec.c
- Moved ima_get_kexec_buffer() and ima_free_kexec_buffer()
to security/integrity/ima/ima_kexec.c
v8:
- Moved remove_ima_kexec_buffer(), do_get_kexec_buffer(), and
delete_fdt_mem_rsv() to drivers/of/fdt.c
- Moved ima_dump_measurement_list() and ima_add_kexec_buffer()
back to security/integrity/ima/ima_kexec.c
v7:
- Renamed remove_ima_buffer() to remove_ima_kexec_buffer() and moved
this function definition to kernel.
- Moved delete_fdt_mem_rsv() definition to kernel
- Moved ima_dump_measurement_list() and ima_add_kexec_buffer() to
a new file namely ima_kexec_fdt.c in IMA
v6:
- Remove any existing FDT_PROP_IMA_KEXEC_BUFFER property in the device
tree and also its corresponding memory reservation in the currently
running kernel.
- Moved the function remove_ima_buffer() defined for powerpc to IMA
and renamed the function to ima_remove_kexec_buffer(). Also, moved
delete_fdt_mem_rsv() from powerpc to IMA.
v5:
- Merged get_addr_size_cells() and do_get_kexec_buffer() into a single
function when moving the arch independent code from powerpc to IMA
- Reverted the change to use FDT functions in powerpc code and added
back the
RE: [PATCH v5 20/22] powerpc/syscall: Avoid storing 'current' in another pointer
From: Christophe Leroy
> Sent: 09 February 2021 17:04
>
> Le 09/02/2021 à 15:31, David Laight a écrit :
> > From: Segher Boessenkool
> >> Sent: 09 February 2021 13:51
> >>
> >> On Tue, Feb 09, 2021 at 12:36:20PM +1000, Nicholas Piggin wrote:
> >>> What if you did this?
> >>
> >>> +static inline struct task_struct *get_current(void)
> >>> +{
> >>> + register struct task_struct *task asm ("r2");
> >>> +
> >>> + return task;
> >>> +}
> >>
> >> Local register asm variables are *only* guaranteed to live in that
> >> register as operands to an asm. See
> >>
> >> https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables
> >> ("The only supported use" etc.)
> >>
> >> You can do something like
> >>
> >> static inline struct task_struct *get_current(void)
> >> {
> >>register struct task_struct *task asm ("r2");
> >>
> >>asm("" : "+r"(task));
> >>
> >>return task;
> >> }
> >>
> >> which makes sure that "task" actually is in r2 at the point of that asm.
> >
> > If "r2" always contains current (and is never assigned by the compiler)
> > why not use a global register variable for it?
> >
>
>
> The change proposed by Nick doesn't solve the issue.
>
> The problem is that at the begining of the function we have:
>
> unsigned long *ti_flagsp = _thread_info()->flags;
>
> When the function uses ti_flagsp for the first time, it does use 112(r2)
>
> Then the function calls some other functions.
>
> Most likely because the function could update 'current', GCC copies r2 into
> r30, so that if r2 get
> changed by the called function, ti_flagsp is still based on the previous
> value of current.
>
> Allthough we know r2 wont change, GCC doesn't know it. And in order to save
> r2 into r30, it needs to
> save r30 in the stack.
>
>
> By using _thread_info()->flags directly instead of this intermediaite
> ti_flagsp pointer, GCC
> uses r2 instead instead of doing a copy.
Does marking current_thread_info() 'pure' (I think that the right one)
work - so that gcc knows its result doesn't depend on external data
and that it doesn't change external data.
Although I'm not 100% how well those attributes actually work.
> Nick, I don't understand the reason why you need that 'ti_flagsp' local var.
Probably to save typing.
I sometimes reload locals after function calls.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT,
UK
Registration No: 1397386 (Wales)
Re: [PATCH v5 20/22] powerpc/syscall: Avoid storing 'current' in another pointer
Le 09/02/2021 à 15:31, David Laight a écrit :
From: Segher Boessenkool
Sent: 09 February 2021 13:51
On Tue, Feb 09, 2021 at 12:36:20PM +1000, Nicholas Piggin wrote:
What if you did this?
+static inline struct task_struct *get_current(void)
+{
+ register struct task_struct *task asm ("r2");
+
+ return task;
+}
Local register asm variables are *only* guaranteed to live in that
register as operands to an asm. See
https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables
("The only supported use" etc.)
You can do something like
static inline struct task_struct *get_current(void)
{
register struct task_struct *task asm ("r2");
asm("" : "+r"(task));
return task;
}
which makes sure that "task" actually is in r2 at the point of that asm.
If "r2" always contains current (and is never assigned by the compiler)
why not use a global register variable for it?
The change proposed by Nick doesn't solve the issue.
The problem is that at the begining of the function we have:
unsigned long *ti_flagsp = _thread_info()->flags;
When the function uses ti_flagsp for the first time, it does use 112(r2)
Then the function calls some other functions.
Most likely because the function could update 'current', GCC copies r2 into r30, so that if r2 get
changed by the called function, ti_flagsp is still based on the previous value of current.
Allthough we know r2 wont change, GCC doesn't know it. And in order to save r2 into r30, it needs to
save r30 in the stack.
By using _thread_info()->flags directly instead of this intermediaite ti_flagsp pointer, GCC
uses r2 instead instead of doing a copy.
Nick, I don't understand the reason why you need that 'ti_flagsp' local var.
Christophe
[PATCH 3/5] powerpc/xive: Remove useless check on XIVE_IPI_HW_IRQ
The IPI interrupt has its own domain now. Testing the HW interrupt number is not needed anymore. Signed-off-by: Cédric Le Goater --- arch/powerpc/sysdev/xive/common.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/powerpc/sysdev/xive/common.c b/arch/powerpc/sysdev/xive/common.c index 4aceac0f3046..21e8f7ef5cc4 100644 --- a/arch/powerpc/sysdev/xive/common.c +++ b/arch/powerpc/sysdev/xive/common.c @@ -1397,13 +1397,12 @@ static void xive_flush_cpu_queue(unsigned int cpu, struct xive_cpu *xc) struct irq_desc *desc = irq_to_desc(irq); struct irq_data *d = irq_desc_get_irq_data(desc); struct xive_irq_data *xd; - unsigned int hw_irq = (unsigned int)irqd_to_hwirq(d); /* * Ignore anything that isn't a XIVE irq and ignore * IPIs, so can just be dropped. */ - if (d->domain != xive_irq_domain || hw_irq == XIVE_IPI_HW_IRQ) + if (d->domain != xive_irq_domain) continue; /* -- 2.26.2
[PATCH 4/5] powerpc/xive: Simplify xive_core_debug_show()
Now that the IPI interrupt has its own domain, the checks on the HW
interrupt number XIVE_IPI_HW_IRQ and on the chip can be replaced by a
check on the domain.
Signed-off-by: Cédric Le Goater
---
arch/powerpc/sysdev/xive/common.c | 18 --
1 file changed, 4 insertions(+), 14 deletions(-)
diff --git a/arch/powerpc/sysdev/xive/common.c
b/arch/powerpc/sysdev/xive/common.c
index 21e8f7ef5cc4..30034af4462b 100644
--- a/arch/powerpc/sysdev/xive/common.c
+++ b/arch/powerpc/sysdev/xive/common.c
@@ -1580,17 +1580,14 @@ static void xive_debug_show_cpu(struct seq_file *m, int
cpu)
seq_puts(m, "\n");
}
-static void xive_debug_show_irq(struct seq_file *m, u32 hw_irq, struct
irq_data *d)
+static void xive_debug_show_irq(struct seq_file *m, struct irq_data *d)
{
- struct irq_chip *chip = irq_data_get_irq_chip(d);
+ unsigned int hw_irq = (unsigned int)irqd_to_hwirq(d);
int rc;
u32 target;
u8 prio;
u32 lirq;
- if (!is_xive_irq(chip))
- return;
-
rc = xive_ops->get_irq_config(hw_irq, , , );
if (rc) {
seq_printf(m, "IRQ 0x%08x : no config rc=%d\n", hw_irq, rc);
@@ -1628,16 +1625,9 @@ static int xive_core_debug_show(struct seq_file *m, void
*private)
for_each_irq_desc(i, desc) {
struct irq_data *d = irq_desc_get_irq_data(desc);
- unsigned int hw_irq;
-
- if (!d)
- continue;
-
- hw_irq = (unsigned int)irqd_to_hwirq(d);
- /* IPIs are special (HW number 0) */
- if (hw_irq != XIVE_IPI_HW_IRQ)
- xive_debug_show_irq(m, hw_irq, d);
+ if (d->domain == xive_irq_domain)
+ xive_debug_show_irq(m, d);
}
return 0;
}
--
2.26.2
[PATCH 1/5] powerpc/xive: Use cpu_to_node() instead of ibm, chip-id property
The 'chip_id' field of the XIVE CPU structure is used to choose a
target for a source located on the same chip when possible. This field
is assigned on the PowerNV platform using the "ibm,chip-id" property
on pSeries under KVM when NUMA nodes are defined but it is undefined
under PowerVM. The XIVE source structure has a similar field
'src_chip' which is only assigned on the PowerNV platform.
cpu_to_node() returns a compatible value on all platforms, 0 being the
default node. It will also give us the opportunity to set the affinity
of a source on pSeries when we can localize them.
Signed-off-by: Cédric Le Goater
---
arch/powerpc/sysdev/xive/common.c | 7 +--
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/arch/powerpc/sysdev/xive/common.c
b/arch/powerpc/sysdev/xive/common.c
index 7e08be5e5e4a..776871274b69 100644
--- a/arch/powerpc/sysdev/xive/common.c
+++ b/arch/powerpc/sysdev/xive/common.c
@@ -1336,16 +1336,11 @@ static int xive_prepare_cpu(unsigned int cpu)
xc = per_cpu(xive_cpu, cpu);
if (!xc) {
- struct device_node *np;
-
xc = kzalloc_node(sizeof(struct xive_cpu),
GFP_KERNEL, cpu_to_node(cpu));
if (!xc)
return -ENOMEM;
- np = of_get_cpu_node(cpu, NULL);
- if (np)
- xc->chip_id = of_get_ibm_chip_id(np);
- of_node_put(np);
+ xc->chip_id = cpu_to_node(cpu);
xc->hw_ipi = XIVE_BAD_IRQ;
per_cpu(xive_cpu, cpu) = xc;
--
2.26.2
[PATCH 5/5] powerpc/xive: Map one IPI interrupt per node
ipistorm [*] can be used to benchmark the raw interrupt rate of an
interrupt controller by measuring the number of IPIs a system can
sustain. When applied to the XIVE interrupt controller of POWER9 and
POWER10 systems, a significant drop of the interrupt rate can be
observed when crossing the second node boundary.
This is due to the fact that a single IPI interrupt is used for all
CPUs of the system. The structure is shared and the cache line updates
impact greatly the traffic between nodes and the overall IPI
performance.
As a workaround, the impact can be reduced by deactivating the IRQ
lockup detector ("noirqdebug") which does a lot of accounting in the
Linux IRQ descriptor structure and is responsible for most of the
performance penalty.
As a fix, this proposal allocates an IPI interrupt per node, to be
shared by all CPUs of that node. It solves the scaling issue, the IRQ
lockup detector still has an impact but the XIVE interrupt rate scales
linearly. It also improves the "noirqdebug" case as showed in the
tables below.
* P9 DD2.2 - 2s * 64 threads
"noirqdebug"
Mint/sMint/s
chips cpus IPI/sys IPI/chip IPI/chipIPI/sys
--
1 0-15 4.984023 4.875405 4.996536 5.048892
0-3110.879164 10.544040 10.757632 11.037859
0-4715.345301 14.688764 14.926520 15.310053
0-6317.064907 17.066812 17.613416 17.874511
2 0-7911.768764 21.650749 22.689120 22.566508
0-9510.616812 26.878789 28.434703 28.320324
0-111 10.151693 31.397803 31.771773 32.388122
0-1279.948502 33.139336 34.875716 35.224548
* P10 DD1 - 4s (not homogeneous) 352 threads
"noirqdebug"
Mint/sMint/s
chips cpus IPI/sys IPI/chip IPI/chipIPI/sys
--
1 0-15 2.409402 2.364108 2.383303 2.395091
0-31 6.028325 6.046075 6.08 6.073750
0-47 8.655178 8.644531 8.712830 8.724702
0-6311.629652 11.735953 12.088203 12.055979
0-7914.392321 14.729959 14.986701 14.973073
0-9512.604158 13.004034 17.528748 17.568095
2 0-1119.767753 13.719831 19.968606 20.024218
0-1276.744566 16.418854 22.898066 22.995110
0-1436.005699 19.174421 25.425622 25.417541
0-1595.649719 21.938836 27.952662 28.059603
0-1755.441410 24.109484 31.133915 31.127996
3 0-1915.318341 24.405322 33.999221 33.775354
0-2075.191382 26.449769 36.050161 35.867307
0-2235.102790 29.356943 39.544135 39.508169
0-2395.035295 31.933051 42.135075 42.071975
0-2554.969209 34.477367 44.655395 44.757074
4 0-2714.907652 35.887016 47.080545 47.318537
0-2874.839581 38.076137 50.464307 50.636219
0-3034.786031 40.881319 53.478684 53.310759
0-3194.743750 43.448424 56.388102 55.973969
0-3354.709936 45.623532 59.400930 58.926857
0-3514.681413 45.646151 62.035804 61.830057
[*] https://github.com/antonblanchard/ipistorm
Signed-off-by: Cédric Le Goater
---
arch/powerpc/sysdev/xive/xive-internal.h | 2 --
arch/powerpc/sysdev/xive/common.c| 39 ++--
2 files changed, 30 insertions(+), 11 deletions(-)
diff --git a/arch/powerpc/sysdev/xive/xive-internal.h
b/arch/powerpc/sysdev/xive/xive-internal.h
index 9cf57c722faa..b3a456fdd3a5 100644
--- a/arch/powerpc/sysdev/xive/xive-internal.h
+++ b/arch/powerpc/sysdev/xive/xive-internal.h
@@ -5,8 +5,6 @@
#ifndef __XIVE_INTERNAL_H
#define __XIVE_INTERNAL_H
-#define XIVE_IPI_HW_IRQ0 /* interrupt source # for IPIs */
-
/*
* A "disabled" interrupt should never fire, to catch problems
* we set its logical number to this
diff --git a/arch/powerpc/sysdev/xive/common.c
b/arch/powerpc/sysdev/xive/common.c
index 30034af4462b..a1e61a5cf927 100644
--- a/arch/powerpc/sysdev/xive/common.c
+++ b/arch/powerpc/sysdev/xive/common.c
@@ -65,8 +65,16 @@ static struct irq_domain *xive_irq_domain;
#ifdef CONFIG_SMP
static struct irq_domain *xive_ipi_irq_domain;
-/* The IPIs all use the same logical irq number */
-static u32 xive_ipi_irq;
+/* The IPIs use the same logical irq number when on the same chip */
+static struct xive_ipi_desc {
+ unsigned int irq;
+ char name[8]; /* enough bytes to fit IPI-XXX */
+} *xive_ipis;
+
+static unsigned int xive_ipi_cpu_to_irq(unsigned int cpu)
+{
+ return xive_ipis[cpu_to_node(cpu)].irq;
+}
#endif
/* Xive
[PATCH 0/5] powerpc/xive: Map one IPI interrupt per node
Hello,
ipistorm [*] can be used to benchmark the raw interrupt rate of an
interrupt controller by measuring the number of IPIs a system can
sustain. When applied to the XIVE interrupt controller of POWER9 and
POWER10 systems, a significant drop of the interrupt rate can be
observed when crossing the second node boundary.
This is due to the fact that a single IPI interrupt is used for all
CPUs of the system. The structure is shared and the cache line updates
impact greatly the traffic between nodes and the overall IPI
performance.
As a workaround, the impact can be reduced by deactivating the IRQ
lockup detector ("noirqdebug") which does a lot of accounting in the
Linux IRQ descriptor structure and is responsible for most of the
performance penalty.
As a fix, this proposal allocates an IPI interrupt per node, to be
shared by all CPUs of that node. It solves the scaling issue, the IRQ
lockup detector still has an impact but the XIVE interrupt rate scales
linearly. It also improves the "noirqdebug" case as showed in the
tables below.
* P9 DD2.2 - 2s * 64 threads
"noirqdebug"
Mint/sMint/s
chips cpus IPI/sys IPI/chip IPI/chipIPI/sys
--
1 0-15 4.984023 4.875405 4.996536 5.048892
0-3110.879164 10.544040 10.757632 11.037859
0-4715.345301 14.688764 14.926520 15.310053
0-6317.064907 17.066812 17.613416 17.874511
2 0-7911.768764 21.650749 22.689120 22.566508
0-9510.616812 26.878789 28.434703 28.320324
0-111 10.151693 31.397803 31.771773 32.388122
0-1279.948502 33.139336 34.875716 35.224548
* P10 DD1 - 4s (not homogeneous) 352 threads
"noirqdebug"
Mint/sMint/s
chips cpus IPI/sys IPI/chip IPI/chipIPI/sys
--
1 0-15 2.409402 2.364108 2.383303 2.395091
0-31 6.028325 6.046075 6.08 6.073750
0-47 8.655178 8.644531 8.712830 8.724702
0-6311.629652 11.735953 12.088203 12.055979
0-7914.392321 14.729959 14.986701 14.973073
0-9512.604158 13.004034 17.528748 17.568095
2 0-1119.767753 13.719831 19.968606 20.024218
0-1276.744566 16.418854 22.898066 22.995110
0-1436.005699 19.174421 25.425622 25.417541
0-1595.649719 21.938836 27.952662 28.059603
0-1755.441410 24.109484 31.133915 31.127996
3 0-1915.318341 24.405322 33.999221 33.775354
0-2075.191382 26.449769 36.050161 35.867307
0-2235.102790 29.356943 39.544135 39.508169
0-2395.035295 31.933051 42.135075 42.071975
0-2554.969209 34.477367 44.655395 44.757074
4 0-2714.907652 35.887016 47.080545 47.318537
0-2874.839581 38.076137 50.464307 50.636219
0-3034.786031 40.881319 53.478684 53.310759
0-3194.743750 43.448424 56.388102 55.973969
0-3354.709936 45.623532 59.400930 58.926857
0-3514.681413 45.646151 62.035804 61.830057
[*] https://github.com/antonblanchard/ipistorm
Thanks,
C.
Cédric Le Goater (5):
powerpc/xive: Use cpu_to_node() instead of ibm,chip-id property
powerpc/xive: Introduce an IPI interrupt domain
powerpc/xive: Remove useless check on XIVE_IPI_HW_IRQ
powerpc/xive: Simplify xive_core_debug_show()
powerpc/xive: Map one IPI interrupt per node
arch/powerpc/sysdev/xive/xive-internal.h | 2 -
arch/powerpc/sysdev/xive/common.c| 114 +++
2 files changed, 56 insertions(+), 60 deletions(-)
--
2.26.2
[PATCH 2/5] powerpc/xive: Introduce an IPI interrupt domain
The IPI interrupt is a special case of the XIVE IRQ domain. When
mapping and unmapping the interrupts in the Linux interrupt number
space, the HW interrupt number 0 (XIVE_IPI_HW_IRQ) is checked to
distinguish the IPI interrupt from other interrupts of the system.
Simplify the XIVE interrupt domain by introducing a specific domain
for the IPI.
Signed-off-by: Cédric Le Goater
---
arch/powerpc/sysdev/xive/common.c | 51 +--
1 file changed, 22 insertions(+), 29 deletions(-)
diff --git a/arch/powerpc/sysdev/xive/common.c
b/arch/powerpc/sysdev/xive/common.c
index 776871274b69..4aceac0f3046 100644
--- a/arch/powerpc/sysdev/xive/common.c
+++ b/arch/powerpc/sysdev/xive/common.c
@@ -63,6 +63,8 @@ static const struct xive_ops *xive_ops;
static struct irq_domain *xive_irq_domain;
#ifdef CONFIG_SMP
+static struct irq_domain *xive_ipi_irq_domain;
+
/* The IPIs all use the same logical irq number */
static u32 xive_ipi_irq;
#endif
@@ -1068,20 +1070,32 @@ static struct irq_chip xive_ipi_chip = {
.irq_unmask = xive_ipi_do_nothing,
};
+/*
+ * IPIs are marked per-cpu. We use separate HW interrupts under the
+ * hood but associated with the same "linux" interrupt
+ */
+static int xive_ipi_irq_domain_map(struct irq_domain *h, unsigned int virq,
+ irq_hw_number_t hw)
+{
+ irq_set_chip_and_handler(virq, _ipi_chip, handle_percpu_irq);
+ return 0;
+}
+
+static const struct irq_domain_ops xive_ipi_irq_domain_ops = {
+ .map = xive_ipi_irq_domain_map,
+};
+
static void __init xive_request_ipi(void)
{
unsigned int virq;
- /*
-* Initialization failed, move on, we might manage to
-* reach the point where we display our errors before
-* the system falls appart
-*/
- if (!xive_irq_domain)
+ xive_ipi_irq_domain = irq_domain_add_linear(NULL, 1,
+ _ipi_irq_domain_ops,
NULL);
+ if (WARN_ON(xive_ipi_irq_domain == NULL))
return;
/* Initialize it */
- virq = irq_create_mapping(xive_irq_domain, XIVE_IPI_HW_IRQ);
+ virq = irq_create_mapping(xive_ipi_irq_domain, XIVE_IPI_HW_IRQ);
xive_ipi_irq = virq;
WARN_ON(request_irq(virq, xive_muxed_ipi_action,
@@ -1179,19 +1193,6 @@ static int xive_irq_domain_map(struct irq_domain *h,
unsigned int virq,
*/
irq_clear_status_flags(virq, IRQ_LEVEL);
-#ifdef CONFIG_SMP
- /* IPIs are special and come up with HW number 0 */
- if (hw == XIVE_IPI_HW_IRQ) {
- /*
-* IPIs are marked per-cpu. We use separate HW interrupts under
-* the hood but associated with the same "linux" interrupt
-*/
- irq_set_chip_and_handler(virq, _ipi_chip,
-handle_percpu_irq);
- return 0;
- }
-#endif
-
rc = xive_irq_alloc_data(virq, hw);
if (rc)
return rc;
@@ -1203,15 +1204,7 @@ static int xive_irq_domain_map(struct irq_domain *h,
unsigned int virq,
static void xive_irq_domain_unmap(struct irq_domain *d, unsigned int virq)
{
- struct irq_data *data = irq_get_irq_data(virq);
- unsigned int hw_irq;
-
- /* XXX Assign BAD number */
- if (!data)
- return;
- hw_irq = (unsigned int)irqd_to_hwirq(data);
- if (hw_irq != XIVE_IPI_HW_IRQ)
- xive_irq_free_data(virq);
+ xive_irq_free_data(virq);
}
static int xive_irq_domain_xlate(struct irq_domain *h, struct device_node *ct,
--
2.26.2
Re: [PATCH v5 16/22] powerpc/syscall: Avoid stack frame in likely part of system_call_exception()
Le 09/02/2021 à 02:55, Nicholas Piggin a écrit : Excerpts from Christophe Leroy's message of February 9, 2021 1:10 am: When r3 is not modified, reload it from regs->orig_r3 to free volatile registers. This avoids a stack frame for the likely part of system_call_exception() This doesn't on my 64s build, but it does reduce one non volatile register save/restore. With quite a bit more register pressure reduction 64s can avoid the stack frame as well. The stack frame is not due to the registers because on PPC64 you have the redzone that you don't have on PPC32. As far as I can see, this is due to a call to .arch_local_irq_restore(). On ppc32 arch_local_irq_restore() is just a write to MSR. It's a cool trick but quite code and compiler specific so I don't know how worthwhile it is to keep considering we're calling out into random kernel C code after this. Maybe just keep it PPC32 specific for the moment, will have to do more tuning for 64 and we have other stuff to do there first. If you are happy to make it 32-bit only then I think we can leave without this, that's only one or two cycles won. Reviewed-by: Nicholas Piggin
Re: [PATCH v5 19/22] powerpc/syscall: Optimise checks in beginning of system_call_exception()
Le 09/02/2021 à 03:06, Nicholas Piggin a écrit :
Excerpts from Christophe Leroy's message of February 9, 2021 1:10 am:
Combine all tests of regs->msr into a single logical one.
Okay by me unless we choose to do the config option and put these all
under it. I think I would prefer that because sometimes the registers
are in a state you can't easily see what the values in the expression
were. In this case it doesn't matter so much because they should be in
regs in the interrupt frame.
Yes indeed. I reword the commit log and tell that.
Thanks,
Nick
Before the patch:
0: 81 6a 00 84 lwz r11,132(r10)
4: 90 6a 00 88 stw r3,136(r10)
8: 69 60 00 02 xorir0,r11,2
c: 54 00 ff fe rlwinm r0,r0,31,31,31
10: 0f 00 00 00 twnei r0,0
14: 69 63 40 00 xorir3,r11,16384
18: 54 63 97 fe rlwinm r3,r3,18,31,31
1c: 0f 03 00 00 twnei r3,0
20: 69 6b 80 00 xorir11,r11,32768
24: 55 6b 8f fe rlwinm r11,r11,17,31,31
28: 0f 0b 00 00 twnei r11,0
After the patch:
0: 81 6a 00 84 lwz r11,132(r10)
4: 90 6a 00 88 stw r3,136(r10)
8: 7d 6b 58 f8 not r11,r11
c: 71 6b c0 02 andi. r11,r11,49154
10: 0f 0b 00 00 twnei r11,0
6 cycles less on powerpc 8xx (328 => 322 cycles).
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/interrupt.c | 10 +++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index 55e1aa18cdb9..8c38e8c95be2 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -28,6 +28,7 @@ notrace long system_call_exception(long r3, long r4, long r5,
unsigned long r0, struct pt_regs *regs)
{
syscall_fn f;
+ unsigned long expected_msr;
regs->orig_gpr3 = r3;
@@ -39,10 +40,13 @@ notrace long system_call_exception(long r3, long r4, long r5,
trace_hardirqs_off(); /* finish reconciling */
+ expected_msr = MSR_PR;
if (!IS_ENABLED(CONFIG_BOOKE) && !IS_ENABLED(CONFIG_40x))
- BUG_ON(!(regs->msr & MSR_RI));
- BUG_ON(!(regs->msr & MSR_PR));
- BUG_ON(arch_irq_disabled_regs(regs));
+ expected_msr |= MSR_RI;
+ if (IS_ENABLED(CONFIG_PPC32))
+ expected_msr |= MSR_EE;
+ BUG_ON((regs->msr & expected_msr) ^ expected_msr);
+ BUG_ON(IS_ENABLED(CONFIG_PPC64) && arch_irq_disabled_regs(regs));
#ifdef CONFIG_PPC_PKEY
if (mmu_has_feature(MMU_FTR_PKEY)) {
--
2.25.0
RE: [PATCH v5 20/22] powerpc/syscall: Avoid storing 'current' in another pointer
From: Segher Boessenkool
> Sent: 09 February 2021 13:51
>
> On Tue, Feb 09, 2021 at 12:36:20PM +1000, Nicholas Piggin wrote:
> > What if you did this?
>
> > +static inline struct task_struct *get_current(void)
> > +{
> > + register struct task_struct *task asm ("r2");
> > +
> > + return task;
> > +}
>
> Local register asm variables are *only* guaranteed to live in that
> register as operands to an asm. See
>
> https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables
> ("The only supported use" etc.)
>
> You can do something like
>
> static inline struct task_struct *get_current(void)
> {
> register struct task_struct *task asm ("r2");
>
> asm("" : "+r"(task));
>
> return task;
> }
>
> which makes sure that "task" actually is in r2 at the point of that asm.
If "r2" always contains current (and is never assigned by the compiler)
why not use a global register variable for it?
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT,
UK
Registration No: 1397386 (Wales)
Re: [PATCH v5 18/22] powerpc/syscall: Remove FULL_REGS verification in system_call_exception
Le 09/02/2021 à 03:02, Nicholas Piggin a écrit : Excerpts from Christophe Leroy's message of February 9, 2021 1:10 am: For book3s/64, FULL_REGS() is 'true' at all time, so the test voids. For others, non volatile registers are saved inconditionally. So the verification is pointless. Should one fail to do it, it would anyway be caught by the CHECK_FULL_REGS() in copy_thread() as we have removed the special versions ppc_fork() and friends. null_syscall benchmark reduction 4 cycles (332 => 328 cycles) I wonder if we rather make a CONFIG option for a bunch of these simpler debug checks here (and also in interrupt exit, wrappers, etc) rather than remove them entirely. We can drop this patch if you prefer. Anyway, like book3s/64, once ppc32 also do interrupt entry/exit in C, FULL_REGS() will already return true. Christophe Thanks, Nick Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/interrupt.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c index 8fafca727b8b..55e1aa18cdb9 100644 --- a/arch/powerpc/kernel/interrupt.c +++ b/arch/powerpc/kernel/interrupt.c @@ -42,7 +42,6 @@ notrace long system_call_exception(long r3, long r4, long r5, if (!IS_ENABLED(CONFIG_BOOKE) && !IS_ENABLED(CONFIG_40x)) BUG_ON(!(regs->msr & MSR_RI)); BUG_ON(!(regs->msr & MSR_PR)); - BUG_ON(!FULL_REGS(regs)); BUG_ON(arch_irq_disabled_regs(regs)); #ifdef CONFIG_PPC_PKEY -- 2.25.0
[PATCH] powerpc/64: Fix stack trace not displaying final frame
In commit bf13718bc57a ("powerpc: show registers when unwinding
interrupt frames") we changed our stack dumping logic to show the full
registers whenever we find an interrupt frame on the stack.
However we didn't notice that on 64-bit this doesn't show the final
frame, ie. the interrupt that brought us in from userspace, whereas on
32-bit it does.
That is due to confusion about the size of that last frame. The code
in show_stack() calls validate_sp(), passing it STACK_INT_FRAME_SIZE
to check the sp is at least that far below the top of the stack.
However on 64-bit that size is too large for the final frame, because
it includes the red zone, but we don't allocate a red zone for the
first frame.
So add a new define that encodes the correct size for 32-bit and
64-bit, and use it in show_stack().
This results in the full trace being shown on 64-bit, eg:
sysrq: Trigger a crash
Kernel panic - not syncing: sysrq triggered crash
CPU: 0 PID: 83 Comm: sh Not tainted
5.11.0-rc2-gcc-8.2.0-00188-g571abcb96b10-dirty #649
Call Trace:
[ca1c3ac0] [c0897b70] dump_stack+0xc4/0x114 (unreliable)
[ca1c3b00] [c014334c] panic+0x178/0x41c
[ca1c3ba0] [c094e600] sysrq_handle_crash+0x40/0x50
[ca1c3c00] [c094ef98] __handle_sysrq+0xd8/0x210
[ca1c3ca0] [c094f820] write_sysrq_trigger+0x100/0x188
[ca1c3ce0] [c05559dc] proc_reg_write+0x10c/0x1b0
[ca1c3d10] [c0479950] vfs_write+0xf0/0x360
[ca1c3d60] [c0479d9c] ksys_write+0x7c/0x140
[ca1c3db0] [c002bf5c] system_call_exception+0x19c/0x2c0
[ca1c3e10] [c000d35c] system_call_common+0xec/0x278
--- interrupt: c00 at 0x7fff9fbab428
NIP: 7fff9fbab428 LR: 1000b724 CTR:
REGS: ca1c3e80 TRAP: 0c00 Not tainted
(5.11.0-rc2-gcc-8.2.0-00188-g571abcb96b10-dirty)
MSR: 9280f033 CR: 22002884
XER:
IRQMASK: 0
GPR00: 0004 7fffc3cb8960 7fff9fc59900 0001
GPR04: 2a4b32d0 0002 0063 0063
GPR08: 2a4b32d0
GPR12: 7fff9fcca9a0
GPR16: 100b8fd0
GPR20: 2a4b3485 100b8f90
GPR24: 2a4b0440 100e77b8 0020 2a4b32d0
GPR28: 0001 0002 2a4b32d0 0001
NIP [7fff9fbab428] 0x7fff9fbab428
LR [1000b724] 0x1000b724
--- interrupt: c00
Signed-off-by: Michael Ellerman
---
arch/powerpc/include/asm/ptrace.h | 3 +++
arch/powerpc/kernel/asm-offsets.c | 2 +-
arch/powerpc/kernel/process.c | 2 +-
3 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/ptrace.h
b/arch/powerpc/include/asm/ptrace.h
index 58f9dc060a7b..8236c5e749e4 100644
--- a/arch/powerpc/include/asm/ptrace.h
+++ b/arch/powerpc/include/asm/ptrace.h
@@ -70,6 +70,9 @@ struct pt_regs
};
#endif
+
+#define STACK_FRAME_WITH_PT_REGS (STACK_FRAME_OVERHEAD + sizeof(struct
pt_regs))
+
#ifdef __powerpc64__
/*
diff --git a/arch/powerpc/kernel/asm-offsets.c
b/arch/powerpc/kernel/asm-offsets.c
index 31edd9bbce75..6109496e5fdf 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -308,7 +308,7 @@ int main(void)
/* Interrupt register frame */
DEFINE(INT_FRAME_SIZE, STACK_INT_FRAME_SIZE);
- DEFINE(SWITCH_FRAME_SIZE, STACK_FRAME_OVERHEAD + sizeof(struct
pt_regs));
+ DEFINE(SWITCH_FRAME_SIZE, STACK_FRAME_WITH_PT_REGS);
STACK_PT_REGS_OFFSET(GPR0, gpr[0]);
STACK_PT_REGS_OFFSET(GPR1, gpr[1]);
STACK_PT_REGS_OFFSET(GPR2, gpr[2]);
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index e296440e9d16..924d023dad0a 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -2179,7 +2179,7 @@ void show_stack(struct task_struct *tsk, unsigned long
*stack,
* See if this is an exception frame.
* We look for the "regshere" marker in the current frame.
*/
- if (validate_sp(sp, tsk, STACK_INT_FRAME_SIZE)
+ if (validate_sp(sp, tsk, STACK_FRAME_WITH_PT_REGS)
&& stack[STACK_FRAME_MARKER] == STACK_FRAME_REGS_MARKER) {
struct pt_regs *regs = (struct pt_regs *)
(sp + STACK_FRAME_OVERHEAD);
--
2.25.1
Re: [PATCH v3 1/8] powerpc/uaccess: Add unsafe_copy_from_user
Le 19/01/2021 à 03:11, Michael Ellerman a écrit : "Christopher M. Riedl" writes: On Mon Jan 11, 2021 at 7:22 AM CST, Christophe Leroy wrote: Le 09/01/2021 à 04:25, Christopher M. Riedl a écrit : Implement raw_copy_from_user_allowed() which assumes that userspace read access is open. Use this new function to implement raw_copy_from_user(). Finally, wrap the new function to follow the usual "unsafe_" convention of taking a label argument. I think there is no point implementing raw_copy_from_user_allowed(), see https://github.com/linuxppc/linux/commit/4b842e4e25b1 and https://patchwork.ozlabs.org/project/linuxppc-dev/patch/8c74fc9ce8131cabb10b3e95dc0e430f396ee83e.1610369143.git.christophe.le...@csgroup.eu/ You should simply do: #define unsafe_copy_from_user(d, s, l, e) \ unsafe_op_wrap(__copy_tofrom_user((__force void __user *)d, s, l), e) I gave this a try and the signal ops decreased by ~8K. Now, to be honest, I am not sure what an "acceptable" benchmark number here actually is - so maybe this is ok? Same loss with both radix and hash: | | hash | radix | | | -- | -- | | linuxppc/next| 118693 | 133296 | | linuxppc/next w/o KUAP+KUEP | 228911 | 228654 | | unsafe-signal64 | 200480 | 234067 | | unsafe-signal64 (__copy_tofrom_user) | 192467 | 225119 | To put this into perspective, prior to KUAP and uaccess flush, signal performance in this benchmark was ~290K on hash. If I'm doing the math right 8K is ~4% of the best number. It seems like 4% is worth a few lines of code to handle these constant sizes. It's not like we have performance to throw away. Or, we should chase down where the call sites are that are doing small constant copies with copy_to/from_user() and change them to use get/put_user(). I have built pmac32_defconfig and ppc64_defconfig with a BUILD_BUG_ON(__builtin_constant_p(n) && (n == 1 || n == 2 || n == 4 || n == 8) in raw_copy_from_user() and raw_copy_to_user(): On pmac32_defconfig, no hit. On ppc64_defconfig, two hits: - copies of sigset_t in signal64. This problem is only on linux/next. On next-test we don't have this problem anymore thanks to the series from Christopher. - in pkey_set() in arch/powerpc/kernel/ptrace/ptrace-view.c, in the copy of new_amr. This is not a hot path I think so we can live with it. Christophe
[PATCH v2 1/3] powerpc/uaccess: get rid of small constant size cases in raw_copy_{to,from}_user()
Copied from commit 4b842e4e25b1 ("x86: get rid of small
constant size cases in raw_copy_{to,from}_user()")
Very few call sites where that would be triggered remain, and none
of those is anywhere near hot enough to bother.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/uaccess.h | 41 --
1 file changed, 41 deletions(-)
diff --git a/arch/powerpc/include/asm/uaccess.h
b/arch/powerpc/include/asm/uaccess.h
index 93d33f7e8b53..a4d2569173ac 100644
--- a/arch/powerpc/include/asm/uaccess.h
+++ b/arch/powerpc/include/asm/uaccess.h
@@ -398,26 +398,6 @@ static inline unsigned long raw_copy_from_user(void *to,
const void __user *from, unsigned long n)
{
unsigned long ret;
- if (__builtin_constant_p(n) && (n <= 8)) {
- ret = 1;
-
- switch (n) {
- case 1:
- __get_user_size(*(u8 *)to, from, 1, ret);
- break;
- case 2:
- __get_user_size(*(u16 *)to, from, 2, ret);
- break;
- case 4:
- __get_user_size(*(u32 *)to, from, 4, ret);
- break;
- case 8:
- __get_user_size(*(u64 *)to, from, 8, ret);
- break;
- }
- if (ret == 0)
- return 0;
- }
allow_read_from_user(from, n);
ret = __copy_tofrom_user((__force void __user *)to, from, n);
@@ -428,27 +408,6 @@ static inline unsigned long raw_copy_from_user(void *to,
static inline unsigned long
raw_copy_to_user_allowed(void __user *to, const void *from, unsigned long n)
{
- if (__builtin_constant_p(n) && (n <= 8)) {
- unsigned long ret = 1;
-
- switch (n) {
- case 1:
- __put_user_size_allowed(*(u8 *)from, (u8 __user *)to,
1, ret);
- break;
- case 2:
- __put_user_size_allowed(*(u16 *)from, (u16 __user *)to,
2, ret);
- break;
- case 4:
- __put_user_size_allowed(*(u32 *)from, (u32 __user *)to,
4, ret);
- break;
- case 8:
- __put_user_size_allowed(*(u64 *)from, (u64 __user *)to,
8, ret);
- break;
- }
- if (ret == 0)
- return 0;
- }
-
return __copy_tofrom_user(to, (__force const void __user *)from, n);
}
--
2.25.0
[PATCH v2 2/3] powerpc/uaccess: Merge __put_user_size_allowed() into __put_user_size()
__put_user_size_allowed() is only called from __put_user_size() now.
Merge them together.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/uaccess.h | 10 +++---
1 file changed, 3 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/include/asm/uaccess.h
b/arch/powerpc/include/asm/uaccess.h
index a4d2569173ac..2fb1d95f10d3 100644
--- a/arch/powerpc/include/asm/uaccess.h
+++ b/arch/powerpc/include/asm/uaccess.h
@@ -108,22 +108,18 @@ static inline bool __access_ok(unsigned long addr,
unsigned long size)
extern long __put_user_bad(void);
-#define __put_user_size_allowed(x, ptr, size, retval) \
+#define __put_user_size(x, ptr, size, retval) \
do { \
__label__ __pu_failed; \
\
retval = 0; \
+ allow_write_to_user(ptr, size); \
__put_user_size_goto(x, ptr, size, __pu_failed);\
+ prevent_write_to_user(ptr, size); \
break; \
\
__pu_failed: \
retval = -EFAULT; \
-} while (0)
-
-#define __put_user_size(x, ptr, size, retval) \
-do { \
- allow_write_to_user(ptr, size); \
- __put_user_size_allowed(x, ptr, size, retval); \
prevent_write_to_user(ptr, size); \
} while (0)
--
2.25.0
[PATCH v2 3/3] powerpc/uaccess: Merge raw_copy_to_user_allowed() into raw_copy_to_user()
Since commit 17bc43367fc2 ("powerpc/uaccess: Implement
unsafe_copy_to_user() as a simple loop"), raw_copy_to_user_allowed()
is only used by raw_copy_to_user().
Merge raw_copy_to_user_allowed() into raw_copy_to_user().
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/uaccess.h | 8 +---
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/arch/powerpc/include/asm/uaccess.h
b/arch/powerpc/include/asm/uaccess.h
index 2fb1d95f10d3..33b2de642120 100644
--- a/arch/powerpc/include/asm/uaccess.h
+++ b/arch/powerpc/include/asm/uaccess.h
@@ -401,19 +401,13 @@ static inline unsigned long raw_copy_from_user(void *to,
return ret;
}
-static inline unsigned long
-raw_copy_to_user_allowed(void __user *to, const void *from, unsigned long n)
-{
- return __copy_tofrom_user(to, (__force const void __user *)from, n);
-}
-
static inline unsigned long
raw_copy_to_user(void __user *to, const void *from, unsigned long n)
{
unsigned long ret;
allow_write_to_user(to, n);
- ret = raw_copy_to_user_allowed(to, from, n);
+ ret = __copy_tofrom_user(to, (__force const void __user *)from, n);
prevent_write_to_user(to, n);
return ret;
}
--
2.25.0
Re: [PATCH v5 20/22] powerpc/syscall: Avoid storing 'current' in another pointer
On Tue, Feb 09, 2021 at 12:36:20PM +1000, Nicholas Piggin wrote:
> What if you did this?
> +static inline struct task_struct *get_current(void)
> +{
> + register struct task_struct *task asm ("r2");
> +
> + return task;
> +}
Local register asm variables are *only* guaranteed to live in that
register as operands to an asm. See
https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables
("The only supported use" etc.)
You can do something like
static inline struct task_struct *get_current(void)
{
register struct task_struct *task asm ("r2");
asm("" : "+r"(task));
return task;
}
which makes sure that "task" actually is in r2 at the point of that asm.
Segher
Re: [PATCH] tools/perf: Fix powerpc gap between kernel end and module start
Em Wed, Feb 03, 2021 at 12:31:48PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Feb 02, 2021 at 04:02:36PM +0530, Athira Rajeev escreveu:
> >
> >
> > On 18-Jan-2021, at 3:51 PM, kajoljain wrote:
> >
> >
> >
> > On 1/12/21 3:08 PM, Jiri Olsa wrote:
> >
> > On Mon, Dec 28, 2020 at 09:14:14PM -0500, Athira Rajeev wrote:
> >
> > SNIP
> >
> >
> > c2799370 b backtrace_flag
> > c2799378 B radix_tree_node_cachep
> > c2799380 B __bss_stop
> > c27a B _end
> > c0080389 t icmp_checkentry [ip_tables]
> > c00803890038 t ipt_alloc_initial_table [ip_tables]
> > c00803890468 T ipt_do_table [ip_tables]
> > c00803890de8 T ipt_unregister_table_pre_exit
> > [ip_tables]
> > ...
> >
> > Perf calls function symbols__fixup_end() which sets the end of
> > symbol
> > to 0xc0080389, which is the next address and this is the
> > start
> > address of first module (icmp_checkentry in above) which will
> > make
> > the
> > huge symbol size of 0x8010f.
> >
> > After symbols__fixup_end:
> > symbols__fixup_end: sym->name: _end, sym->start:
> > 0xc27a,
> > sym->end: 0xc0080389
> >
> > On powerpc, kernel text segment is located at 0xc000
> > whereas the modules are located at very high memory addresses,
> > 0xc0080xxx. Since the gap between end of kernel text
> > segment
> > and beginning of first module's address is high, histogram
> > allocation
> > using calloc fails.
> >
> > Fix this by detecting the kernel's last symbol and limiting
> > the range of last kernel symbol to pagesize.
> >
> >
> > Patch looks good to me.
> >
> > Tested-By: Kajol Jain
> >
> > Thanks,
> > Kajol Jain
> >
> >
> > Signed-off-by: Athira Rajeev
> >
> >
> > I can't test, but since the same approach works for arm and s390,
> > this also looks ok
> >
> > Acked-by: Jiri Olsa
> >
> > thanks,
> > jirka
> >
> >
> > Hi Arnaldo,
> >
> > Can you please help review this patch and merge if this looks good..
>
> Thanks, collected the Tested-by from Kajol and the Acked-by from Jiri
> and applied to my local tree for testing, then up to my perf/core
> branch.
Had to apply this on top.
- Arnaldo
commit 0f000f9c89182950cd3500226729977251529364
Author: Arnaldo Carvalho de Melo
Date: Tue Feb 9 09:41:21 2021 -0300
perf powerpc: Fix printf conversion specifier for IP addresses
We need to use "%#" PRIx64 for u64 values, not "%lx", fixing this build
problem on powerpc 32-bit:
7213.69 ubuntu:18.04-x-powerpc: FAIL powerpc-linux-gnu-gcc
(Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
arch/powerpc/util/machine.c: In function 'arch__symbols__fixup_end':
arch/powerpc/util/machine.c:23:12: error: format '%lx' expects argument
of type 'long unsigned int', but argument 6 has type 'u64 {aka long long
unsigned int}' [-Werror=format=]
pr_debug4("%s sym:%s end:%#lx\n", __func__, p->name, p->end);
^
/git/linux/tools/perf/util/debug.h:18:21: note: in definition of macro
'pr_fmt'
#define pr_fmt(fmt) fmt
^~~
/git/linux/tools/perf/util/debug.h:33:29: note: in expansion of macro
'pr_debugN'
#define pr_debug4(fmt, ...) pr_debugN(4, pr_fmt(fmt), ##__VA_ARGS__)
^
/git/linux/tools/perf/util/debug.h:33:42: note: in expansion of macro
'pr_fmt'
#define pr_debug4(fmt, ...) pr_debugN(4, pr_fmt(fmt), ##__VA_ARGS__)
^~
arch/powerpc/util/machine.c:23:2: note: in expansion of macro
'pr_debug4'
pr_debug4("%s sym:%s end:%#lx\n", __func__, p->name, p->end);
^
cc1: all warnings being treated as errors
/git/linux/tools/build/Makefile.build:139: recipe for target 'util'
failed
make[5]: *** [util] Error 2
/git/linux/tools/build/Makefile.build:139: recipe for target 'powerpc'
failed
make[4]: *** [powerpc] Error 2
/git/linux/tools/build/Makefile.build:139: recipe for target 'arch'
failed
make[3]: *** [arch] Error 2
7330.47 ubuntu:18.04-x-powerpc64 : Ok powerpc64-linux-gnu-gcc
(Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Fixes: 557c3eadb7712741 ("perf powerpc: Fix gap between kernel end and
module start")
Cc: Athira Rajeev
Cc: Jiri Olsa
Cc: Kajol Jain
Cc: Madhavan Srinivasan
Signed-off-by: Arnaldo Carvalho de Melo
diff --git
[PATCH 3/3] powerpc/time: Remove get_tbl()
There are no more users of get_tbl(). Remove it.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/vdso/timebase.h | 6 --
1 file changed, 6 deletions(-)
diff --git a/arch/powerpc/include/asm/vdso/timebase.h
b/arch/powerpc/include/asm/vdso/timebase.h
index 881f655caa0a..891c9d5eaabe 100644
--- a/arch/powerpc/include/asm/vdso/timebase.h
+++ b/arch/powerpc/include/asm/vdso/timebase.h
@@ -43,12 +43,6 @@
#define mttbl(v) asm volatile("mttbl %0":: "r"(v))
#define mttbu(v) asm volatile("mttbu %0":: "r"(v))
-/* For compatibility, get_tbl() is defined as get_tb() on ppc64 */
-static inline unsigned long get_tbl(void)
-{
- return mftb();
-}
-
static __always_inline u64 get_tb(void)
{
unsigned int tbhi, tblo, tbhi2;
--
2.25.0
[PATCH 1/3] spi: mpc52xx: Avoid using get_tbl()
get_tbl() is confusing as it returns the content TBL register
on PPC32 but the concatenation of TBL and TBU on PPC64.
Use mftb() instead.
This will allow the removal of get_tbl() in a following patch.
Signed-off-by: Christophe Leroy
---
drivers/spi/spi-mpc52xx.c | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/spi/spi-mpc52xx.c b/drivers/spi/spi-mpc52xx.c
index ef2f24420460..e6a30f232370 100644
--- a/drivers/spi/spi-mpc52xx.c
+++ b/drivers/spi/spi-mpc52xx.c
@@ -120,7 +120,7 @@ static void mpc52xx_spi_start_transfer(struct mpc52xx_spi
*ms)
ms->cs_change = ms->transfer->cs_change;
/* Write out the first byte */
- ms->wcol_tx_timestamp = get_tbl();
+ ms->wcol_tx_timestamp = mftb();
if (ms->tx_buf)
out_8(ms->regs + SPI_DATA, *ms->tx_buf++);
else
@@ -221,8 +221,8 @@ static int mpc52xx_spi_fsmstate_transfer(int irq, struct
mpc52xx_spi *ms,
* but it can also be worked around simply by retrying the
* transfer which is what we do here. */
ms->wcol_count++;
- ms->wcol_ticks += get_tbl() - ms->wcol_tx_timestamp;
- ms->wcol_tx_timestamp = get_tbl();
+ ms->wcol_ticks += mftb() - ms->wcol_tx_timestamp;
+ ms->wcol_tx_timestamp = mftb();
data = 0;
if (ms->tx_buf)
data = *(ms->tx_buf - 1);
@@ -247,14 +247,14 @@ static int mpc52xx_spi_fsmstate_transfer(int irq, struct
mpc52xx_spi *ms,
/* Is the transfer complete? */
ms->len--;
if (ms->len == 0) {
- ms->timestamp = get_tbl();
+ ms->timestamp = mftb();
ms->timestamp += ms->transfer->delay_usecs * tb_ticks_per_usec;
ms->state = mpc52xx_spi_fsmstate_wait;
return FSM_CONTINUE;
}
/* Write out the next byte */
- ms->wcol_tx_timestamp = get_tbl();
+ ms->wcol_tx_timestamp = mftb();
if (ms->tx_buf)
out_8(ms->regs + SPI_DATA, *ms->tx_buf++);
else
@@ -276,7 +276,7 @@ mpc52xx_spi_fsmstate_wait(int irq, struct mpc52xx_spi *ms,
u8 status, u8 data)
dev_err(>master->dev, "spurious irq, status=0x%.2x\n",
status);
- if (((int)get_tbl()) - ms->timestamp < 0)
+ if (((int)mftb()) - ms->timestamp < 0)
return FSM_POLL;
ms->message->actual_length += ms->transfer->len;
--
2.25.0
[PATCH 2/3] powerpc/time: Avoid using get_tbl()
get_tbl() is confusing as it returns the content TBL register
on PPC32 but the concatenation of TBL and TBU on PPC64.
Use mftb() instead.
This will allow the removal of get_tbl() in a following patch.
Signed-off-by: Christophe Leroy
---
arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c
b/arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c
index 05e19470d523..b91ebebd9ff2 100644
--- a/arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c
+++ b/arch/powerpc/platforms/52xx/mpc52xx_lpbfifo.c
@@ -229,7 +229,7 @@ static irqreturn_t mpc52xx_lpbfifo_irq(int irq, void
*dev_id)
int dma, write, poll_dma;
spin_lock_irqsave(, flags);
- ts = get_tbl();
+ ts = mftb();
req = lpbfifo.req;
if (!req) {
@@ -307,7 +307,7 @@ static irqreturn_t mpc52xx_lpbfifo_irq(int irq, void
*dev_id)
if (irq != 0) /* don't increment on polled case */
req->irq_count++;
- req->irq_ticks += get_tbl() - ts;
+ req->irq_ticks += mftb() - ts;
spin_unlock_irqrestore(, flags);
/* Spinlock is released; it is now safe to call the callback */
@@ -330,7 +330,7 @@ static irqreturn_t mpc52xx_lpbfifo_bcom_irq(int irq, void
*dev_id)
u32 ts;
spin_lock_irqsave(, flags);
- ts = get_tbl();
+ ts = mftb();
req = lpbfifo.req;
if (!req || (req->flags & MPC52XX_LPBFIFO_FLAG_NO_DMA)) {
@@ -361,7 +361,7 @@ static irqreturn_t mpc52xx_lpbfifo_bcom_irq(int irq, void
*dev_id)
lpbfifo.req = NULL;
/* Release the lock before calling out to the callback. */
- req->irq_ticks += get_tbl() - ts;
+ req->irq_ticks += mftb() - ts;
spin_unlock_irqrestore(, flags);
if (req->callback)
--
2.25.0
linux-next: build failure after merge of the powerpc tree
Hi all, After merging the powerpc tree, today's linux-next build (powerpc allyesconfig) failed like this: arch/powerpc/kernel/head_64.o:(__ftr_alt_97+0x0): relocation truncated to fit: R_PPC64_REL24 (OPD) against symbol `do_page_fault' defined in .opd section in arch/powerpc/mm/fault.o arch/powerpc/kernel/head_64.o:(__ftr_alt_97+0x8): relocation truncated to fit: R_PPC64_REL24 (OPD) against symbol `do_page_fault' defined in .opd section in arch/powerpc/mm/fault.o arch/powerpc/kernel/head_64.o:(__ftr_alt_97+0x28): relocation truncated to fit: R_PPC64_REL24 (OPD) against symbol `unknown_exception' defined in .opd section in arch/powerpc/kernel/traps.o Not sure exactly which commit caused this, but it is most likkely part of a series in the powerpc tree. I have left the allyesconfig build broken for today. -- Cheers, Stephen Rothwell pgph9Zs9WiVU6.pgp Description: OpenPGP digital signature
[RFC PATCH v1 40/41] powerpc/8xx: Create C version of kuap save/restore/check helpers
In preparation of porting PPC32 to C syscall entry/exit,
create C version of kuap_save_and_lock() and kuap_user_restore() and
kuap_kernel_restore() and kuap_check() and kuap_get_and_check() on 8xx.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/nohash/32/kup-8xx.h | 33
1 file changed, 33 insertions(+)
diff --git a/arch/powerpc/include/asm/nohash/32/kup-8xx.h
b/arch/powerpc/include/asm/nohash/32/kup-8xx.h
index 17a4a616436f..6f191fe08a2e 100644
--- a/arch/powerpc/include/asm/nohash/32/kup-8xx.h
+++ b/arch/powerpc/include/asm/nohash/32/kup-8xx.h
@@ -34,6 +34,39 @@
#include
+static inline void kuap_save_and_lock(struct pt_regs *regs)
+{
+ regs->kuap = mfspr(SPRN_MD_AP);
+ mtspr(SPRN_MD_AP, MD_APG_KUAP);
+}
+
+static inline void kuap_user_restore(struct pt_regs *regs)
+{
+}
+
+static inline void kuap_kernel_restore(struct pt_regs *regs, unsigned long
kuap)
+{
+ mtspr(SPRN_MD_AP, regs->kuap);
+}
+
+static inline void kuap_check(void)
+{
+ if (!IS_ENABLED(CONFIG_PPC_KUAP_DEBUG))
+ return;
+
+ WARN_ON_ONCE(mfspr(SPRN_MD_AP) >> 16 != MD_APG_KUAP >> 16);
+}
+
+static inline unsigned long kuap_get_and_check(void)
+{
+ unsigned long kuap = mfspr(SPRN_MD_AP);
+
+ if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG))
+ WARN_ON_ONCE(kuap >> 16 != MD_APG_KUAP >> 16);
+
+ return kuap;
+}
+
static inline void allow_user_access(void __user *to, const void __user *from,
unsigned long size, unsigned long dir)
{
--
2.25.0
[RFC PATCH v1 41/41] powerpc/32: Manage KUAP in C
Move all KUAP management in C.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/book3s/32/kup.h | 50 +---
arch/powerpc/include/asm/interrupt.h | 2 +
arch/powerpc/include/asm/kup.h | 19 +---
arch/powerpc/include/asm/nohash/32/kup-8xx.h | 25 +-
arch/powerpc/kernel/entry_32.S | 6 ---
arch/powerpc/kernel/interrupt.c | 19 ++--
arch/powerpc/kernel/process.c| 3 ++
7 files changed, 12 insertions(+), 112 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/32/kup.h
b/arch/powerpc/include/asm/book3s/32/kup.h
index c9d6c28bcd10..27991e0d2cf9 100644
--- a/arch/powerpc/include/asm/book3s/32/kup.h
+++ b/arch/powerpc/include/asm/book3s/32/kup.h
@@ -5,55 +5,7 @@
#include
#include
-#ifdef __ASSEMBLY__
-
-#ifdef CONFIG_PPC_KUAP
-
-.macro kuap_update_sr gpr1, gpr2, gpr3/* NEVER use r0 as gpr2 due to
addis */
-101: mtsrin \gpr1, \gpr2
- addi\gpr1, \gpr1, 0x111 /* next VSID */
- rlwinm \gpr1, \gpr1, 0, 0xf0ff /* clear VSID overflow */
- addis \gpr2, \gpr2, 0x1000/* address of next segment */
- cmplw \gpr2, \gpr3
- blt-101b
- isync
-.endm
-
-.macro kuap_save_and_lock sp, thread, gpr1, gpr2, gpr3
- lwz \gpr2, KUAP(\thread)
- rlwinm. \gpr3, \gpr2, 28, 0xf000
- stw \gpr2, STACK_REGS_KUAP(\sp)
- beq+102f
- li \gpr1, 0
- stw \gpr1, KUAP(\thread)
- mfsrin \gpr1, \gpr2
- oris\gpr1, \gpr1, SR_KS@h /* set Ks */
- kuap_update_sr \gpr1, \gpr2, \gpr3
-102:
-.endm
-
-.macro kuap_restoresp, current, gpr1, gpr2, gpr3
- lwz \gpr2, STACK_REGS_KUAP(\sp)
- rlwinm. \gpr3, \gpr2, 28, 0xf000
- stw \gpr2, THREAD + KUAP(\current)
- beq+102f
- mfsrin \gpr1, \gpr2
- rlwinm \gpr1, \gpr1, 0, ~SR_KS /* Clear Ks */
- kuap_update_sr \gpr1, \gpr2, \gpr3
-102:
-.endm
-
-.macro kuap_check current, gpr
-#ifdef CONFIG_PPC_KUAP_DEBUG
- lwz \gpr, THREAD + KUAP(\current)
-999: twnei \gpr, 0
- EMIT_BUG_ENTRY 999b, __FILE__, __LINE__, (BUGFLAG_WARNING |
BUGFLAG_ONCE)
-#endif
-.endm
-
-#endif /* CONFIG_PPC_KUAP */
-
-#else /* !__ASSEMBLY__ */
+#ifndef __ASSEMBLY__
#ifdef CONFIG_PPC_KUAP
diff --git a/arch/powerpc/include/asm/interrupt.h
b/arch/powerpc/include/asm/interrupt.h
index e5e04e962165..0e38185d4d90 100644
--- a/arch/powerpc/include/asm/interrupt.h
+++ b/arch/powerpc/include/asm/interrupt.h
@@ -24,6 +24,8 @@ static inline void interrupt_enter_prepare(struct pt_regs
*regs, struct interrup
kuep_lock();
current->thread.regs = regs;
account_cpu_user_entry();
+ } else {
+ kuap_save_and_lock(regs);
}
#endif
/*
diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h
index b7efa46b3109..f463256a57a6 100644
--- a/arch/powerpc/include/asm/kup.h
+++ b/arch/powerpc/include/asm/kup.h
@@ -26,24 +26,7 @@
#include
#endif
-#ifdef __ASSEMBLY__
-#ifndef CONFIG_PPC_KUAP
-.macro kuap_save_and_lock sp, thread, gpr1, gpr2, gpr3
-.endm
-
-.macro kuap_restoresp, current, gpr1, gpr2, gpr3
-.endm
-
-.macro kuap_check current, gpr
-.endm
-
-.macro kuap_check_amr gpr1, gpr2
-.endm
-
-#endif
-
-#else /* !__ASSEMBLY__ */
-
+#ifndef __ASSEMBLY__
extern bool disable_kuep;
extern bool disable_kuap;
diff --git a/arch/powerpc/include/asm/nohash/32/kup-8xx.h
b/arch/powerpc/include/asm/nohash/32/kup-8xx.h
index 6f191fe08a2e..add0413f31d4 100644
--- a/arch/powerpc/include/asm/nohash/32/kup-8xx.h
+++ b/arch/powerpc/include/asm/nohash/32/kup-8xx.h
@@ -7,30 +7,7 @@
#ifdef CONFIG_PPC_KUAP
-#ifdef __ASSEMBLY__
-
-.macro kuap_save_and_lock sp, thread, gpr1, gpr2, gpr3
- lis \gpr2, MD_APG_KUAP@h/* only APG0 and APG1 are used */
- mfspr \gpr1, SPRN_MD_AP
- mtspr SPRN_MD_AP, \gpr2
- stw \gpr1, STACK_REGS_KUAP(\sp)
-.endm
-
-.macro kuap_restoresp, current, gpr1, gpr2, gpr3
- lwz \gpr1, STACK_REGS_KUAP(\sp)
- mtspr SPRN_MD_AP, \gpr1
-.endm
-
-.macro kuap_check current, gpr
-#ifdef CONFIG_PPC_KUAP_DEBUG
- mfspr \gpr, SPRN_MD_AP
- rlwinm \gpr, \gpr, 16, 0x
-999: twnei \gpr, MD_APG_KUAP@h
- EMIT_BUG_ENTRY 999b, __FILE__, __LINE__, (BUGFLAG_WARNING |
BUGFLAG_ONCE)
-#endif
-.endm
-
-#else /* !__ASSEMBLY__ */
+#ifndef __ASSEMBLY__
#include
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 3ae790a362d1..2f3b41c8b23f 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -59,13 +59,11 @@
.globl prepare_transfer_to_handler
prepare_transfer_to_handler:
andi. r0,r9,MSR_PR
- addir12, r2, THREAD
bnelr
/* if from kernel, check interrupted DOZE/NAP mode and
[RFC PATCH v1 39/41] powerpc/32s: Create C version of kuap save/restore/check helpers
In preparation of porting PPC32 to C syscall entry/exit,
create C version of kuap_save_and_lock() and kuap_user_restore() and
kuap_kernel_restore() and kuap_check() and kuap_get_and_check()
on book3s/32.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/book3s/32/kup.h | 45
1 file changed, 45 insertions(+)
diff --git a/arch/powerpc/include/asm/book3s/32/kup.h
b/arch/powerpc/include/asm/book3s/32/kup.h
index b97ea60f6fa3..c9d6c28bcd10 100644
--- a/arch/powerpc/include/asm/book3s/32/kup.h
+++ b/arch/powerpc/include/asm/book3s/32/kup.h
@@ -72,6 +72,51 @@ static inline void kuap_update_sr(u32 sr, u32 addr, u32 end)
isync();/* Context sync required after mtsr() */
}
+static inline void kuap_save_and_lock(struct pt_regs *regs)
+{
+ unsigned long kuap = current->thread.kuap;
+ u32 addr = kuap & 0xf000;
+ u32 end = kuap << 28;
+
+ regs->kuap = kuap;
+ if (unlikely(!kuap))
+ return;
+
+ current->thread.kuap = 0;
+ kuap_update_sr(mfsr(addr) | SR_KS, addr, end); /* Set Ks */
+}
+
+static inline void kuap_user_restore(struct pt_regs *regs)
+{
+}
+
+static inline void kuap_kernel_restore(struct pt_regs *regs, unsigned long
kuap)
+{
+ u32 addr = regs->kuap & 0xf000;
+ u32 end = regs->kuap << 28;
+
+ current->thread.kuap = regs->kuap;
+
+ if (unlikely(regs->kuap == kuap))
+ return;
+
+ kuap_update_sr(mfsr(addr) & ~SR_KS, addr, end); /* Clear Ks */
+}
+
+static inline unsigned long kuap_get_and_check(void)
+{
+ unsigned long kuap = current->thread.kuap;
+
+ WARN_ON_ONCE(IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) && kuap != 0);
+
+ return kuap;
+}
+
+static inline void kuap_check(void)
+{
+ kuap_get_and_check();
+}
+
static __always_inline void allow_user_access(void __user *to, const void
__user *from,
u32 size, unsigned long dir)
{
--
2.25.0
[RFC PATCH v1 38/41] powerpc/64s: Make kuap_check_amr() and kuap_get_and_check_amr() generic
In preparation of porting powerpc32 to C syscall entry/exit,
rename kuap_check_amr() and kuap_get_and_check_amr() as kuap_check()
and kuap_get_and_check(), and move in the generic asm/kup.h the stub
for when CONFIG_PPC_KUAP is not selected.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/book3s/64/kup.h | 24 ++--
arch/powerpc/include/asm/kup.h | 10 +-
arch/powerpc/kernel/interrupt.c | 12 ++--
3 files changed, 17 insertions(+), 29 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h
b/arch/powerpc/include/asm/book3s/64/kup.h
index 8bd905050896..d9b07e9998be 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -287,7 +287,7 @@ static inline void kuap_kernel_restore(struct pt_regs *regs,
*/
}
-static inline unsigned long kuap_get_and_check_amr(void)
+static inline unsigned long kuap_get_and_check(void)
{
if (mmu_has_feature(MMU_FTR_BOOK3S_KUAP)) {
unsigned long amr = mfspr(SPRN_AMR);
@@ -298,27 +298,7 @@ static inline unsigned long kuap_get_and_check_amr(void)
return 0;
}
-#else /* CONFIG_PPC_PKEY */
-
-static inline void kuap_user_restore(struct pt_regs *regs)
-{
-}
-
-static inline void kuap_kernel_restore(struct pt_regs *regs, unsigned long amr)
-{
-}
-
-static inline unsigned long kuap_get_and_check_amr(void)
-{
- return 0;
-}
-
-#endif /* CONFIG_PPC_PKEY */
-
-
-#ifdef CONFIG_PPC_KUAP
-
-static inline void kuap_check_amr(void)
+static inline void kuap_check(void)
{
if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) &&
mmu_has_feature(MMU_FTR_BOOK3S_KUAP))
WARN_ON_ONCE(mfspr(SPRN_AMR) != AMR_KUAP_BLOCKED);
diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h
index 25671f711ec2..b7efa46b3109 100644
--- a/arch/powerpc/include/asm/kup.h
+++ b/arch/powerpc/include/asm/kup.h
@@ -74,7 +74,15 @@ bad_kuap_fault(struct pt_regs *regs, unsigned long address,
bool is_write)
return false;
}
-static inline void kuap_check_amr(void) { }
+static inline void kuap_check(void) { }
+static inline void kuap_save_and_lock(struct pt_regs *regs) { }
+static inline void kuap_user_restore(struct pt_regs *regs) { }
+static inline void kuap_kernel_restore(struct pt_regs *regs, unsigned long
amr) { }
+
+static inline unsigned long kuap_get_and_check(void)
+{
+ return 0;
+}
/*
* book3s/64/kup-radix.h defines these functions for the !KUAP case to flush
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index 3c2e9b28b05d..8180ed261a27 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -75,7 +75,7 @@ notrace long system_call_exception(long r3, long r4, long r5,
} else
#endif
#ifdef CONFIG_PPC64
- kuap_check_amr();
+ kuap_check();
#endif
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
@@ -241,7 +241,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
CT_WARN_ON(ct_state() == CONTEXT_USER);
#ifdef CONFIG_PPC64
- kuap_check_amr();
+ kuap_check();
#endif
regs->result = r3;
@@ -381,7 +381,7 @@ notrace unsigned long interrupt_exit_user_prepare(struct
pt_regs *regs, unsigned
* AMR can only have been unlocked if we interrupted the kernel.
*/
#ifdef CONFIG_PPC64
- kuap_check_amr();
+ kuap_check();
#endif
local_irq_save(flags);
@@ -467,7 +467,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct
pt_regs *regs, unsign
unsigned long flags;
unsigned long ret = 0;
#ifdef CONFIG_PPC64
- unsigned long amr;
+ unsigned long kuap;
#endif
if (!IS_ENABLED(CONFIG_BOOKE) && !IS_ENABLED(CONFIG_40x) &&
@@ -483,7 +483,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct
pt_regs *regs, unsign
CT_WARN_ON(ct_state() == CONTEXT_USER);
#ifdef CONFIG_PPC64
- amr = kuap_get_and_check_amr();
+ kuap = kuap_get_and_check();
#endif
if (unlikely(current_thread_info()->flags & _TIF_EMULATE_STACK_STORE)) {
@@ -527,7 +527,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct
pt_regs *regs, unsign
* value from the check above.
*/
#ifdef CONFIG_PPC64
- kuap_kernel_restore(regs, amr);
+ kuap_kernel_restore(regs, kuap);
#endif
return ret;
--
2.25.0
[RFC PATCH v1 37/41] powerpc/32s: Move KUEP locking/unlocking in C
This can be done in C, do it.
Unrolling the loop gains approx. 15% performance.
>From now on, prepare_transfer_to_handler() is only for
interrupts from kernel.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/book3s/32/kup.h | 31 ---
arch/powerpc/include/asm/interrupt.h | 3 ++
arch/powerpc/include/asm/kup.h | 8 +
arch/powerpc/kernel/entry_32.S | 17 ++-
arch/powerpc/kernel/interrupt.c | 4 +++
arch/powerpc/mm/book3s32/Makefile| 1 +
arch/powerpc/mm/book3s32/kuep.c | 38
7 files changed, 56 insertions(+), 46 deletions(-)
create mode 100644 arch/powerpc/mm/book3s32/kuep.c
diff --git a/arch/powerpc/include/asm/book3s/32/kup.h
b/arch/powerpc/include/asm/book3s/32/kup.h
index 73bc5d2c431d..b97ea60f6fa3 100644
--- a/arch/powerpc/include/asm/book3s/32/kup.h
+++ b/arch/powerpc/include/asm/book3s/32/kup.h
@@ -7,37 +7,6 @@
#ifdef __ASSEMBLY__
-.macro kuep_update_sr gpr1, gpr2 /* NEVER use r0 as gpr2 due to
addis */
-101: mtsrin \gpr1, \gpr2
- addi\gpr1, \gpr1, 0x111 /* next VSID */
- rlwinm \gpr1, \gpr1, 0, 0xf0ff /* clear VSID overflow */
- addis \gpr2, \gpr2, 0x1000/* address of next segment */
- bdnz101b
- isync
-.endm
-
-.macro kuep_lock gpr1, gpr2
-#ifdef CONFIG_PPC_KUEP
- li \gpr1, NUM_USER_SEGMENTS
- li \gpr2, 0
- mtctr \gpr1
- mfsrin \gpr1, \gpr2
- oris\gpr1, \gpr1, SR_NX@h /* set Nx */
- kuep_update_sr \gpr1, \gpr2
-#endif
-.endm
-
-.macro kuep_unlock gpr1, gpr2
-#ifdef CONFIG_PPC_KUEP
- li \gpr1, NUM_USER_SEGMENTS
- li \gpr2, 0
- mtctr \gpr1
- mfsrin \gpr1, \gpr2
- rlwinm \gpr1, \gpr1, 0, ~SR_NX /* Clear Nx */
- kuep_update_sr \gpr1, \gpr2
-#endif
-.endm
-
#ifdef CONFIG_PPC_KUAP
.macro kuap_update_sr gpr1, gpr2, gpr3/* NEVER use r0 as gpr2 due to
addis */
diff --git a/arch/powerpc/include/asm/interrupt.h
b/arch/powerpc/include/asm/interrupt.h
index 550ad1d69a7b..e5e04e962165 100644
--- a/arch/powerpc/include/asm/interrupt.h
+++ b/arch/powerpc/include/asm/interrupt.h
@@ -21,6 +21,7 @@ static inline void interrupt_enter_prepare(struct pt_regs
*regs, struct interrup
trace_hardirqs_off();
if (user_mode(regs)) {
+ kuep_lock();
current->thread.regs = regs;
account_cpu_user_entry();
}
@@ -83,6 +84,8 @@ static inline void interrupt_exit_prepare(struct pt_regs
*regs, struct interrupt
exception_exit(state->ctx_state);
#endif
+ if (user_mode(regs))
+ kuep_unlock();
/*
* Book3S exits to user via interrupt_exit_user_prepare(), which does
* context tracking, which is a cleaner way to handle PREEMPT=y
diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h
index 7ec21af49a45..25671f711ec2 100644
--- a/arch/powerpc/include/asm/kup.h
+++ b/arch/powerpc/include/asm/kup.h
@@ -55,6 +55,14 @@ void setup_kuep(bool disabled);
static inline void setup_kuep(bool disabled) { }
#endif /* CONFIG_PPC_KUEP */
+#if defined(CONFIG_PPC_KUEP) && defined(CONFIG_PPC_BOOK3S_32)
+void kuep_lock(void);
+void kuep_unlock(void);
+#else
+static inline void kuep_lock(void) { }
+static inline void kuep_unlock(void) { }
+#endif
+
#ifdef CONFIG_PPC_KUAP
void setup_kuap(bool disabled);
#else
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 544a9a2270ff..3ae790a362d1 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -60,13 +60,9 @@
prepare_transfer_to_handler:
andi. r0,r9,MSR_PR
addir12, r2, THREAD
- beq 2f
-#ifdef CONFIG_PPC_BOOK3S_32
- kuep_lock r11, r12
-#endif
- blr
+ bnelr
-2: /* if from kernel, check interrupted DOZE/NAP mode and
+ /* if from kernel, check interrupted DOZE/NAP mode and
* check for stack overflow
*/
kuap_save_and_lock r11, r12, r9, r5, r6
@@ -93,9 +89,6 @@ _ASM_NOKPROBE_SYMBOL(transfer_to_handler_cont)
.globl transfer_to_syscall
transfer_to_syscall:
SAVE_NVGPRS(r1)
-#ifdef CONFIG_PPC_BOOK3S_32
- kuep_lock r11, r12
-#endif
/* Calling convention has r9 = orig r0, r10 = regs */
addir10,r1,STACK_FRAME_OVERHEAD
@@ -112,9 +105,6 @@ ret_from_syscall:
cmplwi cr0,r5,0
bne-2f
#endif /* CONFIG_PPC_47x */
-#ifdef CONFIG_PPC_BOOK3S_32
- kuep_unlock r5, r7
-#endif
kuap_check r2, r4
lwz r4,_LINK(r1)
lwz r5,_CCR(r1)
@@ -295,9 +285,6 @@ interrupt_return:
bne-.Lrestore_nvgprs
.Lfast_user_interrupt_return:
-#ifdef CONFIG_PPC_BOOK3S_32
- kuep_unlock r10, r11
-#endif
kuap_check r2, r4
lwz r11,_NIP(r1)
lwz
[RFC PATCH v1 36/41] powerpc/32: Only use prepare_transfer_to_handler function on book3s/32 and e500
Only book3s/32 and e500 have significative work to do in prepare_transfer_to_handler. Other 32 bit have nothing to do at all. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 6 ++ arch/powerpc/kernel/head_32.h| 2 ++ arch/powerpc/kernel/head_booke.h | 2 ++ 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 70bc18f18f1a..544a9a2270ff 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -55,6 +55,7 @@ * Note that we rely on the caller having set cr0.eq iff the exception * occurred in kernel mode (i.e. MSR:PR = 0). */ +#if defined(CONFIG_PPC_BOOK3S_32) || defined(CONFIG_E500) .globl prepare_transfer_to_handler prepare_transfer_to_handler: andi. r0,r9,MSR_PR @@ -69,15 +70,12 @@ prepare_transfer_to_handler: * check for stack overflow */ kuap_save_and_lock r11, r12, r9, r5, r6 -#if defined(CONFIG_PPC_BOOK3S_32) || defined(CONFIG_E500) lwz r12,TI_LOCAL_FLAGS(r2) mtcrf 0x01,r12 bt- 31-TLF_NAPPING,4f bt- 31-TLF_SLEEPING,7f -#endif /* CONFIG_PPC_BOOK3S_32 || CONFIG_E500 */ blr -#if defined (CONFIG_PPC_BOOK3S_32) || defined(CONFIG_E500) 4: rlwinm r12,r12,0,~_TLF_NAPPING stw r12,TI_LOCAL_FLAGS(r2) b power_save_ppc32_restore @@ -88,9 +86,9 @@ prepare_transfer_to_handler: rlwinm r9,r9,0,~MSR_EE stw r9,_MSR(r11) b fast_exception_return -#endif _ASM_NOKPROBE_SYMBOL(prepare_transfer_to_handler) _ASM_NOKPROBE_SYMBOL(transfer_to_handler_cont) +#endif /* CONFIG_PPC_BOOK3S_32 || CONFIG_E500 */ .globl transfer_to_syscall transfer_to_syscall: diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index b29c13221baa..e994ee3575d2 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -132,7 +132,9 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) .endm .macro prepare_transfer_to_handler +#ifdef CONFIG_PPC_BOOK3S_32 bl prepare_transfer_to_handler +#endif .endm .macro SYSCALL_ENTRY trapno diff --git a/arch/powerpc/kernel/head_booke.h b/arch/powerpc/kernel/head_booke.h index 7e47b5dfdd9c..750ffc0915df 100644 --- a/arch/powerpc/kernel/head_booke.h +++ b/arch/powerpc/kernel/head_booke.h @@ -92,7 +92,9 @@ END_BTB_FLUSH_SECTION .end .macro prepare_transfer_to_handler +#ifdef CONFIG_E500 bl prepare_transfer_to_handler +#endif .endm .macro SYSCALL_ENTRY trapno intno srr1 -- 2.25.0
[RFC PATCH v1 34/41] powerpc/32: Set current->thread.regs in C interrupt entry
No need to do that is assembly, do it in C.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/interrupt.h | 4 +++-
arch/powerpc/kernel/entry_32.S | 3 +--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/include/asm/interrupt.h
b/arch/powerpc/include/asm/interrupt.h
index d70c761edc00..550ad1d69a7b 100644
--- a/arch/powerpc/include/asm/interrupt.h
+++ b/arch/powerpc/include/asm/interrupt.h
@@ -20,8 +20,10 @@ static inline void interrupt_enter_prepare(struct pt_regs
*regs, struct interrup
if (regs->msr & MSR_EE)
trace_hardirqs_off();
- if (user_mode(regs))
+ if (user_mode(regs)) {
+ current->thread.regs = regs;
account_cpu_user_entry();
+ }
#endif
/*
* Book3E reconciles irq soft mask in asm
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index fd24520b2e0c..71ef5ab84b03 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -59,8 +59,7 @@
prepare_transfer_to_handler:
andi. r0,r9,MSR_PR
addir12, r2, THREAD
- beq 2f /* if from user, fix up THREAD.regs */
- stw r3,PT_REGS(r12)
+ beq 2f
#ifdef CONFIG_PPC_BOOK3S_32
kuep_lock r11, r12
#endif
--
2.25.0
[RFC PATCH v1 33/41] powerpc/32: Save remaining registers in exception prolog
Save non volatile registers, XER, CTR, MSR and NIP in exception prolog. Also assign proper value to r2 and r3 there. For now, recalculate thread pointer in prepare_transfer_to_handler. It will disappear once KUAP is ported to C. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 18 +++--- arch/powerpc/kernel/head_32.h | 12 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 75ca010bd78d..fd24520b2e0c 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -57,20 +57,9 @@ */ .globl prepare_transfer_to_handler prepare_transfer_to_handler: - SAVE_NVGPRS(r11) - addir3,r1,STACK_FRAME_OVERHEAD - stw r2,GPR2(r11) - stw r12,_NIP(r11) - stw r9,_MSR(r11) - andi. r2,r9,MSR_PR - mfctr r12 - mfspr r2,SPRN_XER - stw r12,_CTR(r11) - stw r2,_XER(r11) - mfspr r12,SPRN_SPRG_THREAD - tovirt(r12, r12) + andi. r0,r9,MSR_PR + addir12, r2, THREAD beq 2f /* if from user, fix up THREAD.regs */ - addir2, r12, -THREAD stw r3,PT_REGS(r12) #ifdef CONFIG_PPC_BOOK3S_32 kuep_lock r11, r12 @@ -80,8 +69,7 @@ prepare_transfer_to_handler: 2: /* if from kernel, check interrupted DOZE/NAP mode and * check for stack overflow */ - kuap_save_and_lock r11, r12, r9, r2, r6 - addir2, r12, -THREAD + kuap_save_and_lock r11, r12, r9, r5, r6 #if defined(CONFIG_PPC_BOOK3S_32) || defined(CONFIG_E500) lwz r12,TI_LOCAL_FLAGS(r2) mtcrf 0x01,r12 diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 12c39178a1d2..b29c13221baa 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -116,6 +116,18 @@ stw r10,_TRAP(r1) SAVE_4GPRS(3, r1) SAVE_2GPRS(7, r1) + SAVE_NVGPRS(r1) + stw r2,GPR2(r1) + stw r12,_NIP(r1) + stw r9,_MSR(r1) + mfctr r0 + mfspr r10,SPRN_XER + mfspr r2,SPRN_SPRG_THREAD + stw r0,_CTR(r1) + tovirt(r2, r2) + stw r10,_XER(r1) + addir2, r2, -THREAD + addir3,r1,STACK_FRAME_OVERHEAD _ASM_NOKPROBE_SYMBOL(\name\()_virt) .endm -- 2.25.0
[RFC PATCH v1 32/41] powerpc/32: Refactor saving of volatile registers in exception prologs
Exception prologs all do the same at the end: - Save trapno in stack - Mark stack with exception marker - Save r0 - Save r3 to r8 Refactor that into a COMMON_EXCEPTION_PROLOG_END macro. At the same time use r1 instead of r11. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.h| 14 +- arch/powerpc/kernel/head_40x.S | 9 + arch/powerpc/kernel/head_booke.h | 26 +- 3 files changed, 23 insertions(+), 26 deletions(-) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 84e6251622e8..12c39178a1d2 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -104,14 +104,18 @@ li r10, MSR_KERNEL /* can take exceptions */ mtmsr r10 /* (except for mach check in rtas) */ #endif - stw r0,GPR0(r11) + COMMON_EXCEPTION_PROLOG_END \trapno +.endm + +.macro COMMON_EXCEPTION_PROLOG_END trapno + stw r0,GPR0(r1) lis r10,STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ addir10,r10,STACK_FRAME_REGS_MARKER@l - stw r10,8(r11) + stw r10,8(r1) li r10, \trapno - stw r10,_TRAP(r11) - SAVE_4GPRS(3, r11) - SAVE_2GPRS(7, r11) + stw r10,_TRAP(r1) + SAVE_4GPRS(3, r1) + SAVE_2GPRS(7, r1) _ASM_NOKPROBE_SYMBOL(\name\()_virt) .endm diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 52b40bf529c6..e1360b88b6cb 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -157,14 +157,7 @@ _ENTRY(crit_esr) mfspr r12,SPRN_SRR2 mfspr r9,SPRN_SRR3 rlwinm r9,r9,0,14,12 /* clear MSR_WE (necessary?) */ - stw r0,GPR0(r11) - lis r10, STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ - addir10, r10, STACK_FRAME_REGS_MARKER@l - stw r10, 8(r11) - li r10, \trapno + 2 - stw r10,_TRAP(r11) - SAVE_4GPRS(3, r11) - SAVE_2GPRS(7, r11) + COMMON_EXCEPTION_PROLOG_END \trapno + 2 _ASM_NOKPROBE_SYMBOL(\name\()_virt) .endm diff --git a/arch/powerpc/kernel/head_booke.h b/arch/powerpc/kernel/head_booke.h index 7c8195ae95c4..7e47b5dfdd9c 100644 --- a/arch/powerpc/kernel/head_booke.h +++ b/arch/powerpc/kernel/head_booke.h @@ -78,14 +78,18 @@ END_BTB_FLUSH_SECTION stw r1, 0(r11); \ mr r1, r11; \ rlwinm r9,r9,0,14,12; /* clear MSR_WE (necessary?) */\ - stw r0,GPR0(r11);\ - lis r10, STACK_FRAME_REGS_MARKER@ha;/* exception frame marker */ \ - addir10, r10, STACK_FRAME_REGS_MARKER@l; \ - stw r10, 8(r11); \ - li r10, trapno; \ - stw r10,_TRAP(r11); \ - SAVE_4GPRS(3, r11); \ - SAVE_2GPRS(7, r11) + COMMON_EXCEPTION_PROLOG_END trapno + +.macro COMMON_EXCEPTION_PROLOG_END trapno + stw r0,GPR0(r1) + lis r10, STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ + addir10, r10, STACK_FRAME_REGS_MARKER@l + stw r10, 8(r1) + li r10, trapno + stw r10,_TRAP(r1) + SAVE_4GPRS(3, r1) + SAVE_2GPRS(7, r1) +.end .macro prepare_transfer_to_handler bl prepare_transfer_to_handler @@ -231,11 +235,7 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV) stw r1,0(r11); \ mr r1,r11; \ rlwinm r9,r9,0,14,12; /* clear MSR_WE (necessary?) */\ - li r10, trapno; \ - stw r10,_TRAP(r11); \ - stw r0,GPR0(r11);\ - SAVE_4GPRS(3, r11); \ - SAVE_2GPRS(7, r11) + COMMON_EXCEPTION_PROLOG_END trapno .macro SAVE_xSRR xSRR mfspr r0,SPRN_\xSRR\()0 -- 2.25.0
[RFC PATCH v1 35/41] powerpc/32: Return directly from power_save_ppc32_restore()
transfer_to_handler_cont: is now just a blr. Directly perform blr in power_save_ppc32_restore(). Also remove useless setting of r11 in e500 version of power_save_ppc32_restore(). Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 2 -- arch/powerpc/kernel/idle_6xx.S | 2 +- arch/powerpc/kernel/idle_e500.S | 10 +- 3 files changed, 2 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 71ef5ab84b03..70bc18f18f1a 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -75,8 +75,6 @@ prepare_transfer_to_handler: bt- 31-TLF_NAPPING,4f bt- 31-TLF_SLEEPING,7f #endif /* CONFIG_PPC_BOOK3S_32 || CONFIG_E500 */ - .globl transfer_to_handler_cont -transfer_to_handler_cont: blr #if defined (CONFIG_PPC_BOOK3S_32) || defined(CONFIG_E500) diff --git a/arch/powerpc/kernel/idle_6xx.S b/arch/powerpc/kernel/idle_6xx.S index 153366e178c4..13cad9297d82 100644 --- a/arch/powerpc/kernel/idle_6xx.S +++ b/arch/powerpc/kernel/idle_6xx.S @@ -176,7 +176,7 @@ BEGIN_FTR_SECTION lwz r9,nap_save_hid1@l(r9) mtspr SPRN_HID1, r9 END_FTR_SECTION_IFSET(CPU_FTR_DUAL_PLL_750FX) - b transfer_to_handler_cont + blr _ASM_NOKPROBE_SYMBOL(power_save_ppc32_restore) .data diff --git a/arch/powerpc/kernel/idle_e500.S b/arch/powerpc/kernel/idle_e500.S index 7795727e7f08..9e1bc4502c50 100644 --- a/arch/powerpc/kernel/idle_e500.S +++ b/arch/powerpc/kernel/idle_e500.S @@ -81,13 +81,5 @@ END_FTR_SECTION_IFSET(CPU_FTR_CAN_NAP) _GLOBAL(power_save_ppc32_restore) lwz r9,_LINK(r11) /* interrupted in e500_idle */ stw r9,_NIP(r11)/* make it do a blr */ - -#ifdef CONFIG_SMP - lwz r11,TASK_CPU(r2)/* get cpu number * 4 */ - slwir11,r11,2 -#else - li r11,0 -#endif - - b transfer_to_handler_cont + blr _ASM_NOKPROBE_SYMBOL(power_save_ppc32_restore) -- 2.25.0
[RFC PATCH v1 31/41] powerpc/32: Remove the xfer parameter in EXCEPTION() macro
The xfer parameter is not used anymore, remove it. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.h| 2 +- arch/powerpc/kernel/head_40x.S | 42 arch/powerpc/kernel/head_44x.S | 10 ++-- arch/powerpc/kernel/head_8xx.S | 14 +++--- arch/powerpc/kernel/head_book3s_32.S | 72 ++-- arch/powerpc/kernel/head_booke.h | 2 +- arch/powerpc/kernel/head_fsl_booke.S | 28 +-- 7 files changed, 81 insertions(+), 89 deletions(-) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 412ede8610f7..84e6251622e8 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -186,7 +186,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) #endif -#define EXCEPTION(n, label, hdlr, xfer)\ +#define EXCEPTION(n, label, hdlr) \ START_EXCEPTION(n, label) \ EXCEPTION_PROLOG n label; \ prepare_transfer_to_handler;\ diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 7eb49ebd6000..52b40bf529c6 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -228,7 +228,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) b interrupt_return /* 0x0500 - External Interrupt Exception */ - EXCEPTION(0x0500, HardwareInterrupt, do_IRQ, EXC_XFER_LITE) + EXCEPTION(0x0500, HardwareInterrupt, do_IRQ) /* 0x0600 - Alignment Exception */ START_EXCEPTION(0x0600, Alignment) @@ -246,19 +246,19 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) REST_NVGPRS(r1) b interrupt_return - EXCEPTION(0x0800, Trap_08, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x0900, Trap_09, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x0A00, Trap_0A, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x0B00, Trap_0B, unknown_exception, EXC_XFER_STD) + EXCEPTION(0x0800, Trap_08, unknown_exception) + EXCEPTION(0x0900, Trap_09, unknown_exception) + EXCEPTION(0x0A00, Trap_0A, unknown_exception) + EXCEPTION(0x0B00, Trap_0B, unknown_exception) /* 0x0C00 - System Call Exception */ START_EXCEPTION(0x0C00, SystemCall) SYSCALL_ENTRY 0xc00 /* Trap_0D is commented out to get more space for system call exception */ -/* EXCEPTION(0x0D00, Trap_0D, unknown_exception, EXC_XFER_STD) */ - EXCEPTION(0x0E00, Trap_0E, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x0F00, Trap_0F, unknown_exception, EXC_XFER_STD) +/* EXCEPTION(0x0D00, Trap_0D, unknown_exception) */ + EXCEPTION(0x0E00, Trap_0E, unknown_exception) + EXCEPTION(0x0F00, Trap_0F, unknown_exception) /* 0x1000 - Programmable Interval Timer (PIT) Exception */ START_EXCEPTION(0x1000, DecrementerTrap) @@ -433,19 +433,19 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) mfspr r10, SPRN_SPRG_SCRATCH5 b InstructionAccess - EXCEPTION(0x1300, Trap_13, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1400, Trap_14, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1500, Trap_15, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1600, Trap_16, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1700, Trap_17, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1800, Trap_18, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1900, Trap_19, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1A00, Trap_1A, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1B00, Trap_1B, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1C00, Trap_1C, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1D00, Trap_1D, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1E00, Trap_1E, unknown_exception, EXC_XFER_STD) - EXCEPTION(0x1F00, Trap_1F, unknown_exception, EXC_XFER_STD) + EXCEPTION(0x1300, Trap_13, unknown_exception) + EXCEPTION(0x1400, Trap_14, unknown_exception) + EXCEPTION(0x1500, Trap_15, unknown_exception) + EXCEPTION(0x1600, Trap_16, unknown_exception) + EXCEPTION(0x1700, Trap_17, unknown_exception) + EXCEPTION(0x1800, Trap_18, unknown_exception) + EXCEPTION(0x1900, Trap_19, unknown_exception) + EXCEPTION(0x1A00, Trap_1A, unknown_exception) + EXCEPTION(0x1B00, Trap_1B, unknown_exception) + EXCEPTION(0x1C00, Trap_1C, unknown_exception) + EXCEPTION(0x1D00, Trap_1D, unknown_exception) + EXCEPTION(0x1E00, Trap_1E, unknown_exception) + EXCEPTION(0x1F00, Trap_1F, unknown_exception) /* Check for a single step debug exception while in an exception * handler before state has been saved. This is to catch the case diff --git a/arch/powerpc/kernel/head_44x.S b/arch/powerpc/kernel/head_44x.S index 813fa305c33b..5c106ac36626 100644 --- a/arch/powerpc/kernel/head_44x.S +++ b/arch/powerpc/kernel/head_44x.S @@ -263,8 +263,7 @@ interrupt_base: INSTRUCTION_STORAGE_EXCEPTION /* External Input
[RFC PATCH v1 30/41] powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE
In order to get more control in exception prolog, dismantle all non standard exception macros, finishing with EXC_XFER_STD and EXC_XFER_LITE and EXC_XFER_TEMPLATE. Also remove transfer_to_handler_full and ret_from_except and ret_from_except_full as they are not used anymore. Last parameter of EXCEPTION() is now ignored, will be removed in a later patch to avoid too much churn. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 40 --- arch/powerpc/kernel/head_32.h| 21 arch/powerpc/kernel/head_40x.S | 33 --- arch/powerpc/kernel/head_8xx.S | 12 +-- arch/powerpc/kernel/head_book3s_32.S | 27 ++- arch/powerpc/kernel/head_booke.h | 49 +++- arch/powerpc/kernel/head_fsl_booke.S | 14 +--- 7 files changed, 91 insertions(+), 105 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index d8a913f850ef..75ca010bd78d 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -48,30 +48,6 @@ */ .align 12 -#ifdef CONFIG_BOOKE - .globl mcheck_transfer_to_handler -mcheck_transfer_to_handler: - /* fall through */ -_ASM_NOKPROBE_SYMBOL(mcheck_transfer_to_handler) - - .globl debug_transfer_to_handler -debug_transfer_to_handler: - /* fall through */ -_ASM_NOKPROBE_SYMBOL(debug_transfer_to_handler) - - .globl crit_transfer_to_handler -crit_transfer_to_handler: - /* fall through */ -_ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler) -#endif - -#ifdef CONFIG_40x - .globl crit_transfer_to_handler -crit_transfer_to_handler: - /* fall through */ -_ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler) -#endif - /* * This code finishes saving the registers to the exception frame * and jumps to the appropriate handler for the exception, turning @@ -79,13 +55,6 @@ _ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler) * Note that we rely on the caller having set cr0.eq iff the exception * occurred in kernel mode (i.e. MSR:PR = 0). */ - .globl transfer_to_handler_full -transfer_to_handler_full: -_ASM_NOKPROBE_SYMBOL(transfer_to_handler_full) - /* fall through */ - - .globl transfer_to_handler -transfer_to_handler: .globl prepare_transfer_to_handler prepare_transfer_to_handler: SAVE_NVGPRS(r11) @@ -136,7 +105,6 @@ transfer_to_handler_cont: b fast_exception_return #endif _ASM_NOKPROBE_SYMBOL(prepare_transfer_to_handler) -_ASM_NOKPROBE_SYMBOL(transfer_to_handler) _ASM_NOKPROBE_SYMBOL(transfer_to_handler_cont) .globl transfer_to_syscall @@ -333,14 +301,6 @@ fast_exception_return: #endif _ASM_NOKPROBE_SYMBOL(fast_exception_return) - .globl ret_from_except_full -ret_from_except_full: - /* fall through */ - - .globl ret_from_except -ret_from_except: -_ASM_NOKPROBE_SYMBOL(ret_from_except) - .globl interrupt_return interrupt_return: lwz r4,_MSR(r1) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 3ab0f3ad9a6a..412ede8610f7 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -189,20 +189,9 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) #define EXCEPTION(n, label, hdlr, xfer)\ START_EXCEPTION(n, label) \ EXCEPTION_PROLOG n label; \ - xfer(n, hdlr) - -#define EXC_XFER_TEMPLATE(hdlr, trap, msr, tfer, ret) \ - bl tfer; \ - bl hdlr; \ - b ret - -#define EXC_XFER_STD(n, hdlr) \ - EXC_XFER_TEMPLATE(hdlr, n, MSR_KERNEL, transfer_to_handler_full, \ - ret_from_except_full) - -#define EXC_XFER_LITE(n, hdlr) \ - EXC_XFER_TEMPLATE(hdlr, n, MSR_KERNEL, transfer_to_handler, \ - ret_from_except) + prepare_transfer_to_handler;\ + bl hdlr; \ + b interrupt_return .macro vmap_stack_overflow_exception __HEAD @@ -218,7 +207,9 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) lwz r1, emergency_ctx@l(r1) addir1, r1, THREAD_SIZE - INT_FRAME_SIZE EXCEPTION_PROLOG_2 0 vmap_stack_overflow - EXC_XFER_STD(0, stack_overflow_exception) + prepare_transfer_to_handler + bl stack_overflow_exception + b interrupt_return .endm #endif /* __HEAD_32_H__ */ diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index f3e5b462113f..7eb49ebd6000 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -187,8 +187,9 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) #define CRITICAL_EXCEPTION(n, label, hdlr) \ START_EXCEPTION(n, label); \
[RFC PATCH v1 29/41] powerpc/32: Only restore non volatile registers when required
Until now, non volatile registers were restored everytime they were saved, ie using EXC_XFER_STD meant saving and restoring them while EXC_XFER_LITE meant neither saving not restoring them. Now that they are always saved, EXC_XFER_STD means to restore them and EXC_XFER_LITE means to not restore them. Most of the users of EXC_XFER_STD only need to retrieve the non volatile registers. For them there is no need to restore the non volatile registers as they have not been modified. Only very few exceptions require non volatile registers restore. Opencode the few places which require saving of non volatile registers. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 1 - arch/powerpc/kernel/head_40x.S | 10 -- arch/powerpc/kernel/head_8xx.S | 24 arch/powerpc/kernel/head_book3s_32.S | 17 ++--- arch/powerpc/kernel/head_booke.h | 10 -- arch/powerpc/kernel/head_fsl_booke.S | 16 6 files changed, 62 insertions(+), 16 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 03f4d07f4176..d8a913f850ef 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -335,7 +335,6 @@ _ASM_NOKPROBE_SYMBOL(fast_exception_return) .globl ret_from_except_full ret_from_except_full: - REST_NVGPRS(r1) /* fall through */ .globl ret_from_except diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 7270caff665c..f3e5b462113f 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -228,12 +228,18 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) /* 0x0600 - Alignment Exception */ START_EXCEPTION(0x0600, Alignment) EXCEPTION_PROLOG 0x600 Alignment handle_dar_dsisr=1 - EXC_XFER_STD(0x600, alignment_exception) + prepare_transfer_to_handler + bl alignment_exception + REST_NVGPRS(r1) + b interrupt_return /* 0x0700 - Program Exception */ START_EXCEPTION(0x0700, ProgramCheck) EXCEPTION_PROLOG 0x700 ProgramCheck handle_dar_dsisr=1 - EXC_XFER_STD(0x700, program_check_exception) + prepare_transfer_to_handler + bl program_check_exception + REST_NVGPRS(r1) + b interrupt_return EXCEPTION(0x0800, Trap_08, unknown_exception, EXC_XFER_STD) EXCEPTION(0x0900, Trap_09, unknown_exception, EXC_XFER_STD) diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index c48de97f42fc..86f844eb0e5a 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -131,10 +131,18 @@ instruction_counter: /* Alignment exception */ START_EXCEPTION(0x600, Alignment) EXCEPTION_PROLOG 0x600 Alignment handle_dar_dsisr=1 - EXC_XFER_STD(0x600, alignment_exception) + prepare_transfer_to_handler + bl alignment_exception + REST_NVGPRS(r1) + b interrupt_return /* Program check exception */ - EXCEPTION(0x700, ProgramCheck, program_check_exception, EXC_XFER_STD) + START_EXCEPTION(0x700, ProgramCheck) + EXCEPTION_PROLOG 0x700 ProgramCheck + prepare_transfer_to_handler + bl program_check_exception + REST_NVGPRS(r1) + b interrupt_return /* Decrementer */ EXCEPTION(0x900, Decrementer, timer_interrupt, EXC_XFER_LITE) @@ -149,7 +157,12 @@ instruction_counter: /* On the MPC8xx, this is a software emulation interrupt. It occurs * for all unimplemented and illegal instructions. */ - EXCEPTION(0x1000, SoftEmu, emulation_assist_interrupt, EXC_XFER_STD) + START_EXCEPTION(0x1000, SoftEmu) + EXCEPTION_PROLOG 0x1000 SoftEmu + prepare_transfer_to_handler + bl emulation_assist_interrupt + REST_NVGPRS(r1) + b interrupt_return /* * For the MPC8xx, this is a software tablewalk to load the instruction @@ -348,7 +361,10 @@ DARFixed:/* Return from dcbx instruction bug workaround */ EXCEPTION_PROLOG_2 0x1c00 DataBreakpoint handle_dar_dsisr=1 mfspr r4,SPRN_BAR stw r4,_DAR(r11) - EXC_XFER_STD(0x1c00, do_break) + prepare_transfer_to_handler + bl do_break + REST_NVGPRS(r1) + b interrupt_return #ifdef CONFIG_PERF_EVENTS START_EXCEPTION(0x1d00, InstructionBreakpoint) diff --git a/arch/powerpc/kernel/head_book3s_32.S b/arch/powerpc/kernel/head_book3s_32.S index 67dac65b8ec3..609b2eedd4f9 100644 --- a/arch/powerpc/kernel/head_book3s_32.S +++ b/arch/powerpc/kernel/head_book3s_32.S @@ -300,7 +300,10 @@ ALT_MMU_FTR_SECTION_END_IFSET(MMU_FTR_HPTE_TABLE) andis. r0, r5, DSISR_DABRMATCH@h bne-1f EXC_XFER_LITE(0x300, do_page_fault) -1: EXC_XFER_STD(0x300, do_break) +1: prepare_transfer_to_handler + bl do_break + REST_NVGPRS(r1) + b interrupt_return
[RFC PATCH v1 28/41] powerpc/32: Add a prepare_transfer_to_handler macro for exception prologs
In order to increase flexibility, add a macro that will for now call transfer_to_handler. As transfer_to_handler doesn't do the actual transfer anymore, also name it prepare_transfer_to_handler. The following patches will progressively remove the use of transfer_to_handler label. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 3 +++ arch/powerpc/kernel/head_32.h| 4 arch/powerpc/kernel/head_booke.h | 4 3 files changed, 11 insertions(+) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 890a4508c1f1..03f4d07f4176 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -86,6 +86,8 @@ _ASM_NOKPROBE_SYMBOL(transfer_to_handler_full) .globl transfer_to_handler transfer_to_handler: + .globl prepare_transfer_to_handler +prepare_transfer_to_handler: SAVE_NVGPRS(r11) addir3,r1,STACK_FRAME_OVERHEAD stw r2,GPR2(r11) @@ -133,6 +135,7 @@ transfer_to_handler_cont: stw r9,_MSR(r11) b fast_exception_return #endif +_ASM_NOKPROBE_SYMBOL(prepare_transfer_to_handler) _ASM_NOKPROBE_SYMBOL(transfer_to_handler) _ASM_NOKPROBE_SYMBOL(transfer_to_handler_cont) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index bf4c288173ad..3ab0f3ad9a6a 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -115,6 +115,10 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) .endm +.macro prepare_transfer_to_handler + bl prepare_transfer_to_handler +.endm + .macro SYSCALL_ENTRY trapno mfspr r9, SPRN_SRR1 mfspr r10, SPRN_SRR0 diff --git a/arch/powerpc/kernel/head_booke.h b/arch/powerpc/kernel/head_booke.h index 12add20edbb1..9bc86aab65ef 100644 --- a/arch/powerpc/kernel/head_booke.h +++ b/arch/powerpc/kernel/head_booke.h @@ -87,6 +87,10 @@ END_BTB_FLUSH_SECTION SAVE_4GPRS(3, r11); \ SAVE_2GPRS(7, r11) +.macro prepare_transfer_to_handler + bl prepare_transfer_to_handler +.endm + .macro SYSCALL_ENTRY trapno intno srr1 mfspr r10, SPRN_SPRG_THREAD #ifdef CONFIG_KVM_BOOKE_HV -- 2.25.0
[RFC PATCH v1 27/41] powerpc/32: Save trap number on stack in exception prolog
Saving the trap number into the stack goes into the exception prolog, as EXC_XFER_xxx will soon disappear. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.h| 14 - arch/powerpc/kernel/head_40x.S | 22 +++--- arch/powerpc/kernel/head_8xx.S | 14 - arch/powerpc/kernel/head_book3s_32.S | 14 - arch/powerpc/kernel/head_booke.h | 44 +++- arch/powerpc/kernel/head_fsl_booke.S | 4 +-- 6 files changed, 58 insertions(+), 54 deletions(-) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 4d638d760a96..bf4c288173ad 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -10,10 +10,10 @@ * We assume sprg3 has the physical address of the current * task's thread_struct. */ -.macro EXCEPTION_PROLOGname handle_dar_dsisr=0 +.macro EXCEPTION_PROLOGtrapno name handle_dar_dsisr=0 EXCEPTION_PROLOG_0 handle_dar_dsisr=\handle_dar_dsisr EXCEPTION_PROLOG_1 - EXCEPTION_PROLOG_2 \name handle_dar_dsisr=\handle_dar_dsisr + EXCEPTION_PROLOG_2 \trapno \name handle_dar_dsisr=\handle_dar_dsisr .endm .macro EXCEPTION_PROLOG_0 handle_dar_dsisr=0 @@ -56,7 +56,7 @@ #endif .endm -.macro EXCEPTION_PROLOG_2 name handle_dar_dsisr=0 +.macro EXCEPTION_PROLOG_2 trapno name handle_dar_dsisr=0 #ifdef CONFIG_PPC_8xx .if \handle_dar_dsisr li r11, RPN_PATTERN @@ -108,6 +108,8 @@ lis r10,STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ addir10,r10,STACK_FRAME_REGS_MARKER@l stw r10,8(r11) + li r10, \trapno + stw r10,_TRAP(r11) SAVE_4GPRS(3, r11) SAVE_2GPRS(7, r11) _ASM_NOKPROBE_SYMBOL(\name\()_virt) @@ -182,12 +184,10 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) #define EXCEPTION(n, label, hdlr, xfer)\ START_EXCEPTION(n, label) \ - EXCEPTION_PROLOG label; \ + EXCEPTION_PROLOG n label; \ xfer(n, hdlr) #define EXC_XFER_TEMPLATE(hdlr, trap, msr, tfer, ret) \ - li r10,trap; \ - stw r10,_TRAP(r11); \ bl tfer; \ bl hdlr; \ b ret @@ -213,7 +213,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) #endif lwz r1, emergency_ctx@l(r1) addir1, r1, THREAD_SIZE - INT_FRAME_SIZE - EXCEPTION_PROLOG_2 vmap_stack_overflow + EXCEPTION_PROLOG_2 0 vmap_stack_overflow EXC_XFER_STD(0, stack_overflow_exception) .endm diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index a65778380704..7270caff665c 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -104,7 +104,7 @@ _ENTRY(crit_esr) * Instead we use a couple of words of memory at low physical addresses. * This is OK since we don't support SMP on these processors. */ -.macro CRITICAL_EXCEPTION_PROLOG name +.macro CRITICAL_EXCEPTION_PROLOG trapno name stw r10,crit_r10@l(0) /* save two registers to work with */ stw r11,crit_r11@l(0) mfspr r10,SPRN_SRR0 @@ -161,6 +161,8 @@ _ENTRY(crit_esr) lis r10, STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ addir10, r10, STACK_FRAME_REGS_MARKER@l stw r10, 8(r11) + li r10, \trapno + 2 + stw r10,_TRAP(r11) SAVE_4GPRS(3, r11) SAVE_2GPRS(7, r11) _ASM_NOKPROBE_SYMBOL(\name\()_virt) @@ -184,7 +186,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) */ #define CRITICAL_EXCEPTION(n, label, hdlr) \ START_EXCEPTION(n, label); \ - CRITICAL_EXCEPTION_PROLOG label;\ + CRITICAL_EXCEPTION_PROLOG n label; \ EXC_XFER_TEMPLATE(hdlr, n+2, (MSR_KERNEL & ~(MSR_ME|MSR_DE|MSR_CE)), \ crit_transfer_to_handler, ret_from_crit_exc) @@ -206,7 +208,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) * if they can't resolve the lightweight TLB fault. */ START_EXCEPTION(0x0300, DataStorage) - EXCEPTION_PROLOG DataStorage handle_dar_dsisr=1 + EXCEPTION_PROLOG 0x300 DataStorage handle_dar_dsisr=1 EXC_XFER_LITE(0x300, do_page_fault) /* @@ -214,7 +216,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) * This is caused by a fetch from non-execute or guarded pages. */ START_EXCEPTION(0x0400, InstructionAccess) - EXCEPTION_PROLOG InstructionAccess + EXCEPTION_PROLOG 0x400 InstructionAccess li r5,0 stw r5, _ESR(r11) /* Zero ESR */ stw r12, _DEAR(r11) /* SRR0 as DEAR */ @@ -225,12 +227,12 @@
[RFC PATCH v1 26/41] powerpc/32: Remove handle_page_fault()
Now that non volatile registers are saved at all time, no need to split bad_page_fault() out of do_page_fault(). Remove handle_page_fault() and use do_page_fault() directly. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 16 arch/powerpc/kernel/head_40x.S | 4 ++-- arch/powerpc/kernel/head_8xx.S | 4 ++-- arch/powerpc/kernel/head_book3s_32.S | 4 ++-- arch/powerpc/kernel/head_booke.h | 4 ++-- arch/powerpc/kernel/head_fsl_booke.S | 2 +- arch/powerpc/mm/fault.c | 4 ++-- 7 files changed, 11 insertions(+), 27 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 0b4fdc930aa5..890a4508c1f1 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -220,22 +220,6 @@ ret_from_kernel_thread: li r3,0 b ret_from_syscall -/* - * Top-level page fault handling. - * This is in assembler because if do_page_fault tells us that - * it is a bad kernel page fault, we want to save the non-volatile - * registers before calling bad_page_fault. - */ - .globl handle_page_fault -handle_page_fault: - bl do_page_fault - cmpwi r3,0 - beq+ret_from_except - mr r4,r3 /* err arg for bad_page_fault */ - addir3,r1,STACK_FRAME_OVERHEAD - bl __bad_page_fault - b ret_from_except_full - /* * This routine switches between two different tasks. The process * state of one is saved on its kernel stack. Then the state diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 08563d4170c6..a65778380704 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -207,7 +207,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) */ START_EXCEPTION(0x0300, DataStorage) EXCEPTION_PROLOG DataStorage handle_dar_dsisr=1 - EXC_XFER_LITE(0x300, handle_page_fault) + EXC_XFER_LITE(0x300, do_page_fault) /* * 0x0400 - Instruction Storage Exception @@ -218,7 +218,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) li r5,0 stw r5, _ESR(r11) /* Zero ESR */ stw r12, _DEAR(r11) /* SRR0 as DEAR */ - EXC_XFER_LITE(0x400, handle_page_fault) + EXC_XFER_LITE(0x400, do_page_fault) /* 0x0500 - External Interrupt Exception */ EXCEPTION(0x0500, HardwareInterrupt, do_IRQ, EXC_XFER_LITE) diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index eb1d40a8f2c4..4078d0dc2f18 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -301,7 +301,7 @@ instruction_counter: .Litlbie: stw r12, _DAR(r11) stw r5, _DSISR(r11) - EXC_XFER_LITE(0x400, handle_page_fault) + EXC_XFER_LITE(0x400, do_page_fault) /* This is the data TLB error on the MPC8xx. This could be due to * many reasons, including a dirty update to a pte. We bail out to @@ -322,7 +322,7 @@ DARFixed:/* Return from dcbx instruction bug workaround */ tlbie r4 .Ldtlbie: /* 0x300 is DataAccess exception, needed by bad_page_fault() */ - EXC_XFER_LITE(0x300, handle_page_fault) + EXC_XFER_LITE(0x300, do_page_fault) #ifdef CONFIG_VMAP_STACK vmap_stack_overflow_exception diff --git a/arch/powerpc/kernel/head_book3s_32.S b/arch/powerpc/kernel/head_book3s_32.S index 626e9fbac2cc..81a6ec098dd1 100644 --- a/arch/powerpc/kernel/head_book3s_32.S +++ b/arch/powerpc/kernel/head_book3s_32.S @@ -299,7 +299,7 @@ ALT_MMU_FTR_SECTION_END_IFSET(MMU_FTR_HPTE_TABLE) lwz r5, _DSISR(r11) andis. r0, r5, DSISR_DABRMATCH@h bne-1f - EXC_XFER_LITE(0x300, handle_page_fault) + EXC_XFER_LITE(0x300, do_page_fault) 1: EXC_XFER_STD(0x300, do_break) @@ -328,7 +328,7 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_HPTE_TABLE) andis. r5,r9,DSISR_SRR1_MATCH_32S@h /* Filter relevant SRR1 bits */ stw r5, _DSISR(r11) stw r12, _DAR(r11) - EXC_XFER_LITE(0x400, handle_page_fault) + EXC_XFER_LITE(0x400, do_page_fault) /* External interrupt */ EXCEPTION(0x500, HardwareInterrupt, do_IRQ, EXC_XFER_LITE) diff --git a/arch/powerpc/kernel/head_booke.h b/arch/powerpc/kernel/head_booke.h index 740ddf506efc..9e7b0192bba7 100644 --- a/arch/powerpc/kernel/head_booke.h +++ b/arch/powerpc/kernel/head_booke.h @@ -462,7 +462,7 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV) stw r5,_ESR(r11); \ mfspr r4,SPRN_DEAR; /* Grab the DEAR */ \ stw r4, _DEAR(r11); \ - EXC_XFER_LITE(0x0300, handle_page_fault) + EXC_XFER_LITE(0x0300, do_page_fault) #define INSTRUCTION_STORAGE_EXCEPTION\ START_EXCEPTION(InstructionStorage)
[RFC PATCH v1 25/41] powerpc/32: Set regs parameter in r3 in transfer_to_handler
All exception handlers take regs as first parameter. Instead of setting r3 just before each call to a handler, set it in transfer_to_handler. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 5 ++--- arch/powerpc/kernel/head_32.h| 2 -- arch/powerpc/kernel/head_40x.S | 7 --- arch/powerpc/kernel/head_8xx.S | 3 --- arch/powerpc/kernel/head_book3s_32.S | 9 ++--- arch/powerpc/kernel/head_booke.h | 11 +-- arch/powerpc/kernel/head_fsl_booke.S | 4 +--- 7 files changed, 6 insertions(+), 35 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 9a75ccb800e7..0b4fdc930aa5 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -87,6 +87,7 @@ _ASM_NOKPROBE_SYMBOL(transfer_to_handler_full) .globl transfer_to_handler transfer_to_handler: SAVE_NVGPRS(r11) + addir3,r1,STACK_FRAME_OVERHEAD stw r2,GPR2(r11) stw r12,_NIP(r11) stw r9,_MSR(r11) @@ -99,8 +100,7 @@ transfer_to_handler: tovirt(r12, r12) beq 2f /* if from user, fix up THREAD.regs */ addir2, r12, -THREAD - addir11,r1,STACK_FRAME_OVERHEAD - stw r11,PT_REGS(r12) + stw r3,PT_REGS(r12) #ifdef CONFIG_PPC_BOOK3S_32 kuep_lock r11, r12 #endif @@ -228,7 +228,6 @@ ret_from_kernel_thread: */ .globl handle_page_fault handle_page_fault: - addir3,r1,STACK_FRAME_OVERHEAD bl do_page_fault cmpwi r3,0 beq+ret_from_except diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 087445e45489..4d638d760a96 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -183,7 +183,6 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) #define EXCEPTION(n, label, hdlr, xfer)\ START_EXCEPTION(n, label) \ EXCEPTION_PROLOG label; \ - addir3,r1,STACK_FRAME_OVERHEAD; \ xfer(n, hdlr) #define EXC_XFER_TEMPLATE(hdlr, trap, msr, tfer, ret) \ @@ -215,7 +214,6 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) lwz r1, emergency_ctx@l(r1) addir1, r1, THREAD_SIZE - INT_FRAME_SIZE EXCEPTION_PROLOG_2 vmap_stack_overflow - addir3, r1, STACK_FRAME_OVERHEAD EXC_XFER_STD(0, stack_overflow_exception) .endm diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 86883ccb3dc5..08563d4170c6 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -185,7 +185,6 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) #define CRITICAL_EXCEPTION(n, label, hdlr) \ START_EXCEPTION(n, label); \ CRITICAL_EXCEPTION_PROLOG label;\ - addir3,r1,STACK_FRAME_OVERHEAD; \ EXC_XFER_TEMPLATE(hdlr, n+2, (MSR_KERNEL & ~(MSR_ME|MSR_DE|MSR_CE)), \ crit_transfer_to_handler, ret_from_crit_exc) @@ -227,13 +226,11 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) /* 0x0600 - Alignment Exception */ START_EXCEPTION(0x0600, Alignment) EXCEPTION_PROLOG Alignment handle_dar_dsisr=1 - addir3,r1,STACK_FRAME_OVERHEAD EXC_XFER_STD(0x600, alignment_exception) /* 0x0700 - Program Exception */ START_EXCEPTION(0x0700, ProgramCheck) EXCEPTION_PROLOG ProgramCheck handle_dar_dsisr=1 - addir3,r1,STACK_FRAME_OVERHEAD EXC_XFER_STD(0x700, program_check_exception) EXCEPTION(0x0800, Trap_08, unknown_exception, EXC_XFER_STD) @@ -494,7 +491,6 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) /* continue normal handling for a critical exception... */ 2: mfspr r4,SPRN_DBSR stw r4,_ESR(r11)/* DebugException takes DBSR in _ESR */ - addir3,r1,STACK_FRAME_OVERHEAD EXC_XFER_TEMPLATE(DebugException, 0x2002, \ (MSR_KERNEL & ~(MSR_ME|MSR_DE|MSR_CE)), \ crit_transfer_to_handler, ret_from_crit_exc) @@ -505,21 +501,18 @@ Decrementer: EXCEPTION_PROLOG Decrementer lis r0,TSR_PIS@h mtspr SPRN_TSR,r0 /* Clear the PIT exception */ - addir3,r1,STACK_FRAME_OVERHEAD EXC_XFER_LITE(0x1000, timer_interrupt) /* Fixed Interval Timer (FIT) Exception. (from 0x1010) */ __HEAD FITException: EXCEPTION_PROLOG FITException - addir3,r1,STACK_FRAME_OVERHEAD; EXC_XFER_STD(0x1010, unknown_exception) /* Watchdog Timer (WDT) Exception. (from 0x1020) */ __HEAD WDTException: CRITICAL_EXCEPTION_PROLOG WDTException - addir3,r1,STACK_FRAME_OVERHEAD; EXC_XFER_TEMPLATE(WatchdogException, 0x1020+2, (MSR_KERNEL & ~(MSR_ME|MSR_DE|MSR_CE)),
[RFC PATCH v1 24/41] powerpc/32: Replace ASM exception exit by C exception exit from ppc64
This patch replaces the PPC32 ASM exceptio exit by C exception exit. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 466 + 1 file changed, 122 insertions(+), 344 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index bbce2de4c6a8..9a75ccb800e7 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -130,9 +130,7 @@ transfer_to_handler_cont: stw r12,TI_LOCAL_FLAGS(r2) lwz r9,_MSR(r11)/* if sleeping, clear MSR.EE */ rlwinm r9,r9,0,~MSR_EE - lwz r12,_LINK(r11) /* and return to address in LR */ - kuap_restore r11, r2, r3, r4, r5 - lwz r2, GPR2(r11) + stw r9,_MSR(r11) b fast_exception_return #endif _ASM_NOKPROBE_SYMBOL(transfer_to_handler) @@ -334,69 +332,20 @@ END_FTR_SECTION_IFSET(CPU_FTR_SPE) .globl fast_exception_return fast_exception_return: + lwz r6,_MSR(r1) + andi. r0,r6,MSR_PR + bne .Lfast_user_interrupt_return + li r3,0 /* 0 return value, no EMULATE_STACK_STORE */ #if !(defined(CONFIG_4xx) || defined(CONFIG_BOOKE)) - andi. r10,r9,MSR_RI /* check for recoverable interrupt */ - beq 1f /* if not, we've got problems */ -#endif - -2: REST_4GPRS(3, r11) - lwz r10,_CCR(r11) - REST_GPR(1, r11) - mtcrr10 - lwz r10,_LINK(r11) - mtlrr10 - /* Clear the exception_marker on the stack to avoid confusing stacktrace */ - li r10, 0 - stw r10, 8(r11) - REST_GPR(10, r11) -#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PERF_EVENTS) - mtspr SPRN_NRI, r0 -#endif - mtspr SPRN_SRR1,r9 - mtspr SPRN_SRR0,r12 - REST_GPR(9, r11) - REST_GPR(12, r11) - lwz r11,GPR11(r11) - rfi -#ifdef CONFIG_40x - b . /* Prevent prefetch past rfi */ -#endif -_ASM_NOKPROBE_SYMBOL(fast_exception_return) - -#if !(defined(CONFIG_4xx) || defined(CONFIG_BOOKE)) -/* check if the exception happened in a restartable section */ -1: lis r3,exc_exit_restart_end@ha - addir3,r3,exc_exit_restart_end@l - cmplw r12,r3 - bge 3f - lis r4,exc_exit_restart@ha - addir4,r4,exc_exit_restart@l - cmplw r12,r4 - blt 3f - lis r3,fee_restarts@ha - tophys(r3,r3) - lwz r5,fee_restarts@l(r3) - addir5,r5,1 - stw r5,fee_restarts@l(r3) - mr r12,r4 /* restart at exc_exit_restart */ - b 2b - - .section .bss - .align 2 -fee_restarts: - .space 4 - .previous - -/* aargh, a nonrecoverable interrupt, panic */ -/* aargh, we don't know which trap this is */ -3: - li r10,-1 - stw r10,_TRAP(r11) + andi. r0,r6,MSR_RI + bne+.Lfast_kernel_interrupt_return addir3,r1,STACK_FRAME_OVERHEAD - bl transfer_to_handler_full bl unrecoverable_exception - b ret_from_except + trap/* should not get here */ +#else + b .Lfast_kernel_interrupt_return #endif +_ASM_NOKPROBE_SYMBOL(fast_exception_return) .globl ret_from_except_full ret_from_except_full: @@ -405,213 +354,141 @@ ret_from_except_full: .globl ret_from_except ret_from_except: - /* Hard-disable interrupts so that current_thread_info()->flags -* can't change between when we test it and when we return -* from the interrupt. */ - /* Note: We don't bother telling lockdep about it */ - LOAD_REG_IMMEDIATE(r10,MSR_KERNEL) - mtmsr r10 /* disable interrupts */ - - lwz r3,_MSR(r1) /* Returning to user mode? */ - andi. r0,r3,MSR_PR - beq resume_kernel - -user_exc_return: /* r10 contains MSR_KERNEL here */ - /* Check current_thread_info()->flags */ - lwz r9,TI_FLAGS(r2) - andi. r0,r9,_TIF_USER_WORK_MASK - bne do_work - -restore_user: -#if defined(CONFIG_4xx) || defined(CONFIG_BOOKE) - /* Check whether this process has its own DBCR0 value. The internal - debug mode bit tells us that dbcr0 should be loaded. */ - lwz r0,THREAD+THREAD_DBCR0(r2) - andis. r10,r0,DBCR0_IDM@h - bnel- load_dbcr0 -#endif - ACCOUNT_CPU_USER_EXIT(r2, r10, r11) +_ASM_NOKPROBE_SYMBOL(ret_from_except) + + .globl interrupt_return +interrupt_return: + lwz r4,_MSR(r1) + andi. r0,r4,MSR_PR + beq .Lkernel_interrupt_return + addir3,r1,STACK_FRAME_OVERHEAD + bl interrupt_exit_user_prepare + cmpwi r3,0 + bne-.Lrestore_nvgprs + +.Lfast_user_interrupt_return: #ifdef CONFIG_PPC_BOOK3S_32 kuep_unlock r10, r11 #endif + kuap_check r2, r4 + lwz r11,_NIP(r1) +
[RFC PATCH v1 23/41] powerpc/32: Always save non volatile registers on exception entry
In preparation of handling exception entry and exit in C,
in order to simplify the handling, always save non volatile registers
when entering an exception.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/ptrace.h | 13 -
arch/powerpc/kernel/entry_32.S| 13 +
arch/powerpc/kernel/head_32.h | 3 +--
arch/powerpc/kernel/head_booke.h | 2 +-
4 files changed, 7 insertions(+), 24 deletions(-)
diff --git a/arch/powerpc/include/asm/ptrace.h
b/arch/powerpc/include/asm/ptrace.h
index 2c842b11a924..979bd0d4d371 100644
--- a/arch/powerpc/include/asm/ptrace.h
+++ b/arch/powerpc/include/asm/ptrace.h
@@ -194,7 +194,6 @@ static inline void regs_set_return_value(struct pt_regs
*regs, unsigned long rc)
#define FULL_REGS(regs)(((regs)->trap & 1) == 0)
#define SET_FULL_REGS(regs)((regs)->trap |= 1)
#endif
-#define CHECK_FULL_REGS(regs) BUG_ON(!FULL_REGS(regs))
#define NV_REG_POISON 0xdeadbeefdeadbeefUL
#else
/*
@@ -204,20 +203,16 @@ static inline void regs_set_return_value(struct pt_regs
*regs, unsigned long rc)
* On 4xx we use the next bit to indicate whether the exception
* is a critical exception (1 means it is).
*/
-#define TRAP_FLAGS_MASK0x1F
+#define TRAP_FLAGS_MASK0x1e
#define TRAP(regs) ((regs)->trap & ~TRAP_FLAGS_MASK)
-#define FULL_REGS(regs)(((regs)->trap & 1) == 0)
-#define SET_FULL_REGS(regs)((regs)->trap |= 1)
+#define FULL_REGS(regs)true
+#define SET_FULL_REGS(regs)do { } while (0)
#define IS_CRITICAL_EXC(regs) (((regs)->trap & 2) != 0)
#define IS_MCHECK_EXC(regs)(((regs)->trap & 4) != 0)
#define IS_DEBUG_EXC(regs) (((regs)->trap & 8) != 0)
#define NV_REG_POISON 0xdeadbeef
-#define CHECK_FULL_REGS(regs)\
-do { \
- if ((regs)->trap & 1) \
- printk(KERN_CRIT "%s: partial register set\n", __func__); \
-} while (0)
#endif /* __powerpc64__ */
+#define CHECK_FULL_REGS(regs) BUG_ON(!FULL_REGS(regs))
static inline void set_trap(struct pt_regs *regs, unsigned long val)
{
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index e448ee3f5f15..bbce2de4c6a8 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -81,12 +81,12 @@ _ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler)
*/
.globl transfer_to_handler_full
transfer_to_handler_full:
- SAVE_NVGPRS(r11)
_ASM_NOKPROBE_SYMBOL(transfer_to_handler_full)
/* fall through */
.globl transfer_to_handler
transfer_to_handler:
+ SAVE_NVGPRS(r11)
stw r2,GPR2(r11)
stw r12,_NIP(r11)
stw r9,_MSR(r11)
@@ -234,10 +234,6 @@ handle_page_fault:
bl do_page_fault
cmpwi r3,0
beq+ret_from_except
- SAVE_NVGPRS(r1)
- lwz r0,_TRAP(r1)
- clrrwi r0,r0,1
- stw r0,_TRAP(r1)
mr r4,r3 /* err arg for bad_page_fault */
addir3,r1,STACK_FRAME_OVERHEAD
bl __bad_page_fault
@@ -810,13 +806,6 @@ recheck:
do_user_signal:/* r10 contains MSR_KERNEL here */
ori r10,r10,MSR_EE
mtmsr r10 /* hard-enable interrupts */
- /* save r13-r31 in the exception frame, if not already done */
- lwz r3,_TRAP(r1)
- andi. r0,r3,1
- beq 2f
- SAVE_NVGPRS(r1)
- rlwinm r3,r3,0,0,30
- stw r3,_TRAP(r1)
2: addir3,r1,STACK_FRAME_OVERHEAD
mr r4,r9
bl do_notify_resume
diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h
index e09585b88ba7..087445e45489 100644
--- a/arch/powerpc/kernel/head_32.h
+++ b/arch/powerpc/kernel/head_32.h
@@ -198,7 +198,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt)
ret_from_except_full)
#define EXC_XFER_LITE(n, hdlr) \
- EXC_XFER_TEMPLATE(hdlr, n+1, MSR_KERNEL, transfer_to_handler, \
+ EXC_XFER_TEMPLATE(hdlr, n, MSR_KERNEL, transfer_to_handler, \
ret_from_except)
.macro vmap_stack_overflow_exception
@@ -215,7 +215,6 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt)
lwz r1, emergency_ctx@l(r1)
addir1, r1, THREAD_SIZE - INT_FRAME_SIZE
EXCEPTION_PROLOG_2 vmap_stack_overflow
- SAVE_NVGPRS(r11)
addir3, r1, STACK_FRAME_OVERHEAD
EXC_XFER_STD(0, stack_overflow_exception)
.endm
diff --git a/arch/powerpc/kernel/head_booke.h b/arch/powerpc/kernel/head_booke.h
index 4286f04648f8..c88c1c474381 100644
--- a/arch/powerpc/kernel/head_booke.h
+++ b/arch/powerpc/kernel/head_booke.h
@@ -331,7 +331,7 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV)
ret_from_except_full)
#define
[RFC PATCH v1 22/41] powerpc/32: Perform normal function call in exception entry
Now that the MMU is re-enabled before calling the transfer function, we don't need anymore that hack with the address of the handler and the return function sitting just after the 'bl' to the transfer fonction, that that function is retrieving via a read relative to 'lr'. Do a regular call to the transfer function, then to the handler, then branch to the return function. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 14 -- arch/powerpc/kernel/head_32.h| 4 ++-- arch/powerpc/kernel/head_booke.h | 6 +++--- 3 files changed, 9 insertions(+), 15 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index dce0f03a0e88..e448ee3f5f15 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -104,7 +104,7 @@ transfer_to_handler: #ifdef CONFIG_PPC_BOOK3S_32 kuep_lock r11, r12 #endif - b 3f + blr 2: /* if from kernel, check interrupted DOZE/NAP mode and * check for stack overflow @@ -119,13 +119,7 @@ transfer_to_handler: #endif /* CONFIG_PPC_BOOK3S_32 || CONFIG_E500 */ .globl transfer_to_handler_cont transfer_to_handler_cont: -3: - mflrr9 - lwz r11,0(r9) /* virtual address of handler */ - lwz r9,4(r9)/* where to go when done */ - mtctr r11 - mtlrr9 - bctr/* jump to handler */ + blr #if defined (CONFIG_PPC_BOOK3S_32) || defined(CONFIG_E500) 4: rlwinm r12,r12,0,~_TLF_NAPPING @@ -404,8 +398,8 @@ fee_restarts: stw r10,_TRAP(r11) addir3,r1,STACK_FRAME_OVERHEAD bl transfer_to_handler_full - .long unrecoverable_exception - .long ret_from_except + bl unrecoverable_exception + b ret_from_except #endif .globl ret_from_except_full diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 160ebd573c37..e09585b88ba7 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -190,8 +190,8 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) li r10,trap; \ stw r10,_TRAP(r11); \ bl tfer; \ - .long hdlr; \ - .long ret + bl hdlr; \ + b ret #define EXC_XFER_STD(n, hdlr) \ EXC_XFER_TEMPLATE(hdlr, n, MSR_KERNEL, transfer_to_handler_full, \ diff --git a/arch/powerpc/kernel/head_booke.h b/arch/powerpc/kernel/head_booke.h index a57a312c7f86..4286f04648f8 100644 --- a/arch/powerpc/kernel/head_booke.h +++ b/arch/powerpc/kernel/head_booke.h @@ -322,9 +322,9 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV) #define EXC_XFER_TEMPLATE(hdlr, trap, msr, tfer, ret) \ li r10,trap; \ stw r10,_TRAP(r11); \ - bl tfer; \ - .long hdlr; \ - .long ret + bl tfer; \ + bl hdlr; \ + b ret;\ #define EXC_XFER_STD(n, hdlr) \ EXC_XFER_TEMPLATE(hdlr, n, MSR_KERNEL, transfer_to_handler_full, \ -- 2.25.0
[RFC PATCH v1 21/41] powerpc/32: Refactor booke critical registers saving
Refactor booke critical registers saving into a few macros and move it into the exception prolog directly. Keep the dedicated transfert_to_handler entry point for the moment allthough they are empty. They will be removed in a later patch to reduce churn. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 33 - arch/powerpc/kernel/head_booke.h | 41 2 files changed, 41 insertions(+), 33 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index f244e7846031..dce0f03a0e88 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -51,49 +51,16 @@ #ifdef CONFIG_BOOKE .globl mcheck_transfer_to_handler mcheck_transfer_to_handler: - mfspr r0,SPRN_DSRR0 - stw r0,_DSRR0(r11) - mfspr r0,SPRN_DSRR1 - stw r0,_DSRR1(r11) /* fall through */ _ASM_NOKPROBE_SYMBOL(mcheck_transfer_to_handler) .globl debug_transfer_to_handler debug_transfer_to_handler: - mfspr r0,SPRN_CSRR0 - stw r0,_CSRR0(r11) - mfspr r0,SPRN_CSRR1 - stw r0,_CSRR1(r11) /* fall through */ _ASM_NOKPROBE_SYMBOL(debug_transfer_to_handler) .globl crit_transfer_to_handler crit_transfer_to_handler: -#ifdef CONFIG_PPC_BOOK3E_MMU - mfspr r0,SPRN_MAS0 - stw r0,MAS0(r11) - mfspr r0,SPRN_MAS1 - stw r0,MAS1(r11) - mfspr r0,SPRN_MAS2 - stw r0,MAS2(r11) - mfspr r0,SPRN_MAS3 - stw r0,MAS3(r11) - mfspr r0,SPRN_MAS6 - stw r0,MAS6(r11) -#ifdef CONFIG_PHYS_64BIT - mfspr r0,SPRN_MAS7 - stw r0,MAS7(r11) -#endif /* CONFIG_PHYS_64BIT */ -#endif /* CONFIG_PPC_BOOK3E_MMU */ -#ifdef CONFIG_44x - mfspr r0,SPRN_MMUCR - stw r0,MMUCR(r11) -#endif - mfspr r0,SPRN_SRR0 - stw r0,_SRR0(r11) - mfspr r0,SPRN_SRR1 - stw r0,_SRR1(r11) - /* fall through */ _ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler) #endif diff --git a/arch/powerpc/kernel/head_booke.h b/arch/powerpc/kernel/head_booke.h index ff93ed519b3b..a57a312c7f86 100644 --- a/arch/powerpc/kernel/head_booke.h +++ b/arch/powerpc/kernel/head_booke.h @@ -229,6 +229,36 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV) SAVE_4GPRS(3, r11); \ SAVE_2GPRS(7, r11) +.macro SAVE_xSRR xSRR + mfspr r0,SPRN_\xSRR\()0 + stw r0,_\xSRR\()0(r1) + mfspr r0,SPRN_\xSRR\()1 + stw r0,_\xSRR\()1(r1) +.endm + +.macro SAVE_MMU_REGS +#ifdef CONFIG_PPC_BOOK3E_MMU + mfspr r0,SPRN_MAS0 + stw r0,MAS0(r1) + mfspr r0,SPRN_MAS1 + stw r0,MAS1(r1) + mfspr r0,SPRN_MAS2 + stw r0,MAS2(r1) + mfspr r0,SPRN_MAS3 + stw r0,MAS3(r1) + mfspr r0,SPRN_MAS6 + stw r0,MAS6(r1) +#ifdef CONFIG_PHYS_64BIT + mfspr r0,SPRN_MAS7 + stw r0,MAS7(r1) +#endif /* CONFIG_PHYS_64BIT */ +#endif /* CONFIG_PPC_BOOK3E_MMU */ +#ifdef CONFIG_44x + mfspr r0,SPRN_MMUCR + stw r0,MMUCR(r1) +#endif +.endm + #define CRITICAL_EXCEPTION_PROLOG(intno) \ EXC_LEVEL_EXCEPTION_PROLOG(CRIT, intno, SPRN_CSRR0, SPRN_CSRR1) #define DEBUG_EXCEPTION_PROLOG \ @@ -271,6 +301,8 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV) START_EXCEPTION(label); \ CRITICAL_EXCEPTION_PROLOG(intno); \ addir3,r1,STACK_FRAME_OVERHEAD; \ + SAVE_MMU_REGS; \ + SAVE_xSRR SRR; \ EXC_XFER_TEMPLATE(hdlr, n+2, (MSR_KERNEL & ~(MSR_ME|MSR_DE|MSR_CE)), \ crit_transfer_to_handler, ret_from_crit_exc) @@ -280,6 +312,10 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV) mfspr r5,SPRN_ESR;\ stw r5,_ESR(r11); \ addir3,r1,STACK_FRAME_OVERHEAD; \ + SAVE_xSRR DSRR; \ + SAVE_xSRR CSRR; \ + SAVE_MMU_REGS; \ + SAVE_xSRR SRR; \ EXC_XFER_TEMPLATE(hdlr, n+4, (MSR_KERNEL & ~(MSR_ME|MSR_DE|MSR_CE)), \ mcheck_transfer_to_handler, ret_from_mcheck_exc) @@ -363,6 +399,9 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_EMB_HV) 2: mfspr r4,SPRN_DBSR; \ stw r4,_ESR(r11); /* DebugException takes DBSR in _ESR */\ addir3,r1,STACK_FRAME_OVERHEAD; \ + SAVE_xSRR CSRR;
[RFC PATCH v1 20/41] powerpc/32: Provide a name to exception prolog continuation in virtual mode
Now that the prolog continuation is separated in .text, give it a name and mark it _ASM_NOKPROBE_SYMBOL. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.h| 12 +++- arch/powerpc/kernel/head_40x.S | 22 -- arch/powerpc/kernel/head_8xx.S | 10 +- arch/powerpc/kernel/head_book3s_32.S | 14 +++--- 4 files changed, 31 insertions(+), 27 deletions(-) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 3c0aa4538514..160ebd573c37 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -10,10 +10,10 @@ * We assume sprg3 has the physical address of the current * task's thread_struct. */ -.macro EXCEPTION_PROLOG handle_dar_dsisr=0 +.macro EXCEPTION_PROLOGname handle_dar_dsisr=0 EXCEPTION_PROLOG_0 handle_dar_dsisr=\handle_dar_dsisr EXCEPTION_PROLOG_1 - EXCEPTION_PROLOG_2 handle_dar_dsisr=\handle_dar_dsisr + EXCEPTION_PROLOG_2 \name handle_dar_dsisr=\handle_dar_dsisr .endm .macro EXCEPTION_PROLOG_0 handle_dar_dsisr=0 @@ -56,7 +56,7 @@ #endif .endm -.macro EXCEPTION_PROLOG_2 handle_dar_dsisr=0 +.macro EXCEPTION_PROLOG_2 name handle_dar_dsisr=0 #ifdef CONFIG_PPC_8xx .if \handle_dar_dsisr li r11, RPN_PATTERN @@ -72,6 +72,7 @@ rfi .text +\name\()_virt: 1: stw r11,GPR1(r1) stw r11,0(r1) @@ -109,6 +110,7 @@ stw r10,8(r11) SAVE_4GPRS(3, r11) SAVE_2GPRS(7, r11) +_ASM_NOKPROBE_SYMBOL(\name\()_virt) .endm .macro SYSCALL_ENTRY trapno @@ -180,7 +182,7 @@ #define EXCEPTION(n, label, hdlr, xfer)\ START_EXCEPTION(n, label) \ - EXCEPTION_PROLOG; \ + EXCEPTION_PROLOG label; \ addir3,r1,STACK_FRAME_OVERHEAD; \ xfer(n, hdlr) @@ -212,7 +214,7 @@ #endif lwz r1, emergency_ctx@l(r1) addir1, r1, THREAD_SIZE - INT_FRAME_SIZE - EXCEPTION_PROLOG_2 + EXCEPTION_PROLOG_2 vmap_stack_overflow SAVE_NVGPRS(r11) addir3, r1, STACK_FRAME_OVERHEAD EXC_XFER_STD(0, stack_overflow_exception) diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index e7d8856714d3..86883ccb3dc5 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -104,7 +104,7 @@ _ENTRY(crit_esr) * Instead we use a couple of words of memory at low physical addresses. * This is OK since we don't support SMP on these processors. */ -.macro CRITICAL_EXCEPTION_PROLOG +.macro CRITICAL_EXCEPTION_PROLOG name stw r10,crit_r10@l(0) /* save two registers to work with */ stw r11,crit_r11@l(0) mfspr r10,SPRN_SRR0 @@ -135,6 +135,7 @@ _ENTRY(crit_esr) .text 1: +\name\()_virt: lwz r11,crit_r1@l(0) stw r11,GPR1(r1) stw r11,0(r1) @@ -162,6 +163,7 @@ _ENTRY(crit_esr) stw r10, 8(r11) SAVE_4GPRS(3, r11) SAVE_2GPRS(7, r11) +_ASM_NOKPROBE_SYMBOL(\name\()_virt) .endm /* @@ -182,7 +184,7 @@ _ENTRY(crit_esr) */ #define CRITICAL_EXCEPTION(n, label, hdlr) \ START_EXCEPTION(n, label); \ - CRITICAL_EXCEPTION_PROLOG; \ + CRITICAL_EXCEPTION_PROLOG label;\ addir3,r1,STACK_FRAME_OVERHEAD; \ EXC_XFER_TEMPLATE(hdlr, n+2, (MSR_KERNEL & ~(MSR_ME|MSR_DE|MSR_CE)), \ crit_transfer_to_handler, ret_from_crit_exc) @@ -205,7 +207,7 @@ _ENTRY(crit_esr) * if they can't resolve the lightweight TLB fault. */ START_EXCEPTION(0x0300, DataStorage) - EXCEPTION_PROLOG handle_dar_dsisr=1 + EXCEPTION_PROLOG DataStorage handle_dar_dsisr=1 EXC_XFER_LITE(0x300, handle_page_fault) /* @@ -213,7 +215,7 @@ _ENTRY(crit_esr) * This is caused by a fetch from non-execute or guarded pages. */ START_EXCEPTION(0x0400, InstructionAccess) - EXCEPTION_PROLOG + EXCEPTION_PROLOG InstructionAccess li r5,0 stw r5, _ESR(r11) /* Zero ESR */ stw r12, _DEAR(r11) /* SRR0 as DEAR */ @@ -224,13 +226,13 @@ _ENTRY(crit_esr) /* 0x0600 - Alignment Exception */ START_EXCEPTION(0x0600, Alignment) - EXCEPTION_PROLOG handle_dar_dsisr=1 + EXCEPTION_PROLOG Alignment handle_dar_dsisr=1 addir3,r1,STACK_FRAME_OVERHEAD EXC_XFER_STD(0x600, alignment_exception) /* 0x0700 - Program Exception */ START_EXCEPTION(0x0700, ProgramCheck) - EXCEPTION_PROLOG handle_dar_dsisr=1 + EXCEPTION_PROLOG ProgramCheck handle_dar_dsisr=1 addir3,r1,STACK_FRAME_OVERHEAD EXC_XFER_STD(0x700, program_check_exception) @@ -450,7 +452,7 @@ _ENTRY(crit_esr)
[RFC PATCH v1 19/41] powerpc/32: Move exception prolog code into .text once MMU is back on
The space in the head section is rather constrained by the fact that exception vectors are spread every 0x100 bytes and sometimes we need to have "out of line" code because it doesn't fit. Now that we are enabling MMU early in the prolog, take that opportunity to jump somewhere else in the .text section where we don't have any space constraint. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.h| 5 arch/powerpc/kernel/head_40x.S | 6 + arch/powerpc/kernel/head_8xx.S | 25 arch/powerpc/kernel/head_book3s_32.S | 34 4 files changed, 36 insertions(+), 34 deletions(-) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index d97ec94b34da..3c0aa4538514 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -70,6 +70,8 @@ mtspr SPRN_SRR0, r11 mfspr r11, SPRN_SPRG_SCRATCH2 rfi + + .text 1: stw r11,GPR1(r1) stw r11,0(r1) @@ -163,12 +165,14 @@ */ #ifdef CONFIG_PPC_BOOK3S #defineSTART_EXCEPTION(n, label) \ + __HEAD; \ . = n; \ DO_KVM n; \ label: #else #defineSTART_EXCEPTION(n, label) \ + __HEAD; \ . = n; \ label: @@ -196,6 +200,7 @@ ret_from_except) .macro vmap_stack_overflow_exception + __HEAD vmap_stack_overflow: #ifdef CONFIG_SMP mfspr r1, SPRN_SPRG_THREAD diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index c14a71e0d6d3..e7d8856714d3 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -132,6 +132,8 @@ _ENTRY(crit_esr) ori r11, r11, 1f@l mtspr SPRN_SRR0, r11 rfi + + .text 1: lwz r11,crit_r1@l(0) stw r11,GPR1(r1) @@ -496,6 +498,7 @@ _ENTRY(crit_esr) crit_transfer_to_handler, ret_from_crit_exc) /* Programmable Interval Timer (PIT) Exception. (from 0x1000) */ + __HEAD Decrementer: EXCEPTION_PROLOG lis r0,TSR_PIS@h @@ -504,12 +507,14 @@ Decrementer: EXC_XFER_LITE(0x1000, timer_interrupt) /* Fixed Interval Timer (FIT) Exception. (from 0x1010) */ + __HEAD FITException: EXCEPTION_PROLOG addir3,r1,STACK_FRAME_OVERHEAD; EXC_XFER_STD(0x1010, unknown_exception) /* Watchdog Timer (WDT) Exception. (from 0x1020) */ + __HEAD WDTException: CRITICAL_EXCEPTION_PROLOG; addir3,r1,STACK_FRAME_OVERHEAD; @@ -523,6 +528,7 @@ WDTException: * reserved. */ + __HEAD /* Damn, I came up one instruction too many to fit into the * exception space :-). Both the instruction and data TLB * miss get to this point to load the TLB. diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index 11789a077d76..d16d0ec71bb2 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -133,7 +133,7 @@ instruction_counter: START_EXCEPTION(0x600, Alignment) EXCEPTION_PROLOG handle_dar_dsisr=1 addir3,r1,STACK_FRAME_OVERHEAD - b .Lalignment_exception_ool + EXC_XFER_STD(0x600, alignment_exception) /* Program check exception */ EXCEPTION(0x700, ProgramCheck, program_check_exception, EXC_XFER_STD) @@ -141,11 +141,6 @@ instruction_counter: /* Decrementer */ EXCEPTION(0x900, Decrementer, timer_interrupt, EXC_XFER_LITE) - /* With VMAP_STACK there's not enough room for this at 0x600 */ - . = 0xa00 -.Lalignment_exception_ool: - EXC_XFER_STD(0x600, alignment_exception) - /* System call */ START_EXCEPTION(0xc00, SystemCall) SYSCALL_ENTRY 0xc00 @@ -339,26 +334,25 @@ DARFixed:/* Return from dcbx instruction bug workaround */ * support of breakpoints and such. Someday I will get around to * using them. */ -do_databreakpoint: - EXCEPTION_PROLOG_1 - EXCEPTION_PROLOG_2 handle_dar_dsisr=1 - addir3,r1,STACK_FRAME_OVERHEAD - mfspr r4,SPRN_BAR - stw r4,_DAR(r11) - EXC_XFER_STD(0x1c00, do_break) - START_EXCEPTION(0x1c00, DataBreakpoint) EXCEPTION_PROLOG_0 handle_dar_dsisr=1 mfspr r11, SPRN_SRR0 cmplwi cr1, r11, (.Ldtlbie - PAGE_OFFSET)@l cmplwi cr7, r11, (.Litlbie - PAGE_OFFSET)@l cror4*cr1+eq, 4*cr1+eq, 4*cr7+eq - bne cr1, do_databreakpoint + bne cr1, 1f mtcrr10 mfspr r10, SPRN_SPRG_SCRATCH0 mfspr r11, SPRN_SPRG_SCRATCH1 rfi +1: EXCEPTION_PROLOG_1 + EXCEPTION_PROLOG_2 handle_dar_dsisr=1 + addir3,r1,STACK_FRAME_OVERHEAD + mfspr r4,SPRN_BAR
[RFC PATCH v1 18/41] powerpc/32: Use START_EXCEPTION() as much as possible
Everywhere where it is possible, use START_EXCEPTION(). This will help for proper exception init in future patches. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_40x.S | 12 +-- arch/powerpc/kernel/head_8xx.S | 27 + arch/powerpc/kernel/head_book3s_32.S | 30 3 files changed, 22 insertions(+), 47 deletions(-) diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 55fa99c5085c..c14a71e0d6d3 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -247,17 +247,15 @@ _ENTRY(crit_esr) EXCEPTION(0x0F00, Trap_0F, unknown_exception, EXC_XFER_STD) /* 0x1000 - Programmable Interval Timer (PIT) Exception */ - . = 0x1000 + START_EXCEPTION(0x1000, DecrementerTrap) b Decrementer -/* 0x1010 - Fixed Interval Timer (FIT) Exception -*/ - . = 0x1010 +/* 0x1010 - Fixed Interval Timer (FIT) Exception */ + START_EXCEPTION(0x1010, FITExceptionTrap) b FITException -/* 0x1020 - Watchdog Timer (WDT) Exception -*/ - . = 0x1020 +/* 0x1020 - Watchdog Timer (WDT) Exception */ + START_EXCEPTION(0x1020, WDTExceptionTrap) b WDTException /* 0x1100 - Data TLB Miss Exception diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index b63445c55f4d..11789a077d76 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -121,8 +121,7 @@ instruction_counter: EXCEPTION(0x100, Reset, system_reset_exception, EXC_XFER_STD) /* Machine check */ - . = 0x200 -MachineCheck: + START_EXCEPTION(0x200, MachineCheck) EXCEPTION_PROLOG handle_dar_dsisr=1 addi r3,r1,STACK_FRAME_OVERHEAD EXC_XFER_STD(0x200, machine_check_exception) @@ -131,8 +130,7 @@ MachineCheck: EXCEPTION(0x500, HardwareInterrupt, do_IRQ, EXC_XFER_LITE) /* Alignment exception */ - . = 0x600 -Alignment: + START_EXCEPTION(0x600, Alignment) EXCEPTION_PROLOG handle_dar_dsisr=1 addir3,r1,STACK_FRAME_OVERHEAD b .Lalignment_exception_ool @@ -149,8 +147,7 @@ Alignment: EXC_XFER_STD(0x600, alignment_exception) /* System call */ - . = 0xc00 -SystemCall: + START_EXCEPTION(0xc00, SystemCall) SYSCALL_ENTRY 0xc00 /* Single step - not used on 601 */ @@ -161,7 +158,6 @@ SystemCall: */ EXCEPTION(0x1000, SoftEmu, emulation_assist_interrupt, EXC_XFER_STD) - . = 0x1100 /* * For the MPC8xx, this is a software tablewalk to load the instruction * TLB. The task switch loads the M_TWB register with the pointer to the first @@ -183,7 +179,7 @@ SystemCall: #define INVALIDATE_ADJACENT_PAGES_CPU15(addr, tmp) #endif -InstructionTLBMiss: + START_EXCEPTION(0x1100, InstructionTLBMiss) mtspr SPRN_SPRG_SCRATCH2, r10 mtspr SPRN_M_TW, r11 @@ -239,8 +235,7 @@ InstructionTLBMiss: rfi #endif - . = 0x1200 -DataStoreTLBMiss: + START_EXCEPTION(0x1200, DataStoreTLBMiss) mtspr SPRN_SPRG_SCRATCH2, r10 mtspr SPRN_M_TW, r11 mfcrr11 @@ -303,8 +298,7 @@ DataStoreTLBMiss: * to many reasons, such as executing guarded memory or illegal instruction * addresses. There is nothing to do but handle a big time error fault. */ - . = 0x1300 -InstructionTLBError: + START_EXCEPTION(0x1300, InstructionTLBError) EXCEPTION_PROLOG andis. r5,r9,DSISR_SRR1_MATCH_32S@h /* Filter relevant SRR1 bits */ andis. r10,r9,SRR1_ISI_NOPT@h @@ -320,8 +314,7 @@ InstructionTLBError: * many reasons, including a dirty update to a pte. We bail out to * a higher level function that can handle it. */ - . = 0x1400 -DataTLBError: + START_EXCEPTION(0x1400, DataTLBError) EXCEPTION_PROLOG_0 handle_dar_dsisr=1 mfspr r11, SPRN_DAR cmpwi cr1, r11, RPN_PATTERN @@ -354,8 +347,7 @@ do_databreakpoint: stw r4,_DAR(r11) EXC_XFER_STD(0x1c00, do_break) - . = 0x1c00 -DataBreakpoint: + START_EXCEPTION(0x1c00, DataBreakpoint) EXCEPTION_PROLOG_0 handle_dar_dsisr=1 mfspr r11, SPRN_SRR0 cmplwi cr1, r11, (.Ldtlbie - PAGE_OFFSET)@l @@ -368,8 +360,7 @@ DataBreakpoint: rfi #ifdef CONFIG_PERF_EVENTS - . = 0x1d00 -InstructionBreakpoint: + START_EXCEPTION(0x1d00, InstructionBreakpoint) mtspr SPRN_SPRG_SCRATCH0, r10 lwz r10, (instruction_counter - PAGE_OFFSET)@l(0) addir10, r10, -1 diff --git a/arch/powerpc/kernel/head_book3s_32.S b/arch/powerpc/kernel/head_book3s_32.S index 9dc05890477d..8f5c8c8da63d 100644 --- a/arch/powerpc/kernel/head_book3s_32.S +++ b/arch/powerpc/kernel/head_book3s_32.S @@ -255,9 +255,7 @@ __secondary_hold_acknowledge: * pointer when we take an exception from supervisor mode.) * -- paulus. */ - . = 0x200 - DO_KVM 0x200 -MachineCheck: +
[RFC PATCH v1 17/41] powerpc/32: Add vmap_stack_overflow label inside the macro
For consistancy, add in the macro the label used by exception prolog to branch to stack overflow processing. While at it, enclose the macro in #ifdef CONFIG_VMAP_STACK on the 8xx as already done on book3s/32. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.h| 3 ++- arch/powerpc/kernel/head_8xx.S | 3 ++- arch/powerpc/kernel/head_book3s_32.S | 1 - 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 15c6fc7cbbf5..d97ec94b34da 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -52,7 +52,7 @@ 1: #ifdef CONFIG_VMAP_STACK mtcrf 0x3f, r1 - bt 32 - THREAD_ALIGN_SHIFT, stack_overflow + bt 32 - THREAD_ALIGN_SHIFT, vmap_stack_overflow #endif .endm @@ -196,6 +196,7 @@ ret_from_except) .macro vmap_stack_overflow_exception +vmap_stack_overflow: #ifdef CONFIG_SMP mfspr r1, SPRN_SPRG_THREAD lwz r1, TASK_CPU - THREAD(r1) diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index cdbfa9d41353..b63445c55f4d 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -338,8 +338,9 @@ DARFixed:/* Return from dcbx instruction bug workaround */ /* 0x300 is DataAccess exception, needed by bad_page_fault() */ EXC_XFER_LITE(0x300, handle_page_fault) -stack_overflow: +#ifdef CONFIG_VMAP_STACK vmap_stack_overflow_exception +#endif /* On the MPC8xx, these next four traps are used for development * support of breakpoints and such. Someday I will get around to diff --git a/arch/powerpc/kernel/head_book3s_32.S b/arch/powerpc/kernel/head_book3s_32.S index 59efbee7c080..9dc05890477d 100644 --- a/arch/powerpc/kernel/head_book3s_32.S +++ b/arch/powerpc/kernel/head_book3s_32.S @@ -729,7 +729,6 @@ fast_hash_page_return: #endif /* CONFIG_PPC_BOOK3S_604 */ #ifdef CONFIG_VMAP_STACK -stack_overflow: vmap_stack_overflow_exception #endif -- 2.25.0
[RFC PATCH v1 16/41] powerpc/32: Statically initialise first emergency context
The check of the emergency context initialisation in
vmap_stack_overflow is buggy for the SMP case, as it
compares r1 with 0 while in the SMP case r1 is offseted
by the CPU id.
Instead of fixing it, just perform static initialisation
of the first emergency context.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/head_32.h | 6 +-
arch/powerpc/kernel/setup_32.c | 2 +-
2 files changed, 2 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h
index 88b02bd91e8e..15c6fc7cbbf5 100644
--- a/arch/powerpc/kernel/head_32.h
+++ b/arch/powerpc/kernel/head_32.h
@@ -205,11 +205,7 @@
lis r1, emergency_ctx@ha
#endif
lwz r1, emergency_ctx@l(r1)
- cmpwi cr1, r1, 0
- bne cr1, 1f
- lis r1, init_thread_union@ha
- addir1, r1, init_thread_union@l
-1: addir1, r1, THREAD_SIZE - INT_FRAME_SIZE
+ addir1, r1, THREAD_SIZE - INT_FRAME_SIZE
EXCEPTION_PROLOG_2
SAVE_NVGPRS(r11)
addir3, r1, STACK_FRAME_OVERHEAD
diff --git a/arch/powerpc/kernel/setup_32.c b/arch/powerpc/kernel/setup_32.c
index 8ba49a6bf515..d7c1f92152af 100644
--- a/arch/powerpc/kernel/setup_32.c
+++ b/arch/powerpc/kernel/setup_32.c
@@ -164,7 +164,7 @@ void __init irqstack_early_init(void)
}
#ifdef CONFIG_VMAP_STACK
-void *emergency_ctx[NR_CPUS] __ro_after_init;
+void *emergency_ctx[NR_CPUS] __ro_after_init = {[0] = _stack};
void __init emergency_stack_init(void)
{
--
2.25.0
[RFC PATCH v1 15/41] powerpc/32: Enable instruction translation at the same time as data translation
On 40x and 8xx, kernel text is pinned. On book3s/32, kernel text is mapped by BATs. Enable instruction translation at the same time as data translation, it makes things simpler. In syscall handler, MSR_RI can also be set at the same time because srr0/srr1 are already saved and r1 is set properly. On booke, translation is always on, so at the end all PPC32 have translation on early. Just update msr. Also update comment in power_save_ppc32_restore(). Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 30 -- arch/powerpc/kernel/head_32.h| 13 - arch/powerpc/kernel/head_40x.S | 10 +++--- arch/powerpc/kernel/head_booke.h | 6 -- 4 files changed, 31 insertions(+), 28 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 0b6af35acdfd..f244e7846031 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -154,19 +154,11 @@ transfer_to_handler: transfer_to_handler_cont: 3: mflrr9 - tovirt(r9, r9) lwz r11,0(r9) /* virtual address of handler */ lwz r9,4(r9)/* where to go when done */ -#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PERF_EVENTS) - mtspr SPRN_NRI, r0 -#endif - mtspr SPRN_SRR0,r11 - mtspr SPRN_SRR1,r10 + mtctr r11 mtlrr9 - rfi /* jump to handler, enable MMU */ -#ifdef CONFIG_40x - b . /* Prevent prefetch past rfi */ -#endif + bctr/* jump to handler */ #if defined (CONFIG_PPC_BOOK3S_32) || defined(CONFIG_E500) 4: rlwinm r12,r12,0,~_TLF_NAPPING @@ -444,8 +436,6 @@ fee_restarts: li r10,-1 stw r10,_TRAP(r11) addir3,r1,STACK_FRAME_OVERHEAD - lis r10,MSR_KERNEL@h - ori r10,r10,MSR_KERNEL@l bl transfer_to_handler_full .long unrecoverable_exception .long ret_from_except @@ -945,16 +935,20 @@ _GLOBAL(enter_rtas) mtspr SPRN_SRR1,r9 rfi 1: - li r0, MSR_KERNEL & ~MSR_IR/* can take DTLB miss */ - mtmsr r0 - isync + lis r8, 1f@h + ori r8, r8, 1f@l + LOAD_REG_IMMEDIATE(r9,MSR_KERNEL) + mtspr SPRN_SRR0,r8 + mtspr SPRN_SRR1,r9 + rfi /* Reactivate MMU translation */ +1: lwz r8,INT_FRAME_SIZE+4(r1) /* get return address */ lwz r9,8(r1)/* original msr value */ addir1,r1,INT_FRAME_SIZE li r0,0 stw r0, THREAD + RTAS_SP(r2) - mtspr SPRN_SRR0,r8 - mtspr SPRN_SRR1,r9 - rfi /* return to caller */ + mtlrr8 + mtmsr r9 + blr /* return to caller */ _ASM_NOKPROBE_SYMBOL(enter_rtas) #endif /* CONFIG_PPC_RTAS */ diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 910f86642eec..88b02bd91e8e 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -63,10 +63,14 @@ mtspr SPRN_DAR, r11 /* Tag DAR, to be used in DTLB Error */ .endif #endif - LOAD_REG_IMMEDIATE(r11, MSR_KERNEL & ~(MSR_IR | MSR_RI)) /* can take DTLB miss */ - mtmsr r11 - isync + LOAD_REG_IMMEDIATE(r11, MSR_KERNEL & ~MSR_RI) /* re-enable MMU */ + mtspr SPRN_SRR1, r11 + lis r11, 1f@h + ori r11, r11, 1f@l + mtspr SPRN_SRR0, r11 mfspr r11, SPRN_SPRG_SCRATCH2 + rfi +1: stw r11,GPR1(r1) stw r11,0(r1) mr r11, r1 @@ -94,7 +98,7 @@ #elif defined(CONFIG_PPC_8xx) mtspr SPRN_EID, r2/* Set MSR_RI */ #else - li r10, MSR_KERNEL & ~MSR_IR /* can take exceptions */ + li r10, MSR_KERNEL /* can take exceptions */ mtmsr r10 /* (except for mach check in rtas) */ #endif stw r0,GPR0(r11) @@ -179,7 +183,6 @@ #define EXC_XFER_TEMPLATE(hdlr, trap, msr, tfer, ret) \ li r10,trap; \ stw r10,_TRAP(r11); \ - LOAD_REG_IMMEDIATE(r10, msr); \ bl tfer; \ .long hdlr; \ .long ret diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 7da673ec63ef..55fa99c5085c 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -126,9 +126,13 @@ _ENTRY(crit_esr) lwz r11,TASK_STACK-THREAD(r11) /* this thread's kernel stack */ 1: stw r1,crit_r1@l(0) addir1,r11,THREAD_SIZE-INT_FRAME_SIZE /* Alloc an excpt frm */ - LOAD_REG_IMMEDIATE(r11,MSR_KERNEL & ~(MSR_IR | MSR_RI)) - mtmsr r11 -
[RFC PATCH v1 14/41] powerpc/32: Tag DAR in EXCEPTION_PROLOG_2 for the 8xx
8xx requires to tag the DAR with a magic value in order to detect fixup DAR on faults generated by 'dcbX', as the 8xx forgets to update the DAR for those faults. Do the tagging as early as possible, that is before enabling MMU. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.h | 6 ++ arch/powerpc/kernel/head_8xx.S | 18 ++ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 1b707755c68e..910f86642eec 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -57,6 +57,12 @@ .endm .macro EXCEPTION_PROLOG_2 handle_dar_dsisr=0 +#ifdef CONFIG_PPC_8xx + .if \handle_dar_dsisr + li r11, RPN_PATTERN + mtspr SPRN_DAR, r11 /* Tag DAR, to be used in DTLB Error */ + .endif +#endif LOAD_REG_IMMEDIATE(r11, MSR_KERNEL & ~(MSR_IR | MSR_RI)) /* can take DTLB miss */ mtmsr r11 isync diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index 792e2fd86479..cdbfa9d41353 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -30,6 +30,12 @@ #include #include +/* + * Value for the bits that have fixed value in RPN entries. + * Also used for tagging DAR for DTLBerror. + */ +#define RPN_PATTERN0x00f0 + #include "head_32.h" .macro compare_to_kernel_boundary scratch, addr @@ -42,12 +48,6 @@ #endif .endm -/* - * Value for the bits that have fixed value in RPN entries. - * Also used for tagging DAR for DTLBerror. - */ -#define RPN_PATTERN0x00f0 - #define PAGE_SHIFT_512K19 #define PAGE_SHIFT_8M 23 @@ -124,8 +124,6 @@ instruction_counter: . = 0x200 MachineCheck: EXCEPTION_PROLOG handle_dar_dsisr=1 - li r6, RPN_PATTERN - mtspr SPRN_DAR, r6/* Tag DAR, to be used in DTLB Error */ addi r3,r1,STACK_FRAME_OVERHEAD EXC_XFER_STD(0x200, machine_check_exception) @@ -136,8 +134,6 @@ MachineCheck: . = 0x600 Alignment: EXCEPTION_PROLOG handle_dar_dsisr=1 - li r6, RPN_PATTERN - mtspr SPRN_DAR, r6/* Tag DAR, to be used in DTLB Error */ addir3,r1,STACK_FRAME_OVERHEAD b .Lalignment_exception_ool @@ -331,8 +327,6 @@ DataTLBError: cmpwi cr1, r11, RPN_PATTERN beq-cr1, FixupDAR /* must be a buggy dcbX, icbi insn. */ DARFixed:/* Return from dcbx instruction bug workaround */ - li r11, RPN_PATTERN - mtspr SPRN_DAR, r11 /* Tag DAR, to be used in DTLB Error */ EXCEPTION_PROLOG_1 EXCEPTION_PROLOG_2 handle_dar_dsisr=1 lwz r4, _DAR(r11) -- 2.25.0
[RFC PATCH v1 13/41] powerpc/32: Always enable data translation in exception prolog
If the code can use a stack in vm area, it can also use a
stack in linear space.
Simplify code by removing old non VMAP stack code on PPC32.
That means the data translation is now re-enabled early in
exception prolog in all cases, not only when using VMAP stacks.
While we are touching EXCEPTION_PROLOG macros, remove the
unused for_rtas parameter in EXCEPTION_PROLOG_1.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/processor.h | 4 +-
arch/powerpc/kernel/asm-offsets.c| 2 -
arch/powerpc/kernel/entry_32.S | 19 +++
arch/powerpc/kernel/fpu.S| 2 -
arch/powerpc/kernel/head_32.h| 85 +---
arch/powerpc/kernel/head_40x.S | 23
arch/powerpc/kernel/head_8xx.S | 19 +--
arch/powerpc/kernel/head_book3s_32.S | 47 +--
arch/powerpc/kernel/idle_6xx.S | 12 +---
arch/powerpc/kernel/idle_e500.S | 4 +-
arch/powerpc/kernel/vector.S | 2 -
arch/powerpc/mm/book3s32/hash_low.S | 14 -
12 files changed, 17 insertions(+), 216 deletions(-)
diff --git a/arch/powerpc/include/asm/processor.h
b/arch/powerpc/include/asm/processor.h
index 43cbd9281055..eae16facc390 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -147,11 +147,9 @@ struct thread_struct {
#ifdef CONFIG_PPC_RTAS
unsigned long rtas_sp;/* stack pointer for when in RTAS */
#endif
-#endif
#if defined(CONFIG_PPC_BOOK3S_32) && defined(CONFIG_PPC_KUAP)
unsigned long kuap; /* opened segments for user access */
#endif
-#ifdef CONFIG_VMAP_STACK
unsigned long srr0;
unsigned long srr1;
unsigned long dar;
@@ -160,7 +158,7 @@ struct thread_struct {
unsigned long r0, r3, r4, r5, r6, r8, r9, r11;
unsigned long lr, ctr;
#endif
-#endif
+#endif /* CONFIG_PPC32 */
/* Debug Registers */
struct debug_reg debug;
#ifdef CONFIG_PPC_FPU_REGS
diff --git a/arch/powerpc/kernel/asm-offsets.c
b/arch/powerpc/kernel/asm-offsets.c
index 1f47d4e0f3b5..d500ee82212a 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -131,7 +131,6 @@ int main(void)
OFFSET(KSP_VSID, thread_struct, ksp_vsid);
#else /* CONFIG_PPC64 */
OFFSET(PGDIR, thread_struct, pgdir);
-#ifdef CONFIG_VMAP_STACK
OFFSET(SRR0, thread_struct, srr0);
OFFSET(SRR1, thread_struct, srr1);
OFFSET(DAR, thread_struct, dar);
@@ -148,7 +147,6 @@ int main(void)
OFFSET(THLR, thread_struct, lr);
OFFSET(THCTR, thread_struct, ctr);
#endif
-#endif
#ifdef CONFIG_SPE
OFFSET(THREAD_EVR0, thread_struct, evr[0]);
OFFSET(THREAD_ACC, thread_struct, acc);
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 23f46e697546..0b6af35acdfd 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -129,7 +129,7 @@ transfer_to_handler:
stw r12,_CTR(r11)
stw r2,_XER(r11)
mfspr r12,SPRN_SPRG_THREAD
- tovirt_vmstack r12, r12
+ tovirt(r12, r12)
beq 2f /* if from user, fix up THREAD.regs */
addir2, r12, -THREAD
addir11,r1,STACK_FRAME_OVERHEAD
@@ -154,8 +154,7 @@ transfer_to_handler:
transfer_to_handler_cont:
3:
mflrr9
- tovirt_novmstack r2, r2 /* set r2 to current */
- tovirt_vmstack r9, r9
+ tovirt(r9, r9)
lwz r11,0(r9) /* virtual address of handler */
lwz r9,4(r9)/* where to go when done */
#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PERF_EVENTS)
@@ -933,7 +932,6 @@ _GLOBAL(enter_rtas)
lis r6,1f@ha/* physical return address for rtas */
addir6,r6,1f@l
tophys(r6,r6)
- tophys_novmstack r7, r1
lwz r8,RTASENTRY(r4)
lwz r4,RTASBASE(r4)
mfmsr r9
@@ -942,22 +940,19 @@ _GLOBAL(enter_rtas)
mtmsr r0 /* disable interrupts so SRR0/1 don't get trashed */
li r9,MSR_KERNEL & ~(MSR_IR|MSR_DR)
mtlrr6
- stw r7, THREAD + RTAS_SP(r2)
+ stw r1, THREAD + RTAS_SP(r2)
mtspr SPRN_SRR0,r8
mtspr SPRN_SRR1,r9
rfi
-1: tophys_novmstack r9, r1
-#ifdef CONFIG_VMAP_STACK
+1:
li r0, MSR_KERNEL & ~MSR_IR/* can take DTLB miss */
mtmsr r0
isync
-#endif
- lwz r8,INT_FRAME_SIZE+4(r9) /* get return address */
- lwz r9,8(r9)/* original msr value */
+ lwz r8,INT_FRAME_SIZE+4(r1) /* get return address */
+ lwz r9,8(r1)/* original msr value */
addir1,r1,INT_FRAME_SIZE
li r0,0
- tophys_novmstack r7, r2
- stw r0, THREAD + RTAS_SP(r7)
+ stw r0, THREAD + RTAS_SP(r2)
mtspr SPRN_SRR0,r8
mtspr SPRN_SRR1,r9
rfi
[RFC PATCH v1 12/41] powerpc/32: Remove ksp_limit
ksp_limit is there to help detect stack overflows.
That is specific to ppc32 as it was removed from ppc64 in
commit cbc9565ee826 ("powerpc: Remove ksp_limit on ppc64").
There are other means for detecting stack overflows.
As ppc64 has proven to not need it, ppc32 should be able to do
without it too.
Lets remove it and simplify exception handling.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/processor.h | 2 -
arch/powerpc/kernel/asm-offsets.c| 2 -
arch/powerpc/kernel/entry_32.S | 63
arch/powerpc/kernel/head_40x.S | 2 -
arch/powerpc/kernel/head_booke.h | 1 -
arch/powerpc/kernel/misc_32.S| 14 ---
arch/powerpc/kernel/process.c| 3 --
arch/powerpc/kernel/traps.c | 9
arch/powerpc/lib/sstep.c | 9
9 files changed, 105 deletions(-)
diff --git a/arch/powerpc/include/asm/processor.h
b/arch/powerpc/include/asm/processor.h
index 8acc3590c971..43cbd9281055 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -144,7 +144,6 @@ struct thread_struct {
#endif
#ifdef CONFIG_PPC32
void*pgdir; /* root of page-table tree */
- unsigned long ksp_limit; /* if ksp <= ksp_limit stack overflow */
#ifdef CONFIG_PPC_RTAS
unsigned long rtas_sp;/* stack pointer for when in RTAS */
#endif
@@ -282,7 +281,6 @@ struct thread_struct {
#ifdef CONFIG_PPC32
#define INIT_THREAD { \
.ksp = INIT_SP, \
- .ksp_limit = INIT_SP_LIMIT, \
.pgdir = swapper_pg_dir, \
.fpexc_mode = MSR_FE0 | MSR_FE1, \
SPEFSCR_INIT \
diff --git a/arch/powerpc/kernel/asm-offsets.c
b/arch/powerpc/kernel/asm-offsets.c
index b12d7c049bfe..1f47d4e0f3b5 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -91,7 +91,6 @@ int main(void)
DEFINE(SIGSEGV, SIGSEGV);
DEFINE(NMI_MASK, NMI_MASK);
#else
- OFFSET(KSP_LIMIT, thread_struct, ksp_limit);
#ifdef CONFIG_PPC_RTAS
OFFSET(RTAS_SP, thread_struct, rtas_sp);
#endif
@@ -382,7 +381,6 @@ int main(void)
DEFINE(_CSRR1, STACK_INT_FRAME_SIZE+offsetof(struct exception_regs,
csrr1));
DEFINE(_DSRR0, STACK_INT_FRAME_SIZE+offsetof(struct exception_regs,
dsrr0));
DEFINE(_DSRR1, STACK_INT_FRAME_SIZE+offsetof(struct exception_regs,
dsrr1));
- DEFINE(SAVED_KSP_LIMIT, STACK_INT_FRAME_SIZE+offsetof(struct
exception_regs, saved_ksp_limit));
#endif
#endif
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 971980d71b04..23f46e697546 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -94,12 +94,6 @@ crit_transfer_to_handler:
mfspr r0,SPRN_SRR1
stw r0,_SRR1(r11)
- /* set the stack limit to the current stack */
- mfspr r8,SPRN_SPRG_THREAD
- lwz r0,KSP_LIMIT(r8)
- stw r0,SAVED_KSP_LIMIT(r11)
- rlwinm r0,r1,0,0,(31 - THREAD_SHIFT)
- stw r0,KSP_LIMIT(r8)
/* fall through */
_ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler)
#endif
@@ -107,12 +101,6 @@ _ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler)
#ifdef CONFIG_40x
.globl crit_transfer_to_handler
crit_transfer_to_handler:
- /* set the stack limit to the current stack */
- mfspr r8,SPRN_SPRG_THREAD
- lwz r0,KSP_LIMIT(r8)
- stw r0,saved_ksp_limit@l(0)
- rlwinm r0,r1,0,0,(31 - THREAD_SHIFT)
- stw r0,KSP_LIMIT(r8)
/* fall through */
_ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler)
#endif
@@ -156,12 +144,6 @@ transfer_to_handler:
*/
kuap_save_and_lock r11, r12, r9, r2, r6
addir2, r12, -THREAD
-#ifndef CONFIG_VMAP_STACK
- lwz r9,KSP_LIMIT(r12)
- cmplw r1,r9 /* if r1 <= ksp_limit */
- ble-stack_ovf /* then the kernel stack overflowed */
-#endif
-5:
#if defined(CONFIG_PPC_BOOK3S_32) || defined(CONFIG_E500)
lwz r12,TI_LOCAL_FLAGS(r2)
mtcrf 0x01,r12
@@ -204,37 +186,6 @@ transfer_to_handler_cont:
_ASM_NOKPROBE_SYMBOL(transfer_to_handler)
_ASM_NOKPROBE_SYMBOL(transfer_to_handler_cont)
-#ifndef CONFIG_VMAP_STACK
-/*
- * On kernel stack overflow, load up an initial stack pointer
- * and call StackOverflow(regs), which should not return.
- */
-stack_ovf:
- /* sometimes we use a statically-allocated stack, which is OK. */
- lis r12,_end@h
- ori r12,r12,_end@l
- cmplw r1,r12
- ble 5b /* r1 <= &_end is OK */
- SAVE_NVGPRS(r11)
- addir3,r1,STACK_FRAME_OVERHEAD
- lis r1,init_thread_union@ha
- addir1,r1,init_thread_union@l
- addir1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD
- lis r9,StackOverflow@ha
- addir9,r9,StackOverflow@l
- LOAD_REG_IMMEDIATE(r10,MSR_KERNEL)
-#if defined(CONFIG_PPC_8xx)
[RFC PATCH v1 11/41] powerpc/32: Use fast instruction to set MSR RI in exception prolog on 8xx
8xx has registers SPRN_NRI, SPRN_EID and SPRN_EIE for changing MSR EE and RI. Use SPRN_EID in exception prolog to set RI. On an 8xx, it reduces the null_syscall test by 3 cycles. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index ac6b391f1493..25ee6b26ef5a 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -107,6 +107,8 @@ #endif #ifdef CONFIG_40x rlwinm r9,r9,0,14,12 /* clear MSR_WE (necessary?) */ +#elif defined(CONFIG_PPC_8xx) + mtspr SPRN_EID, r2/* Set MSR_RI */ #else #ifdef CONFIG_VMAP_STACK li r10, MSR_KERNEL & ~MSR_IR /* can take exceptions */ -- 2.25.0
[RFC PATCH v1 10/41] powerpc/32: Handle bookE debugging in C in exception entry
The handling of SPRN_DBCR0 and other registers can easily
be done in C instead of ASM.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/interrupt.h | 7 +++
arch/powerpc/kernel/entry_32.S | 23 ---
2 files changed, 7 insertions(+), 23 deletions(-)
diff --git a/arch/powerpc/include/asm/interrupt.h
b/arch/powerpc/include/asm/interrupt.h
index e794111d7c03..d70c761edc00 100644
--- a/arch/powerpc/include/asm/interrupt.h
+++ b/arch/powerpc/include/asm/interrupt.h
@@ -52,6 +52,13 @@ static inline void interrupt_enter_prepare(struct pt_regs
*regs, struct interrup
if (user_mode(regs))
account_cpu_user_entry();
#endif
+
+#ifdef CONFIG_PPC_ADV_DEBUG_REGS
+ if (IS_ENABLED(CONFIG_PPC32) && unlikely(ts->debug.dbcr0 & DBCR0_IDM)) {
+ mtspr(SPRN_DBSR, -1);
+ mtspr(SPRN_DBCR0, global_dbcr0[smp_processor_id()]);
+ }
+#endif
}
/*
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 87c06e241fc2..971980d71b04 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -146,32 +146,9 @@ transfer_to_handler:
addir2, r12, -THREAD
addir11,r1,STACK_FRAME_OVERHEAD
stw r11,PT_REGS(r12)
-#if defined(CONFIG_40x) || defined(CONFIG_BOOKE)
- /* Check to see if the dbcr0 register is set up to debug. Use the
- internal debug mode bit to do this. */
- lwz r12,THREAD_DBCR0(r12)
- andis. r12,r12,DBCR0_IDM@h
-#endif
#ifdef CONFIG_PPC_BOOK3S_32
kuep_lock r11, r12
#endif
-#if defined(CONFIG_40x) || defined(CONFIG_BOOKE)
- beq+3f
- /* From user and task is ptraced - load up global dbcr0 */
- li r12,-1 /* clear all pending debug events */
- mtspr SPRN_DBSR,r12
- lis r11,global_dbcr0@ha
- tophys_novmstack r11,r11
- addir11,r11,global_dbcr0@l
-#ifdef CONFIG_SMP
- lwz r9,TASK_CPU(r2)
- slwir9,r9,2
- add r11,r11,r9
-#endif
- lwz r12,0(r11)
- mtspr SPRN_DBCR0,r12
-#endif
-
b 3f
2: /* if from kernel, check interrupted DOZE/NAP mode and
--
2.25.0
[RFC PATCH v1 09/41] powerpc/32: Entry cpu time accounting in C
There is no need for this to be in asm, use the new interrupt entry wrapper. Signed-off-by: Christophe Leroy --- arch/powerpc/include/asm/interrupt.h | 3 +++ arch/powerpc/include/asm/ppc_asm.h | 10 -- arch/powerpc/kernel/entry_32.S | 1 - 3 files changed, 3 insertions(+), 11 deletions(-) diff --git a/arch/powerpc/include/asm/interrupt.h b/arch/powerpc/include/asm/interrupt.h index 24671c43f930..e794111d7c03 100644 --- a/arch/powerpc/include/asm/interrupt.h +++ b/arch/powerpc/include/asm/interrupt.h @@ -19,6 +19,9 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs, struct interrup #ifdef CONFIG_PPC32 if (regs->msr & MSR_EE) trace_hardirqs_off(); + + if (user_mode(regs)) + account_cpu_user_entry(); #endif /* * Book3E reconciles irq soft mask in asm diff --git a/arch/powerpc/include/asm/ppc_asm.h b/arch/powerpc/include/asm/ppc_asm.h index 3dceb64fc9af..8998122fc7e2 100644 --- a/arch/powerpc/include/asm/ppc_asm.h +++ b/arch/powerpc/include/asm/ppc_asm.h @@ -23,18 +23,8 @@ */ #ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE -#define ACCOUNT_CPU_USER_ENTRY(ptr, ra, rb) #define ACCOUNT_CPU_USER_EXIT(ptr, ra, rb) #else -#define ACCOUNT_CPU_USER_ENTRY(ptr, ra, rb)\ - MFTB(ra); /* get timebase */ \ - PPC_LL rb, ACCOUNT_STARTTIME_USER(ptr);\ - PPC_STL ra, ACCOUNT_STARTTIME(ptr); \ - subfrb,rb,ra; /* subtract start value */ \ - PPC_LL ra, ACCOUNT_USER_TIME(ptr); \ - add ra,ra,rb; /* add on to user time */ \ - PPC_STL ra, ACCOUNT_USER_TIME(ptr); \ - #define ACCOUNT_CPU_USER_EXIT(ptr, ra, rb) \ MFTB(ra); /* get timebase */ \ PPC_LL rb, ACCOUNT_STARTTIME(ptr); \ diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 53fb16f21b07..87c06e241fc2 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -152,7 +152,6 @@ transfer_to_handler: lwz r12,THREAD_DBCR0(r12) andis. r12,r12,DBCR0_IDM@h #endif - ACCOUNT_CPU_USER_ENTRY(r2, r11, r12) #ifdef CONFIG_PPC_BOOK3S_32 kuep_lock r11, r12 #endif -- 2.25.0
[RFC PATCH v1 06/41] powerpc/40x: Prepare for enabling MMU in critical exception prolog
In order the enable MMU early in exception prolog, implement CONFIG_VMAP_STACK principles in critical exception prolog. There is no intention to use CONFIG_VMAP_STACK on 40x, but related code will be used to enable MMU early in exception in a later patch. Also address (critirq_ctx-PAGE_OFFSET) directly instead of using tophys() in order to win one instruction. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_40x.S | 40 +++--- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 5b337bf49bcb..1468f38c3860 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -89,6 +89,12 @@ _ENTRY(crit_srr0) .space 4 _ENTRY(crit_srr1) .space 4 +_ENTRY(crit_r1) + .space 4 +_ENTRY(crit_dear) + .space 4 +_ENTRY(crit_esr) + .space 4 _ENTRY(saved_ksp_limit) .space 4 @@ -107,32 +113,60 @@ _ENTRY(saved_ksp_limit) mfspr r11,SPRN_SRR1 stw r10,crit_srr0@l(0) stw r11,crit_srr1@l(0) +#ifdef CONFIG_VMAP_STACK + mfspr r10,SPRN_DEAR + mfspr r11,SPRN_ESR + stw r10,crit_dear@l(0) + stw r11,crit_esr@l(0) +#endif mfcrr10 /* save CR in r10 for now */ mfspr r11,SPRN_SRR3 /* check whether user or kernel*/ andi. r11,r11,MSR_PR - lis r11,critirq_ctx@ha - tophys(r11,r11) - lwz r11,critirq_ctx@l(r11) + lis r11,(critirq_ctx-PAGE_OFFSET)@ha + lwz r11,(critirq_ctx-PAGE_OFFSET)@l(r11) beq 1f /* COMING FROM USER MODE */ mfspr r11,SPRN_SPRG_THREAD/* if from user, start at top of */ lwz r11,TASK_STACK-THREAD(r11) /* this thread's kernel stack */ +#ifdef CONFIG_VMAP_STACK +1: stw r1,crit_r1@l(0) + addir1,r11,THREAD_SIZE-INT_FRAME_SIZE /* Alloc an excpt frm */ + LOAD_REG_IMMEDIATE(r11,MSR_KERNEL & ~(MSR_IR | MSR_RI)) + mtmsr r11 + isync + lwz r11,crit_r1@l(0) + stw r11,GPR1(r1) + stw r11,0(r1) + mr r11,r1 +#else 1: addir11,r11,THREAD_SIZE-INT_FRAME_SIZE /* Alloc an excpt frm */ tophys(r11,r11) stw r1,GPR1(r11) stw r1,0(r11) tovirt(r1,r11) +#endif stw r10,_CCR(r11) /* save various registers */ stw r12,GPR12(r11) stw r9,GPR9(r11) mflrr10 stw r10,_LINK(r11) +#ifdef CONFIG_VMAP_STACK + lis r9,PAGE_OFFSET@ha + lwz r10,crit_r10@l(r9) + lwz r12,crit_r11@l(r9) +#else lwz r10,crit_r10@l(0) lwz r12,crit_r11@l(0) +#endif stw r10,GPR10(r11) stw r12,GPR11(r11) +#ifdef CONFIG_VMAP_STACK + lwz r12,crit_dear@l(r9) + lwz r9,crit_esr@l(r9) +#else mfspr r12,SPRN_DEAR /* save DEAR and ESR in the frame */ mfspr r9,SPRN_ESR /* in them at the point where the */ +#endif stw r12,_DEAR(r11) /* since they may have had stuff */ stw r9,_ESR(r11)/* exception was taken */ mfspr r12,SPRN_SRR2 -- 2.25.0
[RFC PATCH v1 05/41] powerpc/40x: Reorder a few instructions in critical exception prolog
In order to ease preparation for CONFIG_VMAP_STACK, reorder a few instruction, especially save r1 into stack frame earlier. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_40x.S | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 067ae1302c1c..5b337bf49bcb 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -119,6 +119,9 @@ _ENTRY(saved_ksp_limit) lwz r11,TASK_STACK-THREAD(r11) /* this thread's kernel stack */ 1: addir11,r11,THREAD_SIZE-INT_FRAME_SIZE /* Alloc an excpt frm */ tophys(r11,r11) + stw r1,GPR1(r11) + stw r1,0(r11) + tovirt(r1,r11) stw r10,_CCR(r11) /* save various registers */ stw r12,GPR12(r11) stw r9,GPR9(r11) @@ -129,14 +132,11 @@ _ENTRY(saved_ksp_limit) stw r10,GPR10(r11) stw r12,GPR11(r11) mfspr r12,SPRN_DEAR /* save DEAR and ESR in the frame */ - stw r12,_DEAR(r11) /* since they may have had stuff */ mfspr r9,SPRN_ESR /* in them at the point where the */ + stw r12,_DEAR(r11) /* since they may have had stuff */ stw r9,_ESR(r11)/* exception was taken */ mfspr r12,SPRN_SRR2 - stw r1,GPR1(r11) mfspr r9,SPRN_SRR3 - stw r1,0(r11) - tovirt(r1,r11) rlwinm r9,r9,0,14,12 /* clear MSR_WE (necessary?) */ stw r0,GPR0(r11) lis r10, STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ -- 2.25.0
[RFC PATCH v1 07/41] powerpc/40x: Prepare normal exception handler for enabling MMU early
Ensure normal exception handler are able to manage stuff with MMU enabled. For that we use CONFIG_VMAP_STACK related code allthough there is no intention to really activate CONFIG_VMAP_STACK on powerpc 40x for the moment. 40x uses SPRN_DEAR instead of SPRN_DAR and SPRN_ESR instead of SPRN_DSISR. Take it into account in common macros. 40x MSR value doesn't fit on 15 bits, use LOAD_REG_IMMEDIATE() in common macros that will be used also with 40x. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 2 +- arch/powerpc/kernel/head_32.h | 15 ++- arch/powerpc/kernel/head_40x.S | 17 ++--- 3 files changed, 21 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 1e59d0bb1a6f..9312634085ba 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -162,7 +162,7 @@ transfer_to_handler: li r12,-1 /* clear all pending debug events */ mtspr SPRN_DBSR,r12 lis r11,global_dbcr0@ha - tophys(r11,r11) + tophys_novmstack r11,r11 addir11,r11,global_dbcr0@l #ifdef CONFIG_SMP lwz r9,TASK_CPU(r2) diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 5d4706c14572..ac6b391f1493 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -22,9 +22,17 @@ #ifdef CONFIG_VMAP_STACK mfspr r10, SPRN_SPRG_THREAD .if \handle_dar_dsisr +#ifdef CONFIG_40x + mfspr r11, SPRN_DEAR +#else mfspr r11, SPRN_DAR +#endif stw r11, DAR(r10) +#ifdef CONFIG_40x + mfspr r11, SPRN_ESR +#else mfspr r11, SPRN_DSISR +#endif stw r11, DSISR(r10) .endif mfspr r11, SPRN_SRR0 @@ -61,7 +69,7 @@ .macro EXCEPTION_PROLOG_2 handle_dar_dsisr=0 #ifdef CONFIG_VMAP_STACK - li r11, MSR_KERNEL & ~(MSR_IR | MSR_RI) /* can take DTLB miss */ + LOAD_REG_IMMEDIATE(r11, MSR_KERNEL & ~(MSR_IR | MSR_RI)) /* can take DTLB miss */ mtmsr r11 isync mfspr r11, SPRN_SPRG_SCRATCH2 @@ -158,8 +166,13 @@ .macro save_dar_dsisr_on_stack reg1, reg2, sp #ifndef CONFIG_VMAP_STACK +#ifdef CONFIG_40x + mfspr \reg1, SPRN_DEAR + mfspr \reg2, SPRN_ESR +#else mfspr \reg1, SPRN_DAR mfspr \reg2, SPRN_DSISR +#endif stw \reg1, _DAR(\sp) stw \reg2, _DSISR(\sp) #endif diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 1468f38c3860..4bf0aee858eb 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -221,11 +221,8 @@ _ENTRY(saved_ksp_limit) * if they can't resolve the lightweight TLB fault. */ START_EXCEPTION(0x0300, DataStorage) - EXCEPTION_PROLOG - mfspr r5, SPRN_ESR/* Grab the ESR, save it */ - stw r5, _ESR(r11) - mfspr r4, SPRN_DEAR /* Grab the DEAR, save it */ - stw r4, _DEAR(r11) + EXCEPTION_PROLOG handle_dar_dsisr=1 + save_dar_dsisr_on_stack r4, r5, r11 EXC_XFER_LITE(0x300, handle_page_fault) /* @@ -244,17 +241,15 @@ _ENTRY(saved_ksp_limit) /* 0x0600 - Alignment Exception */ START_EXCEPTION(0x0600, Alignment) - EXCEPTION_PROLOG - mfspr r4,SPRN_DEAR/* Grab the DEAR and save it */ - stw r4,_DEAR(r11) + EXCEPTION_PROLOG handle_dar_dsisr=1 + save_dar_dsisr_on_stack r4, r5, r11 addir3,r1,STACK_FRAME_OVERHEAD EXC_XFER_STD(0x600, alignment_exception) /* 0x0700 - Program Exception */ START_EXCEPTION(0x0700, ProgramCheck) - EXCEPTION_PROLOG - mfspr r4,SPRN_ESR /* Grab the ESR and save it */ - stw r4,_ESR(r11) + EXCEPTION_PROLOG handle_dar_dsisr=1 + save_dar_dsisr_on_stack r4, r5, r11 addir3,r1,STACK_FRAME_OVERHEAD EXC_XFER_STD(0x700, program_check_exception) -- 2.25.0
[RFC PATCH v1 08/41] powerpc/32: Reconcile interrupts in C
There is no need for this to be in asm anymore,
use the new interrupt entry wrapper.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/interrupt.h | 4 ++
arch/powerpc/kernel/entry_32.S | 58
2 files changed, 4 insertions(+), 58 deletions(-)
diff --git a/arch/powerpc/include/asm/interrupt.h
b/arch/powerpc/include/asm/interrupt.h
index 4badb3e51c19..24671c43f930 100644
--- a/arch/powerpc/include/asm/interrupt.h
+++ b/arch/powerpc/include/asm/interrupt.h
@@ -16,6 +16,10 @@ struct interrupt_state {
static inline void interrupt_enter_prepare(struct pt_regs *regs, struct
interrupt_state *state)
{
+#ifdef CONFIG_PPC32
+ if (regs->msr & MSR_EE)
+ trace_hardirqs_off();
+#endif
/*
* Book3E reconciles irq soft mask in asm
*/
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 9312634085ba..53fb16f21b07 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -202,22 +202,6 @@ transfer_to_handler_cont:
lwz r9,4(r9)/* where to go when done */
#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PERF_EVENTS)
mtspr SPRN_NRI, r0
-#endif
-#ifdef CONFIG_TRACE_IRQFLAGS
- /*
-* When tracing IRQ state (lockdep) we enable the MMU before we call
-* the IRQ tracing functions as they might access vmalloc space or
-* perform IOs for console output.
-*
-* To speed up the syscall path where interrupts stay on, let's check
-* first if we are changing the MSR value at all.
-*/
- tophys_novmstack r12, r1
- lwz r12,_MSR(r12)
- andi. r12,r12,MSR_EE
- bne 1f
-
- /* MSR isn't changing, just transition directly */
#endif
mtspr SPRN_SRR0,r11
mtspr SPRN_SRR1,r10
@@ -244,48 +228,6 @@ transfer_to_handler_cont:
_ASM_NOKPROBE_SYMBOL(transfer_to_handler)
_ASM_NOKPROBE_SYMBOL(transfer_to_handler_cont)
-#ifdef CONFIG_TRACE_IRQFLAGS
-1: /* MSR is changing, re-enable MMU so we can notify lockdep. We need to
-* keep interrupts disabled at this point otherwise we might risk
-* taking an interrupt before we tell lockdep they are enabled.
-*/
- lis r12,reenable_mmu@h
- ori r12,r12,reenable_mmu@l
- LOAD_REG_IMMEDIATE(r0, MSR_KERNEL)
- mtspr SPRN_SRR0,r12
- mtspr SPRN_SRR1,r0
- rfi
-#ifdef CONFIG_40x
- b . /* Prevent prefetch past rfi */
-#endif
-
-reenable_mmu:
- /*
-* We save a bunch of GPRs,
-* r3 can be different from GPR3(r1) at this point, r9 and r11
-* contains the old MSR and handler address respectively,
-* r0, r4-r8, r12, CCR, CTR, XER etc... are left
-* clobbered as they aren't useful past this point.
-*/
-
- stwur1,-32(r1)
- stw r9,8(r1)
- stw r11,12(r1)
- stw r3,16(r1)
-
- /* If we are disabling interrupts (normal case), simply log it with
-* lockdep
-*/
-1: bl trace_hardirqs_off
- lwz r3,16(r1)
- lwz r11,12(r1)
- lwz r9,8(r1)
- addir1,r1,32
- mtctr r11
- mtlrr9
- bctr/* jump to handler */
-#endif /* CONFIG_TRACE_IRQFLAGS */
-
#ifndef CONFIG_VMAP_STACK
/*
* On kernel stack overflow, load up an initial stack pointer
--
2.25.0
[RFC PATCH v1 02/41] powerpc/40x: Don't use SPRN_SPRG_SCRATCH0/1 in TLB miss handlers
SPRN_SPRG_SCRATCH5 is used to save SPRN_PID. SPRN_SPRG_SCRATCH6 is already available. SPRN_PID is only 8 bits. We have r12 that contains CR. We only need to preserve CR0, so we have space available in r12 to save PID. Keep PID in r12 and free up SPRN_SPRG_SCRATCH5. Then In TLB miss handlers, instead of using SPRN_SPRG_SCRATCH0 and SPRN_SPRG_SCRATCH1, use SPRN_SPRG_SCRATCH5 and SPRN_SPRG_SCRATCH6 to avoid future conflicts with normal exception prologs. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_40x.S | 39 -- 1 file changed, 18 insertions(+), 21 deletions(-) diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 24724a7dad49..383238a98f77 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -249,13 +249,13 @@ _ENTRY(saved_ksp_limit) * load TLB entries from the page table if they exist. */ START_EXCEPTION(0x1100, DTLBMiss) - mtspr SPRN_SPRG_SCRATCH0, r10 /* Save some working registers */ - mtspr SPRN_SPRG_SCRATCH1, r11 + mtspr SPRN_SPRG_SCRATCH5, r10 /* Save some working registers */ + mtspr SPRN_SPRG_SCRATCH6, r11 mtspr SPRN_SPRG_SCRATCH3, r12 mtspr SPRN_SPRG_SCRATCH4, r9 mfcrr12 mfspr r9, SPRN_PID - mtspr SPRN_SPRG_SCRATCH5, r9 + rlwimi r12, r9, 0, 0xff mfspr r10, SPRN_DEAR /* Get faulting address */ /* If we are faulting a kernel address, we have to use the @@ -316,13 +316,12 @@ _ENTRY(saved_ksp_limit) /* The bailout. Restore registers to pre-exception conditions * and call the heavyweights to help us out. */ - mfspr r9, SPRN_SPRG_SCRATCH5 - mtspr SPRN_PID, r9 - mtcrr12 + mtspr SPRN_PID, r12 + mtcrf 0x80, r12 mfspr r9, SPRN_SPRG_SCRATCH4 mfspr r12, SPRN_SPRG_SCRATCH3 - mfspr r11, SPRN_SPRG_SCRATCH1 - mfspr r10, SPRN_SPRG_SCRATCH0 + mfspr r11, SPRN_SPRG_SCRATCH6 + mfspr r10, SPRN_SPRG_SCRATCH5 b DataStorage /* 0x1200 - Instruction TLB Miss Exception @@ -330,13 +329,13 @@ _ENTRY(saved_ksp_limit) * registers and bailout to a different point. */ START_EXCEPTION(0x1200, ITLBMiss) - mtspr SPRN_SPRG_SCRATCH0, r10 /* Save some working registers */ - mtspr SPRN_SPRG_SCRATCH1, r11 + mtspr SPRN_SPRG_SCRATCH5, r10 /* Save some working registers */ + mtspr SPRN_SPRG_SCRATCH6, r11 mtspr SPRN_SPRG_SCRATCH3, r12 mtspr SPRN_SPRG_SCRATCH4, r9 mfcrr12 mfspr r9, SPRN_PID - mtspr SPRN_SPRG_SCRATCH5, r9 + rlwimi r12, r9, 0, 0xff mfspr r10, SPRN_SRR0 /* Get faulting address */ /* If we are faulting a kernel address, we have to use the @@ -397,13 +396,12 @@ _ENTRY(saved_ksp_limit) /* The bailout. Restore registers to pre-exception conditions * and call the heavyweights to help us out. */ - mfspr r9, SPRN_SPRG_SCRATCH5 - mtspr SPRN_PID, r9 - mtcrr12 + mtspr SPRN_PID, r12 + mtcrf 0x80, r12 mfspr r9, SPRN_SPRG_SCRATCH4 mfspr r12, SPRN_SPRG_SCRATCH3 - mfspr r11, SPRN_SPRG_SCRATCH1 - mfspr r10, SPRN_SPRG_SCRATCH0 + mfspr r11, SPRN_SPRG_SCRATCH6 + mfspr r10, SPRN_SPRG_SCRATCH5 b InstructionAccess EXCEPTION(0x1300, Trap_13, unknown_exception, EXC_XFER_STD) @@ -543,13 +541,12 @@ finish_tlb_load: /* Done...restore registers and get out of here. */ - mfspr r9, SPRN_SPRG_SCRATCH5 - mtspr SPRN_PID, r9 - mtcrr12 + mtspr SPRN_PID, r12 + mtcrf 0x80, r12 mfspr r9, SPRN_SPRG_SCRATCH4 mfspr r12, SPRN_SPRG_SCRATCH3 - mfspr r11, SPRN_SPRG_SCRATCH1 - mfspr r10, SPRN_SPRG_SCRATCH0 + mfspr r11, SPRN_SPRG_SCRATCH6 + mfspr r10, SPRN_SPRG_SCRATCH5 rfi /* Should sync shadow TLBs */ b . /* prevent prefetch past rfi */ -- 2.25.0
[RFC PATCH v1 03/41] powerpc/40x: Change CRITICAL_EXCEPTION_PROLOG macro to a gas macro
Change CRITICAL_EXCEPTION_PROLOG macro to a gas macro to remove the ugly ; and \ on each line. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_40x.S | 71 +- 1 file changed, 36 insertions(+), 35 deletions(-) diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 383238a98f77..9cef423d574b 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -100,42 +100,43 @@ _ENTRY(saved_ksp_limit) * Instead we use a couple of words of memory at low physical addresses. * This is OK since we don't support SMP on these processors. */ -#define CRITICAL_EXCEPTION_PROLOG \ - stw r10,crit_r10@l(0); /* save two registers to work with */\ - stw r11,crit_r11@l(0); \ - mfcrr10;/* save CR in r10 for now */\ - mfspr r11,SPRN_SRR3; /* check whether user or kernel*/\ - andi. r11,r11,MSR_PR; \ - lis r11,critirq_ctx@ha; \ - tophys(r11,r11); \ - lwz r11,critirq_ctx@l(r11); \ - beq 1f; \ - /* COMING FROM USER MODE */ \ - mfspr r11,SPRN_SPRG_THREAD; /* if from user, start at top of */\ - lwz r11,TASK_STACK-THREAD(r11); /* this thread's kernel stack */\ -1: addir11,r11,THREAD_SIZE-INT_FRAME_SIZE; /* Alloc an excpt frm */\ - tophys(r11,r11); \ - stw r10,_CCR(r11); /* save various registers */\ - stw r12,GPR12(r11); \ - stw r9,GPR9(r11);\ - mflrr10; \ - stw r10,_LINK(r11); \ - mfspr r12,SPRN_DEAR; /* save DEAR and ESR in the frame */\ - stw r12,_DEAR(r11); /* since they may have had stuff */\ - mfspr r9,SPRN_ESR;/* in them at the point where the */\ - stw r9,_ESR(r11); /* exception was taken */\ - mfspr r12,SPRN_SRR2; \ - stw r1,GPR1(r11);\ - mfspr r9,SPRN_SRR3;\ - stw r1,0(r11); \ - tovirt(r1,r11); \ - rlwinm r9,r9,0,14,12; /* clear MSR_WE (necessary?) */\ - stw r0,GPR0(r11);\ - lis r10, STACK_FRAME_REGS_MARKER@ha; /* exception frame marker */\ - addir10, r10, STACK_FRAME_REGS_MARKER@l; \ - stw r10, 8(r11); \ - SAVE_4GPRS(3, r11); \ +.macro CRITICAL_EXCEPTION_PROLOG + stw r10,crit_r10@l(0) /* save two registers to work with */ + stw r11,crit_r11@l(0) + mfcrr10 /* save CR in r10 for now */ + mfspr r11,SPRN_SRR3 /* check whether user or kernel*/ + andi. r11,r11,MSR_PR + lis r11,critirq_ctx@ha + tophys(r11,r11) + lwz r11,critirq_ctx@l(r11) + beq 1f + /* COMING FROM USER MODE */ + mfspr r11,SPRN_SPRG_THREAD/* if from user, start at top of */ + lwz r11,TASK_STACK-THREAD(r11) /* this thread's kernel stack */ +1: addir11,r11,THREAD_SIZE-INT_FRAME_SIZE /* Alloc an excpt frm */ + tophys(r11,r11) + stw r10,_CCR(r11) /* save various registers */ + stw r12,GPR12(r11) + stw r9,GPR9(r11) + mflrr10 + stw r10,_LINK(r11) + mfspr r12,SPRN_DEAR /* save DEAR and ESR in the frame */ + stw r12,_DEAR(r11) /* since they may have had stuff */ + mfspr r9,SPRN_ESR /* in them at the point where the */ + stw r9,_ESR(r11)/* exception was taken */ + mfspr r12,SPRN_SRR2 + stw r1,GPR1(r11) + mfspr r9,SPRN_SRR3 + stw r1,0(r11) + tovirt(r1,r11) + rlwinm r9,r9,0,14,12 /* clear MSR_WE (necessary?) */ + stw r0,GPR0(r11) + lis r10, STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ + addir10, r10,
[RFC PATCH v1 00/41] powerpc/32: Switch to interrupt entry/exit in C
This series aims at porting interrupt entry/exit in C on PPC32, using the work already merged for PPC64. First part do minimal changes in 40x in order to be able to enable MMU earlier in exception entry. Second part prepares and switches interrupt exit in C. Third part moves more and more things in C, ending with KUAP management. v1 is boot tested on 8xx and 83xx, releasing it as an RFC to get early feedback. This series applies on top of the one switching ppc32 syscall entry/exit in C. First patch is a bug fix already submitted but not yet merged that interracts with the series. Christophe Leroy (41): powerpc/32: Preserve cr1 in exception prolog stack check to fix build error powerpc/40x: Don't use SPRN_SPRG_SCRATCH0/1 in TLB miss handlers powerpc/40x: Change CRITICAL_EXCEPTION_PROLOG macro to a gas macro powerpc/40x: Save SRR0/SRR1 and r10/r11 earlier in critical exception powerpc/40x: Reorder a few instructions in critical exception prolog powerpc/40x: Prepare for enabling MMU in critical exception prolog powerpc/40x: Prepare normal exception handler for enabling MMU early powerpc/32: Reconcile interrupts in C powerpc/32: Entry cpu time accounting in C powerpc/32: Handle bookE debugging in C in exception entry powerpc/32: Use fast instruction to set MSR RI in exception prolog on 8xx powerpc/32: Remove ksp_limit powerpc/32: Always enable data translation in exception prolog powerpc/32: Tag DAR in EXCEPTION_PROLOG_2 for the 8xx powerpc/32: Enable instruction translation at the same time as data translation powerpc/32: Statically initialise first emergency context powerpc/32: Add vmap_stack_overflow label inside the macro powerpc/32: Use START_EXCEPTION() as much as possible powerpc/32: Move exception prolog code into .text once MMU is back on powerpc/32: Provide a name to exception prolog continuation in virtual mode powerpc/32: Refactor booke critical registers saving powerpc/32: Perform normal function call in exception entry powerpc/32: Always save non volatile registers on exception entry powerpc/32: Replace ASM exception exit by C exception exit from ppc64 powerpc/32: Set regs parameter in r3 in transfer_to_handler powerpc/32: Remove handle_page_fault() powerpc/32: Save trap number on stack in exception prolog powerpc/32: Add a prepare_transfer_to_handler macro for exception prologs powerpc/32: Only restore non volatile registers when required powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE powerpc/32: Remove the xfer parameter in EXCEPTION() macro powerpc/32: Refactor saving of volatile registers in exception prologs powerpc/32: Save remaining registers in exception prolog powerpc/32: Set current->thread.regs in C interrupt entry powerpc/32: Return directly from power_save_ppc32_restore() powerpc/32: Only use prepare_transfer_to_handler function on book3s/32 and e500 powerpc/32s: Move KUEP locking/unlocking in C powerpc/64s: Make kuap_check_amr() and kuap_get_and_check_amr() generic powerpc/32s: Create C version of kuap save/restore/check helpers powerpc/8xx: Create C version of kuap save/restore/check helpers powerpc/32: Manage KUAP in C arch/powerpc/include/asm/book3s/32/kup.h | 126 ++- arch/powerpc/include/asm/book3s/64/kup.h | 24 +- arch/powerpc/include/asm/interrupt.h | 21 + arch/powerpc/include/asm/kup.h | 37 +- arch/powerpc/include/asm/nohash/32/kup-8xx.h | 58 +- arch/powerpc/include/asm/ppc_asm.h | 10 - arch/powerpc/include/asm/processor.h | 6 +- arch/powerpc/include/asm/ptrace.h| 13 +- arch/powerpc/kernel/asm-offsets.c| 4 - arch/powerpc/kernel/entry_32.S | 810 --- arch/powerpc/kernel/fpu.S| 2 - arch/powerpc/kernel/head_32.h| 197 ++--- arch/powerpc/kernel/head_40x.S | 271 --- arch/powerpc/kernel/head_44x.S | 10 +- arch/powerpc/kernel/head_8xx.S | 151 ++-- arch/powerpc/kernel/head_book3s_32.S | 239 +++--- arch/powerpc/kernel/head_booke.h | 188 +++-- arch/powerpc/kernel/head_fsl_booke.S | 64 +- arch/powerpc/kernel/idle_6xx.S | 14 +- arch/powerpc/kernel/idle_e500.S | 14 +- arch/powerpc/kernel/interrupt.c | 35 +- arch/powerpc/kernel/misc_32.S| 14 - arch/powerpc/kernel/process.c| 6 +- arch/powerpc/kernel/setup_32.c | 2 +- arch/powerpc/kernel/traps.c | 9 - arch/powerpc/kernel/vector.S | 2 - arch/powerpc/lib/sstep.c | 9 - arch/powerpc/mm/book3s32/Makefile| 1 + arch/powerpc/mm/book3s32/hash_low.S | 14 - arch/powerpc/mm/book3s32/kuep.c | 38 + arch/powerpc/mm/fault.c | 4 +- 31 files changed, 875 insertions(+), 1518
[RFC PATCH v1 04/41] powerpc/40x: Save SRR0/SRR1 and r10/r11 earlier in critical exception
In order to be able to switch MMU on in exception prolog, save SRR0 and SRR1 earlier. Also save r10 and r11 into stack earlier to better match with the normal exception prolog. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/entry_32.S | 9 - arch/powerpc/kernel/head_40x.S | 8 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 8dea4d3b1d06..1e59d0bb1a6f 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -107,15 +107,6 @@ _ASM_NOKPROBE_SYMBOL(crit_transfer_to_handler) #ifdef CONFIG_40x .globl crit_transfer_to_handler crit_transfer_to_handler: - lwz r0,crit_r10@l(0) - stw r0,GPR10(r11) - lwz r0,crit_r11@l(0) - stw r0,GPR11(r11) - mfspr r0,SPRN_SRR0 - stw r0,crit_srr0@l(0) - mfspr r0,SPRN_SRR1 - stw r0,crit_srr1@l(0) - /* set the stack limit to the current stack */ mfspr r8,SPRN_SPRG_THREAD lwz r0,KSP_LIMIT(r8) diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 9cef423d574b..067ae1302c1c 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -103,6 +103,10 @@ _ENTRY(saved_ksp_limit) .macro CRITICAL_EXCEPTION_PROLOG stw r10,crit_r10@l(0) /* save two registers to work with */ stw r11,crit_r11@l(0) + mfspr r10,SPRN_SRR0 + mfspr r11,SPRN_SRR1 + stw r10,crit_srr0@l(0) + stw r11,crit_srr1@l(0) mfcrr10 /* save CR in r10 for now */ mfspr r11,SPRN_SRR3 /* check whether user or kernel*/ andi. r11,r11,MSR_PR @@ -120,6 +124,10 @@ _ENTRY(saved_ksp_limit) stw r9,GPR9(r11) mflrr10 stw r10,_LINK(r11) + lwz r10,crit_r10@l(0) + lwz r12,crit_r11@l(0) + stw r10,GPR10(r11) + stw r12,GPR11(r11) mfspr r12,SPRN_DEAR /* save DEAR and ESR in the frame */ stw r12,_DEAR(r11) /* since they may have had stuff */ mfspr r9,SPRN_ESR /* in them at the point where the */ -- 2.25.0
[RFC PATCH v1 01/41] powerpc/32: Preserve cr1 in exception prolog stack check to fix build error
THREAD_ALIGN_SHIFT = THREAD_SHIFT + 1 = PAGE_SHIFT + 1
Maximum PAGE_SHIFT is 18 for 256k pages so
THREAD_ALIGN_SHIFT is 19 at the maximum.
No need to clobber cr1, it can be preserved when moving r1
into CR when we check stack overflow.
This reduces the number of instructions in Machine Check Exception
prolog and fixes a build failure reported by the kernel test robot
on v5.10 stable when building with RTAS + VMAP_STACK + KVM. That
build failure is due to too many instructions in the prolog hence
not fitting between 0x200 and 0x300. Allthough the problem doesn't
show up in mainline, it is still worth the change.
Reported-by: kernel test robot
Fixes: 98bf2d3f4970 ("powerpc/32s: Fix RTAS machine check with VMAP stack")
Cc: sta...@vger.kernel.org
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/head_32.h| 2 +-
arch/powerpc/kernel/head_book3s_32.S | 6 --
2 files changed, 1 insertion(+), 7 deletions(-)
diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h
index 961b1ce3b6bf..5d4706c14572 100644
--- a/arch/powerpc/kernel/head_32.h
+++ b/arch/powerpc/kernel/head_32.h
@@ -47,7 +47,7 @@
lwz r1,TASK_STACK-THREAD(r1)
addir1, r1, THREAD_SIZE - INT_FRAME_SIZE
1:
- mtcrf 0x7f, r1
+ mtcrf 0x3f, r1
bt 32 - THREAD_ALIGN_SHIFT, stack_overflow
#else
subir11, r1, INT_FRAME_SIZE /* use r1 if kernel */
diff --git a/arch/powerpc/kernel/head_book3s_32.S
b/arch/powerpc/kernel/head_book3s_32.S
index 086970bec32c..727fdab557c9 100644
--- a/arch/powerpc/kernel/head_book3s_32.S
+++ b/arch/powerpc/kernel/head_book3s_32.S
@@ -278,12 +278,6 @@ MachineCheck:
7: EXCEPTION_PROLOG_2
addir3,r1,STACK_FRAME_OVERHEAD
#ifdef CONFIG_PPC_CHRP
-#ifdef CONFIG_VMAP_STACK
- mfspr r4, SPRN_SPRG_THREAD
- tovirt(r4, r4)
- lwz r4, RTAS_SP(r4)
- cmpwi cr1, r4, 0
-#endif
beq cr1, machine_check_tramp
twi 31, 0, 0
#else
--
2.25.0
[PATCH V3] powerpc/perf: Adds support for programming of Thresholding in P10
Thresholding, a performance monitoring unit feature, can be
used to identify marked instructions which take more than
expected cycles between start event and end event.
Threshold compare (thresh_cmp) bits are programmed in MMCRA
register. In Power9, thresh_cmp bits were part of the
event code. But in case of P10, thresh_cmp are not part of
event code due to inclusion of MMCR3 bits.
Patch here adds an option to use attr.config1 variable
to be used to pass thresh_cmp value to be programmed in
MMCRA register. A new ppmu flag called PPMU_HAS_ATTR_CONFIG1
has been added and this flag is used to notify the use of
attr.config1 variable.
Patch has extended the parameter list of 'compute_mmcr',
to include power_pmu's 'flags' element and parameter list of
get_constraint to include attr.config1 value. It also extend
parameter list of power_check_constraints inorder to pass
perf_event list.
As stated by commit ef0e3b650f8d ("powerpc/perf: Fix Threshold
Event Counter Multiplier width for P10"), constraint bits for
thresh_cmp is also needed to be increased to 11 bits, which is
handled as part of this patch. We added bit number 53 as part
of constraint bits of thresh_cmp for power10 to make it an
11 bit field.
Updated layout for p10:
/*
* Layout of constraint bits:
*
*60565248444036
32
* | - - - - | - - - - | - - - - | - - - - | - - - - | - - - - | - - - - | - -
- - |
* [ fab_match ] [ thresh_cmp ] [ thresh_ctl] [
]
* | |
* [ thresh_cmp bits for p10] thresh_sel -*
*
*2824201612 8 4
0
* | - - - - | - - - - | - - - - | - - - - | - - - - | - - - - | - - - - | - -
- - |
* [ ] | [ ] | [ sample ] [ ] [6] [5] [4] [3] [2]
[1]
*| || | |
* BHRB IFM -* || |*radix_scope | Count of events for each
PMC.
* EBB -*| |p1, p2, p3, p4, p5, p6.
* L1 I/D qualifier -* |
* nc - number of counters -*
*
* The PMC fields P1..P6, and NC, are adder fields. As we accumulate constraints
* we want the low bit of each field to be added to any existing value.
*
* Everything else is a value field.
*/
Result:
command#: cat /sys/devices/cpu/format/thresh_cmp
config1:0-17
ex. usage:
command#: perf record -I --weight -d -e
cpu/event=0x67340101EC,thresh_cmp=500/ ./ebizzy -S 2 -t 1 -s 4096
1826636 records/s
real 2.00 s
user 2.00 s
sys 0.00 s
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.038 MB perf.data (61 samples) ]
Signed-off-by: Kajol Jain
---
arch/powerpc/include/asm/perf_event_server.h | 5 +-
arch/powerpc/perf/core-book3s.c | 15 +++--
arch/powerpc/perf/isa207-common.c| 67 +---
arch/powerpc/perf/isa207-common.h| 15 +++--
arch/powerpc/perf/mpc7450-pmu.c | 5 +-
arch/powerpc/perf/power10-pmu.c | 4 +-
arch/powerpc/perf/power5+-pmu.c | 5 +-
arch/powerpc/perf/power5-pmu.c | 5 +-
arch/powerpc/perf/power6-pmu.c | 5 +-
arch/powerpc/perf/power7-pmu.c | 5 +-
arch/powerpc/perf/ppc970-pmu.c | 5 +-
11 files changed, 102 insertions(+), 34 deletions(-)
---
Changelog
v2 -> v3
- Removed field 'events_config1' from struct cpu_hw_events
and directly using attr.config1 field instead as suggested
by Michael Ellerman.
- Extended the parameter list of 'power_check_constraints' function
to also pass perf_event struct array as we need to access
event's attr.config1 value. And since when we call
'power_check_constraints' from 'power_pmu_event_init' the
cpu_hw_events structure not get updated.
v1 -> v2
- Add new function 'p10_thresh_cmp_val' to evaluate thresh_cmp
value.
- Extended the parameter list of get_constraint function
to include attr.config1 value.
- Added bit number 53 as part of constraint bits of thresh_cmp
to make it a 11 bit field for power10.
- Updated PPMU_HAS_ATTR_CONFIG1 value to 0x0800
- Add new field 'events_config1' in struct cpu_hw_events and also
update this field in all required functions accordingly.
---
diff --git a/arch/powerpc/include/asm/perf_event_server.h
b/arch/powerpc/include/asm/perf_event_server.h
index 3b7baba01c92..00e7e671bb4b 100644
--- a/arch/powerpc/include/asm/perf_event_server.h
+++ b/arch/powerpc/include/asm/perf_event_server.h
@@ -36,9 +36,9 @@ struct power_pmu {
unsigned long test_adder;
int (*compute_mmcr)(u64 events[], int n_ev,
unsigned int hwc[], struct mmcr_regs *mmcr,
- struct perf_event *pevents[]);
+
