Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
Benjamin Herrenschmidt b...@kernel.crashing.org wrote on 09/12/2009 09:56:35: From: Benjamin Herrenschmidt b...@kernel.crashing.org To: Joakim Tjernlund joakim.tjernl...@transmode.se Cc: linuxppc-...@ozlabs.org linuxppc-...@ozlabs.org, Rex Feany rfe...@mrv.com, Scott Wood scottw...@freescale.com Date: 09/12/2009 09:56 Subject: Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB On Wed, 2009-12-09 at 08:39 +0100, Joakim Tjernlund wrote: The later isn't as simple :) I believe the bulk of such code in entry_32.S. Yeah but it would be useful for hash I suppose if one really wants to boot with nobats. Though at least on hash most of the time we have ways to recover by mean of MSR:RI being cleared, which your TLB miss code doesn't check... Anyhow, the config option is still valid as if enabled it will pin several DTLB's too. Scott had some concerns about removing the config option completely so this was the next best thing. Well, if you want to pin at least one entry, then just remove the #if completely but don't leave a #if 1 :-) As you wish. Here is a new version without the #if 1 This is required for 8xx to work in Linus current tree. From 5a4fa5078ba1774b037dbfd88b9c87b74cf76db7 Mon Sep 17 00:00:00 2001 From: Joakim Tjernlund joakim.tjernl...@transmode.se Date: Fri, 13 Nov 2009 00:26:59 +0100 Subject: [PATCH] 8xx: Always pin kernel instruction TLB Various kernel asm modifies SRR0/SRR1 just before executing a rfi. If such code crosses a page boundary you risk a TLB miss which will clobber SRR0/SRR1. Avoid this by always pinning kernel instruction TLB space. Signed-off-by: Joakim Tjernlund joakim.tjernl...@transmode.se --- arch/powerpc/kernel/head_8xx.S |8 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index adc5a32..c80e7c5 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -767,12 +767,12 @@ start_here: */ initial_mmu: tlbia /* Invalidate all TLB entries */ -#ifdef CONFIG_PIN_TLB +/* Always pin the first 8 MB ITLB to prevent ITLB + misses while mucking around with SRR0/SRR1 in asm +*/ lis r8, mi_rs...@h ori r8, r8, 0x1c00 -#else - li r8, 0 -#endif + mtspr SPRN_MI_CTR, r8 /* Set instruction MMU control */ #ifdef CONFIG_PIN_TLB -- 1.6.4.4 ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
On Wed, 2009-12-09 at 08:39 +0100, Joakim Tjernlund wrote: The later isn't as simple :) I believe the bulk of such code in entry_32.S. Yeah but it would be useful for hash I suppose if one really wants to boot with nobats. Though at least on hash most of the time we have ways to recover by mean of MSR:RI being cleared, which your TLB miss code doesn't check... Anyhow, the config option is still valid as if enabled it will pin several DTLB's too. Scott had some concerns about removing the config option completely so this was the next best thing. Well, if you want to pin at least one entry, then just remove the #if completely but don't leave a #if 1 :-) Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
Benjamin Herrenschmidt b...@kernel.crashing.org wrote on 09/12/2009 09:56:35: On Wed, 2009-12-09 at 08:39 +0100, Joakim Tjernlund wrote: The later isn't as simple :) I believe the bulk of such code in entry_32.S. Yeah but it would be useful for hash I suppose if one really wants to boot with nobats. Though at least on hash most of the time we have ways to recover by mean of MSR:RI being cleared, which your TLB miss code doesn't check... Never looked at this so I cannot comment. Anyhow, the config option is still valid as if enabled it will pin several DTLB's too. Scott had some concerns about removing the config option completely so this was the next best thing. Well, if you want to pin at least one entry, then just remove the #if completely but don't leave a #if 1 :-) I could, but I wanted to have an easy way back to the old way if the need arises. Anyway, I am rather busy ATM with more pressing matters so I would appreciate if you could take the patch as is. Jocke ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
On Fri, 2009-11-20 at 11:21 +0100, Joakim Tjernlund wrote: Various kernel asm modifies SRR0/SRR1 just before executing a rfi. If such code crosses a page boundary you risk a TLB miss which will clobber SRR0/SRR1. Avoid this by always pinning kernel instruction TLB space. Signed-off-by: Joakim Tjernlund joakim.tjernl...@transmode.se --- arch/powerpc/kernel/head_8xx.S |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index a9f1ace..e70503d 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -705,7 +705,7 @@ start_here: */ initial_mmu: tlbia /* Invalidate all TLB entries */ -#ifdef CONFIG_PIN_TLB +#if 1 /* CONFIG_PIN_TLB */ lis r8, mi_rs...@h ori r8, r8, 0x1c00 #else Not nice. Either remove the config option or make sure all those code sequences are appropriately aligned so it doesn't happen. I recommend the later :-) I'll apply the other patches. Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
Benjamin Herrenschmidt b...@kernel.crashing.org wrote on 09/12/2009 05:19:59: From: Benjamin Herrenschmidt b...@kernel.crashing.org To: Joakim Tjernlund joakim.tjernl...@transmode.se Cc: Scott Wood scottw...@freescale.com, linuxppc-...@ozlabs.org linuxppc- d...@ozlabs.org, Rex Feany rfe...@mrv.com Date: 09/12/2009 05:20 Subject: Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB On Fri, 2009-11-20 at 11:21 +0100, Joakim Tjernlund wrote: Various kernel asm modifies SRR0/SRR1 just before executing a rfi. If such code crosses a page boundary you risk a TLB miss which will clobber SRR0/SRR1. Avoid this by always pinning kernel instruction TLB space. Signed-off-by: Joakim Tjernlund joakim.tjernl...@transmode.se --- arch/powerpc/kernel/head_8xx.S |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index a9f1ace..e70503d 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -705,7 +705,7 @@ start_here: */ initial_mmu: tlbia /* Invalidate all TLB entries */ -#ifdef CONFIG_PIN_TLB +#if 1 /* CONFIG_PIN_TLB */ lis r8, mi_rs...@h ori r8, r8, 0x1c00 #else Not nice. Either remove the config option or make sure all those code sequences are appropriately aligned so it doesn't happen. I recommend the later :-) The later isn't as simple :) I believe the bulk of such code in entry_32.S. Anyhow, the config option is still valid as if enabled it will pin several DTLB's too. Scott had some concerns about removing the config option completely so this was the next best thing. I'll apply the other patches. OK, great. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
Dan Malek d...@embeddedalley.com wrote on 14/11/2009 19:08:43: On Nov 14, 2009, at 2:42 AM, Joakim Tjernlund wrote: . Avoid this by always pinning kernel instruction TLB space. You may as well map the data space, too, since you have reserved the entries. Take advantage of that performance. Just to be clear, I don't reserve and DTLB entries(I think). That remains under the config flag. Also, some processor variants have very few TLB entries, and may only reserve two entries (although the flag says reserve 4). Ensure there are sufficient resources to do what you want. This is the reason the option is configurable. Scott had some concerns about pinning the data space too. That is is why I left the data TLB pinning behind the the config option. How to make better use of the remaining ITLB slots is tricky. Somehow one would want to map at lest one to modules but I cannot see how. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
On Nov 14, 2009, at 2:42 AM, Joakim Tjernlund wrote: . Avoid this by always pinning kernel instruction TLB space. You may as well map the data space, too, since you have reserved the entries. Take advantage of that performance. Also, some processor variants have very few TLB entries, and may only reserve two entries (although the flag says reserve 4). Ensure there are sufficient resources to do what you want. This is the reason the option is configurable. Thanks. -- Dan ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
Dan Malek d...@embeddedalley.com wrote on 14/11/2009 19:08:43: On Nov 14, 2009, at 2:42 AM, Joakim Tjernlund wrote: . Avoid this by always pinning kernel instruction TLB space. You may as well map the data space, too, since you have reserved the entries. Take advantage of that performance. Also, some processor variants have very few TLB entries, and may only reserve two entries (although the flag says reserve 4). Ensure there are sufficient resources to do what you want. This is the reason the option is configurable. Scott had some concerns about pinning the data space too. That is is why I left the data TLB pinning behind the the config option. How to make better use of the remaining ITLB slots is tricky. Somehow one would want to map at lest one to modules but I cannot see how. Jocke ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
On Sat, 2009-11-14 at 20:08 +0100, Joakim Tjernlund wrote: Dan Malek d...@embeddedalley.com wrote on 14/11/2009 19:08:43: On Nov 14, 2009, at 2:42 AM, Joakim Tjernlund wrote: . Avoid this by always pinning kernel instruction TLB space. You may as well map the data space, too, since you have reserved the entries. Take advantage of that performance. Also, some processor variants have very few TLB entries, and may only reserve two entries (although the flag says reserve 4). Ensure there are sufficient resources to do what you want. This is the reason the option is configurable. Scott had some concerns about pinning the data space too. That is is why I left the data TLB pinning behind the the config option. How to make better use of the remaining ITLB slots is tricky. Somehow one would want to map at lest one to modules but I cannot see how. No. If you use modules, you pay the price. Sane embedded solutions running in tight environments don't use modules :-) No point pinning TLB entries on the vmalloc space, really. What -might- be more useful is to look at Grant work on re-doing the early ioremap and providing a way to provide what the old io_block_mapping() did, but with dynamically chosen virtual addresses, to have a pinned entry covering most common IOs. Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 04/10] 8xx: Always pin kernel instruction TLB
How to make better use of the remaining ITLB slots is tricky. Somehow one would want to map at lest one to modules but I cannot see how. No. If you use modules, you pay the price. Sane embedded solutions running in tight environments don't use modules :-) No point pinning TLB entries on the vmalloc space, really. Long ago (2.4 days I think) when using modules on ppc 4xx we hacked the module_alloc function (or whatever it was called back then) to allocate space in the kernel pinned TLB instead of using vmalloc. Gave something like a 2x speedup for module code, since the 4xx TLB is so small and the miss handling is so expensive. I assume it should still be possible to do a similar hack with current kernels. - R. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
