Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
(sorry, I'm slow at replying, attending some internal ARM conference) On 31 January 2012 18:09, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 31 Jan 2012, Catalin Marinas wrote: Maybe we could factor out the CPU-specific settings from proc-v*.S into a separate arch/arm/boot/preload directory. We keep proc-v*.S entirely generic and the implementation-defined bits setting in the preload code. You could have an option to link the preload with the kernel but we could recommend that people run this code from boot-loader before invoking the kernel. This code would be dependent on platform and chosen at build-time. But the actual kernel image would be generic. I'd recommend against that. Again, the reason is not technical but rather has to do with the tendency to laziness of human beings. So please let's not go there or it'll become the de facto standard. ... The whole idea behind having a generic kernel is all about distributions. If a generic kernel has to be distributed with a platform specific pre-kernel blob then we've gained nothing. The problem we have with a generic kernel in this context (e.g. errata bits) is that even though we use a read-check-write sequence for setting various bits during __v*_setup, as long as the boot loader did not set them the kernel will try a write access and fail. So with the current scenario, there is no way for an OMAP platform to run a generic kernel in non-secure mode (with all errata workarounds enabled) without modifying the boot loader. Same goes for the DTB. Those platform specific blobs must be distributed _with_ the hardware and _not_ with the software distribution. Having the dts files in the kernel right now is only a convenience for the transition to device tree. We could do the same and move the bit enabling to separate repository :). An alternative would be to also move those bits setting at a later time in the kernel booting (and re-disabling/enabling the MMU) as we don't have an easy way of figuring out whether the kernel is running in secure or non-secure mode. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Thu, Feb 02, 2012 at 02:32:03PM +, Catalin Marinas wrote: We could do the same and move the bit enabling to separate repository :). We must certainly could do but that doesn't get around the errata issues when you have things before the kernel (or before these blobs) enabling things like the caches and MMU. The only answer which assures complete system stability is for the earliest reasonable point in the booting sequence to handle the errata, before any of the potential errata scenarios occur. For things like working around L2 cache problems, that means before the L2 cache is enabled for the first time. If the boot loader enables the L2 cache, then _it_ has to take care of the errata before it enables the L2 cache, and not some blob that it loads. If the boot loader enables the cache, and there are workarounds for buggy cache behaviour, then the boot loader has to implement those errata itself. And so the list goes on. I think the one issue which Santosh is justified about is: if that is the case, why have the errata workarounds in the kernel in the first place. I agree - it makes no sense when we have things like the decompressor enabling the caches. So, I propose that we rip out all those work-arounds that are just 'set this bit in some register' at boot time from the kernel itself right now, and reconsider how these are handled. If that means boot loader people need to update their code to deal with errata, because they want to use CPU features which have errata associated with them, they then get to deal with the errata updates themselves, and also have to carry the pain of dealing with running in non-secure mode. Or, they will have to chose to avoid using those features. But, having errata for things like DMB being faulty in the kernel after things like the boot loader will _already_ have had to issue DMB instructions, or for failing I-cache invalidate after we've already done some I-cache invalidates (eg, boot loader or the decompressor) is quite absurd. So, I think we need to rip out a fair number of these errata from the kernel itself; it's quite clear they're already being done far too late in the system boot sequence. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On 2 February 2012 14:49, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Thu, Feb 02, 2012 at 02:32:03PM +, Catalin Marinas wrote: We could do the same and move the bit enabling to separate repository :). We must certainly could do but that doesn't get around the errata issues when you have things before the kernel (or before these blobs) enabling things like the caches and MMU. That was my original point - allow such code to be built into a blob that the boot loader can execute. But I got your point that not all boot loaders are able to execute two binary images. The only answer which assures complete system stability is for the earliest reasonable point in the booting sequence to handle the errata, before any of the potential errata scenarios occur. For things like working around L2 cache problems, that means before the L2 cache is enabled for the first time. If the boot loader enables the L2 cache, then _it_ has to take care of the errata before it enables the L2 cache, and not some blob that it loads. If the boot loader enables the cache, and there are workarounds for buggy cache behaviour, then the boot loader has to implement those errata itself. But as it was pointed out already, many of those are very rare conditions that you can't actually trigger during the boot sequence. There are other errata that require SMP for example and the boot loader (or decompressor) doesn't care. So, I propose that we rip out all those work-arounds that are just 'set this bit in some register' at boot time from the kernel itself right now, and reconsider how these are handled. Just don't remove them until we agree where we should *move* them. They are still good documentation so far. I would say just disable the workarounds in the kernel .config until we decide. But, having errata for things like DMB being faulty in the kernel after things like the boot loader will _already_ have had to issue DMB instructions, or for failing I-cache invalidate after we've already done some I-cache invalidates (eg, boot loader or the decompressor) is quite absurd. As I said, most errata are really hard to trigger (but ARM cannot guarantee that it is impossible, so they need to be published). -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On 31 January 2012 07:38, Shilimkar, Santosh santosh.shilim...@ti.com wrote: On Tue, Jan 31, 2012 at 1:01 PM, Catalin Marinas catalin.mari...@arm.com wrote: On 31 January 2012 05:21, Aneesh V ane...@ti.com wrote: On Friday 27 January 2012 11:00 PM, Catalin Marinas wrote: On Fri, Jan 20, 2012 at 08:57:11AM +, Joe Woodward wrote: So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? Probably not, because it depends on many variables. See below my take on this. I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) This depends on whether the L2 is inner or outer: L2 inner - leave it enabled in the boot loader L2 outer - leave it disabled in the boot loader 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Same as above: L2 inner - don't do anything, it gets used when SCTLR.M is enabled L2 outer - enabled at boot time via the platform code (later, after MMU was enabled). What is the reasoning behind this recommendation? Why the distinction between L2 being inner or outer. I don't see anything to this effect in the Cortex-A8 TRM? In fact the only recommendation I could find(section 8.3) is asking to set L2EN to 1 before setting C bit to 1 irrespective of inner/outer? Actually you have a good point. I was thinking about outer caches like PL310 or L220 that Linux enables later during platform initialisation. They could be left enabled or disabled at boot time but they must be cleaned/invalidated (you would need to flush after disabling just to make sure there are no further lines fetched; alternatively, there shouldn't be any cacheable mappings). So I would say in the context of A8, just leave the L2 enabled in the boot loader. As long as the MMU is disabled, it won't be used. The alternative is to enable L2 slightly later in Linux in the platform code to avoid SoC-dependent code in proc-v7.S. So in summary, either option is possible. You could take the approach of the cache-l2x0.c - check whether it is already enabled and, if not, issue an SMC from the platform code to enable the L2 cache. if done late in kernel, wouldn't it violate set L2EN to 1 before setting C bit to 1 Not if you write the code sequence in such a way that it also switches the C bit to 0 and back to 1. I think Nicolas suggested that the platform code could go through the power management code. But I think with some carefully coded asm code and cache flushing you could set L2EN without a full MMU reset. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 31, 2012 at 2:24 PM, Catalin Marinas catalin.mari...@arm.com wrote: On 31 January 2012 07:38, Shilimkar, Santosh santosh.shilim...@ti.com wrote: On Tue, Jan 31, 2012 at 1:01 PM, Catalin Marinas catalin.mari...@arm.com wrote: On 31 January 2012 05:21, Aneesh V ane...@ti.com wrote: On Friday 27 January 2012 11:00 PM, Catalin Marinas wrote: On Fri, Jan 20, 2012 at 08:57:11AM +, Joe Woodward wrote: So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? Probably not, because it depends on many variables. See below my take on this. I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) This depends on whether the L2 is inner or outer: L2 inner - leave it enabled in the boot loader L2 outer - leave it disabled in the boot loader 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Same as above: L2 inner - don't do anything, it gets used when SCTLR.M is enabled L2 outer - enabled at boot time via the platform code (later, after MMU was enabled). What is the reasoning behind this recommendation? Why the distinction between L2 being inner or outer. I don't see anything to this effect in the Cortex-A8 TRM? In fact the only recommendation I could find(section 8.3) is asking to set L2EN to 1 before setting C bit to 1 irrespective of inner/outer? Actually you have a good point. I was thinking about outer caches like PL310 or L220 that Linux enables later during platform initialisation. They could be left enabled or disabled at boot time but they must be cleaned/invalidated (you would need to flush after disabling just to make sure there are no further lines fetched; alternatively, there shouldn't be any cacheable mappings). So I would say in the context of A8, just leave the L2 enabled in the boot loader. As long as the MMU is disabled, it won't be used. The alternative is to enable L2 slightly later in Linux in the platform code to avoid SoC-dependent code in proc-v7.S. So in summary, either option is possible. You could take the approach of the cache-l2x0.c - check whether it is already enabled and, if not, issue an SMC from the platform code to enable the L2 cache. if done late in kernel, wouldn't it violate set L2EN to 1 before setting C bit to 1 Not if you write the code sequence in such a way that it also switches the C bit to 0 and back to 1. I think Nicolas suggested that the platform code could go through the power management code. But I think with some carefully coded asm code and cache flushing you could set L2EN without a full MMU reset. That would need identity mapping and also correct cache flushing as you said above. L2 enabled for A8 is fine but not sure about the A15 where L2 is enabled with C bit. Then there are some settings for ACTLR, L2ACCTRL and L2PFR which needs to be setup before MMU is ebabled. And then the errata's on A9, needing setting up dignostic register, changing cache policy etc Hopefully we don't get hit by those issues/erratas in the window from initial MMU setup to this new sequence code. Will this approach be ok for other cases I mentioned ? Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On 31 January 2012 09:05, Shilimkar, Santosh santosh.shilim...@ti.com wrote: On Tue, Jan 31, 2012 at 2:24 PM, Catalin Marinas catalin.mari...@arm.com wrote: On 31 January 2012 07:38, Shilimkar, Santosh santosh.shilim...@ti.com wrote: On Tue, Jan 31, 2012 at 1:01 PM, Catalin Marinas catalin.mari...@arm.com wrote: On 31 January 2012 05:21, Aneesh V ane...@ti.com wrote: On Friday 27 January 2012 11:00 PM, Catalin Marinas wrote: On Fri, Jan 20, 2012 at 08:57:11AM +, Joe Woodward wrote: So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? Probably not, because it depends on many variables. See below my take on this. I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) This depends on whether the L2 is inner or outer: L2 inner - leave it enabled in the boot loader L2 outer - leave it disabled in the boot loader 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Same as above: L2 inner - don't do anything, it gets used when SCTLR.M is enabled L2 outer - enabled at boot time via the platform code (later, after MMU was enabled). What is the reasoning behind this recommendation? Why the distinction between L2 being inner or outer. I don't see anything to this effect in the Cortex-A8 TRM? In fact the only recommendation I could find(section 8.3) is asking to set L2EN to 1 before setting C bit to 1 irrespective of inner/outer? Actually you have a good point. I was thinking about outer caches like PL310 or L220 that Linux enables later during platform initialisation. They could be left enabled or disabled at boot time but they must be cleaned/invalidated (you would need to flush after disabling just to make sure there are no further lines fetched; alternatively, there shouldn't be any cacheable mappings). So I would say in the context of A8, just leave the L2 enabled in the boot loader. As long as the MMU is disabled, it won't be used. The alternative is to enable L2 slightly later in Linux in the platform code to avoid SoC-dependent code in proc-v7.S. So in summary, either option is possible. You could take the approach of the cache-l2x0.c - check whether it is already enabled and, if not, issue an SMC from the platform code to enable the L2 cache. if done late in kernel, wouldn't it violate set L2EN to 1 before setting C bit to 1 Not if you write the code sequence in such a way that it also switches the C bit to 0 and back to 1. I think Nicolas suggested that the platform code could go through the power management code. But I think with some carefully coded asm code and cache flushing you could set L2EN without a full MMU reset. That would need identity mapping and also correct cache flushing as you said above. L2 enabled for A8 is fine but not sure about the A15 where L2 is enabled with C bit. I don't see any problem here - why would you want to treat the L2 separately form the L1 cache? They are both inner caches, just treat them as such - enabled when the C and M bits are set. Then there are some settings for ACTLR, L2ACCTRL and L2PFR which needs to be setup before MMU is ebabled. These in general should go in the boot monitor/loader. There isn't other easy clean solution if we want to keep the kernel booting generic. Maybe we could factor out the CPU-specific settings from proc-v*.S into a separate arch/arm/boot/preload directory. We keep proc-v*.S entirely generic and the implementation-defined bits setting in the preload code. You could have an option to link the preload with the kernel but we could recommend that people run this code from boot-loader before invoking the kernel. This code would be dependent on platform and chosen at build-time. But the actual kernel image would be generic. And then the errata's on A9, needing setting up dignostic register, changing cache policy etc Hopefully we don't get hit by those issues/erratas in the window from initial MMU setup to this new sequence code. We already enable the MMU in the decompressor and it doesn't have any knowledge of CPU errata. But the preload code could solve this as well. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 31, 2012 at 02:35:51PM +0530, Shilimkar, Santosh wrote:
On Tue, Jan 31, 2012 at 2:24 PM, Catalin Marinas
catalin.mari...@arm.com wrote:
Not if you write the code sequence in such a way that it also switches
the C bit to 0 and back to 1. I think Nicolas suggested that the
platform code could go through the power management code. But I think
with some carefully coded asm code and cache flushing you could set
L2EN without a full MMU reset.
That would need identity mapping and also correct cache flushing
as you said above. L2 enabled for A8 is fine but not sure about the
A15 where L2 is enabled with C bit. Then there are some settings for
ACTLR, L2ACCTRL and L2PFR which needs to be setup before
MMU is ebabled.
And then the errata's on A9, needing setting up dignostic register,
changing cache policy etc
Hopefully we don't get hit by those issues/erratas
in the window from initial MMU setup to this new sequence code.
Will this approach be ok for other cases I mentioned ?
Santosh,
It is very clear to me from the discussions, and the discussions I've
had with folk in ARM, that everyone thinks that the idea of putting
a platform specific hook in the early bring up code is insane.
Everyone seems to think that the place it is to be solved is either
in the boot loader or with a wrapper around the kernel.
One of my points that I brought up against it was the limited environment,
and the lack of stack at that point. Let's look at what you're
proposing to do (thanks for the patch):
Your secure hook:
stmia r12, {r1-r11, r13-r14}
The stack you're writing to:
.align 2
__v7_setup_stack:
.space 4 * 11 @ 11 registers
And it's layout in the kernel image:
c02ee75c __v7_setup_stack:
...
c02ee788: 0c08.word 0x0c08
c02ee78c: 0c09.word 0x0c09
c02ee790: ff0a81a8.word 0xff0a81a8
c02ee794: 40e040e0.word 0x40e040e0
c02ee798 gic_secondary_init:
c02ee798: e1a0c00dmov ip, sp
...
You're lucky that what you're _only_ overwriting is the two Cortex ID
values which have already been used, and not the following words, which
are the control register masks and the GIC initialization function.
Thank you for proving my point about why platform code can't be allowed
to get its grubby hands that early in the kernel boot - you are making
it more fragile by doing this stuff, because you haven't taken the time
to properly analyse the code before you've modified it. As such, you
will see more silent boot failures.
So, as many people have said, we're not going to go down the route of
allowing platforms to hook into this code. There's plenty of other
solutions to this problem in different parts of the system that would
be more than adequate, and would benefit everyone not just the kernel.
You're making bones about whether there's a window if you do it later
in the kernel. You already have a window of opportunity for things to
have gone wrong and that's the zImage decompressor, which runs with
caches on. So if you've managed to get to your existing code in the
v7 setup function, then you've already proven that it's fine.
Your continued resistance to every alternative suggestion but your
(broken) hook is getting extremely frustrating. Please try some of
these other approaches which are being proposed. Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-omap in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 31, 2012 at 09:53:25AM +, Catalin Marinas wrote: Maybe we could factor out the CPU-specific settings from proc-v*.S into a separate arch/arm/boot/preload directory. We keep proc-v*.S entirely generic and the implementation-defined bits setting in the preload code. You could have an option to link the preload with the kernel but we could recommend that people run this code from boot-loader before invoking the kernel. You've seen the resistance from uboot to stuff we do with the kernel, this will be no different. There's also quite a number of boot loaders out there which aren't capable of running a binary, returning and then loading and running the kernel. Plus, of course, it's far from friendly towards TFTP setups. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 31, 2012 at 3:26 PM, Russell King - ARM Linux
li...@arm.linux.org.uk wrote:
On Tue, Jan 31, 2012 at 02:35:51PM +0530, Shilimkar, Santosh wrote:
On Tue, Jan 31, 2012 at 2:24 PM, Catalin Marinas
catalin.mari...@arm.com wrote:
Not if you write the code sequence in such a way that it also switches
the C bit to 0 and back to 1. I think Nicolas suggested that the
platform code could go through the power management code. But I think
with some carefully coded asm code and cache flushing you could set
L2EN without a full MMU reset.
That would need identity mapping and also correct cache flushing
as you said above. L2 enabled for A8 is fine but not sure about the
A15 where L2 is enabled with C bit. Then there are some settings for
ACTLR, L2ACCTRL and L2PFR which needs to be setup before
MMU is ebabled.
And then the errata's on A9, needing setting up dignostic register,
changing cache policy etc
Hopefully we don't get hit by those issues/erratas
in the window from initial MMU setup to this new sequence code.
Will this approach be ok for other cases I mentioned ?
Santosh,
It is very clear to me from the discussions, and the discussions I've
had with folk in ARM, that everyone thinks that the idea of putting
a platform specific hook in the early bring up code is insane.
Everyone seems to think that the place it is to be solved is either
in the boot loader or with a wrapper around the kernel.
One of my points that I brought up against it was the limited environment,
and the lack of stack at that point. Let's look at what you're
proposing to do (thanks for the patch):
Your secure hook:
stmia r12, {r1-r11, r13-r14}
The stack you're writing to:
.align 2
__v7_setup_stack:
.space 4 * 11 @ 11 registers
And it's layout in the kernel image:
c02ee75c __v7_setup_stack:
...
c02ee788: 0c08 .word 0x0c08
c02ee78c: 0c09 .word 0x0c09
c02ee790: ff0a81a8 .word 0xff0a81a8
c02ee794: 40e040e0 .word 0x40e040e0
c02ee798 gic_secondary_init:
c02ee798: e1a0c00d mov ip, sp
...
You're lucky that what you're _only_ overwriting is the two Cortex ID
values which have already been used, and not the following words, which
are the control register masks and the GIC initialization function.
Thank you for proving my point about why platform code can't be allowed
to get its grubby hands that early in the kernel boot - you are making
it more fragile by doing this stuff, because you haven't taken the time
to properly analyse the code before you've modified it. As such, you
will see more silent boot failures.
So, as many people have said, we're not going to go down the route of
allowing platforms to hook into this code. There's plenty of other
solutions to this problem in different parts of the system that would
be more than adequate, and would benefit everyone not just the kernel.
Sure. The marco has missed those details. Thanks for the
useful information. At least I can fix the stack size for internal
use.
You're making bones about whether there's a window if you do it later
in the kernel. You already have a window of opportunity for things to
have gone wrong and that's the zImage decompressor, which runs with
caches on. So if you've managed to get to your existing code in the
v7 setup function, then you've already proven that it's fine.
I see your point.
Your continued resistance to every alternative suggestion but your
(broken) hook is getting extremely frustrating. Please try some of
these other approaches which are being proposed. Thanks.
I was just trying to get more information about the alternatives and
trying to bring out possible issues. I know for sure that boot-loader is
not an option since we have had many instances where it has not worked.
I also understand that patching early common code is going to be tricky
and of-course against the single zImage.
So the option mentioned by Nicolas and Catalin about 1:1 mapping
and configuring the registers in platform specific code was looking
a way forward. So was asking more questions about whether this can
work in all cases. Specifically for A15. As per Catalin's email, it is
best to handle those cases before MMU is enabled with
boot-monitor or pre-load code.
I am not against trying option. Just trying to understand what is the
best option we have.
Regards
santosh
--
To unsubscribe from this list: send the line unsubscribe linux-omap in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On 31 January 2012 10:10, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Tue, Jan 31, 2012 at 09:53:25AM +, Catalin Marinas wrote: Maybe we could factor out the CPU-specific settings from proc-v*.S into a separate arch/arm/boot/preload directory. We keep proc-v*.S entirely generic and the implementation-defined bits setting in the preload code. You could have an option to link the preload with the kernel but we could recommend that people run this code from boot-loader before invoking the kernel. You've seen the resistance from uboot to stuff we do with the kernel, this will be no different. There's also quite a number of boot loaders out there which aren't capable of running a binary, returning and then loading and running the kernel. Plus, of course, it's far from friendly towards TFTP setups. For platforms that can't/don't want to modify the boot loader, they could wrap (prepend) the kernel image with the preload code and they just use that to kexec a distro generic kernel. I know it can go to 3-4 stages boot process on some platforms but you have the advantage of a generic kernel image. If SoC vendors don't like the lengthier boot process, they have the option of modifying the boot loader and include the preload code that the kernel provides (or find a way to load it together with the kernel at boot-time). -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, 31 Jan 2012, Catalin Marinas wrote: Maybe we could factor out the CPU-specific settings from proc-v*.S into a separate arch/arm/boot/preload directory. We keep proc-v*.S entirely generic and the implementation-defined bits setting in the preload code. You could have an option to link the preload with the kernel but we could recommend that people run this code from boot-loader before invoking the kernel. This code would be dependent on platform and chosen at build-time. But the actual kernel image would be generic. I'd recommend against that. Again, the reason is not technical but rather has to do with the tendency to laziness of human beings. So please let's not go there or it'll become the de facto standard. Furthermore, if this is risky to leave a vulnerable window by activating some workarounds later during the boot process, then the risk won't go away entirely by moving those workarounds a bit earlier. In which case the only real solution is to have them enabled by the bootloader before the cache is even turned on for the first time, and this is hardly Linux specific anyway. The whole idea behind having a generic kernel is all about distributions. If a generic kernel has to be distributed with a platform specific pre-kernel blob then we've gained nothing. Same goes for the DTB. Those platform specific blobs must be distributed _with_ the hardware and _not_ with the software distribution. Having the dts files in the kernel right now is only a convenience for the transition to device tree. Eventually they all will be removed from the tree when we get to a minimum of stability in the device tree implementation and the DTB will also have to come with the hardware along with the bootloader. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, 31 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 31, 2012 at 3:26 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: So, as many people have said, we're not going to go down the route of allowing platforms to hook into this code. There's plenty of other solutions to this problem in different parts of the system that would be more than adequate, and would benefit everyone not just the kernel. Sure. The marco has missed those details. Thanks for the useful information. At least I can fix the stack size for internal use. If you intend to ignore our advises and continue with the quick and dirty solution for your internal tree, then at least don't brag on a public list about it. Your continued resistance to every alternative suggestion but your (broken) hook is getting extremely frustrating. Please try some of these other approaches which are being proposed. Thanks. I was just trying to get more information about the alternatives and trying to bring out possible issues. I know for sure that boot-loader is not an option since we have had many instances where it has not worked. I also understand that patching early common code is going to be tricky and of-course against the single zImage. So the option mentioned by Nicolas and Catalin about 1:1 mapping and configuring the registers in platform specific code was looking a way forward. So was asking more questions about whether this can work in all cases. Specifically for A15. So far this is apparently your best option. And luckily the code to create a 1:1 mapping, turn off MMU, then turn MMU back on and tear down the 1:1 mapping already exists. All you need to do is to insert the cache workaround activation in the middle of that instead of shutting the CPU down. As per Catalin's email, it is best to handle those cases before MMU is enabled with boot-monitor or pre-load code. These days bootloaders are turning the MMU on for themselves, so you'll need pre-load code for the bootloader already. Either that, or the issue is not necessarily _that_ critical in which case you can afford to enable your workaround a bit later during the kernel boot. Nicolas
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 31, 2012 at 11:57 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 31 Jan 2012, Shilimkar, Santosh wrote: [...] I also understand that patching early common code is going to be tricky and of-course against the single zImage. So the option mentioned by Nicolas and Catalin about 1:1 mapping and configuring the registers in platform specific code was looking a way forward. So was asking more questions about whether this can work in all cases. Specifically for A15. So far this is apparently your best option. And luckily the code to create a 1:1 mapping, turn off MMU, then turn MMU back on and tear down the 1:1 mapping already exists. All you need to do is to insert the cache workaround activation in the middle of that instead of shutting the CPU down. thanks. As per Catalin's email, it is best to handle those cases before MMU is enabled with boot-monitor or pre-load code. These days bootloaders are turning the MMU on for themselves, so you'll need pre-load code for the bootloader already. Either that, or the issue is not necessarily _that_ critical in which case you can afford to enable your workaround a bit later during the kernel boot. This is indeed a good point. The mainline u-boot(denx) now a days have MMU support so we need to have the necessary WA before that code anyways. We will attempt the 1:1 mapping option as well but to be 100 % safe for the previous 3 statges of MMU ON ( 1. Bootloader, 2. De-compressor and 3. Kernel setup), looks like moving all of this security settings before all 3 stages is most appropriate. Thanks for the good discussion. Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
Hi Catalin, On Friday 27 January 2012 11:00 PM, Catalin Marinas wrote: On Fri, Jan 20, 2012 at 08:57:11AM +, Joe Woodward wrote: So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? Probably not, because it depends on many variables. See below my take on this. I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) This depends on whether the L2 is inner or outer: L2 inner - leave it enabled in the boot loader L2 outer - leave it disabled in the boot loader 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Same as above: L2 inner - don't do anything, it gets used when SCTLR.M is enabled L2 outer - enabled at boot time via the platform code (later, after MMU was enabled). What is the reasoning behind this recommendation? Why the distinction between L2 being inner or outer. I don't see anything to this effect in the Cortex-A8 TRM? In fact the only recommendation I could find(section 8.3) is asking to set L2EN to 1 before setting C bit to 1 irrespective of inner/outer? br, Aneesh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On 31 January 2012 05:21, Aneesh V ane...@ti.com wrote: On Friday 27 January 2012 11:00 PM, Catalin Marinas wrote: On Fri, Jan 20, 2012 at 08:57:11AM +, Joe Woodward wrote: So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? Probably not, because it depends on many variables. See below my take on this. I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) This depends on whether the L2 is inner or outer: L2 inner - leave it enabled in the boot loader L2 outer - leave it disabled in the boot loader 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Same as above: L2 inner - don't do anything, it gets used when SCTLR.M is enabled L2 outer - enabled at boot time via the platform code (later, after MMU was enabled). What is the reasoning behind this recommendation? Why the distinction between L2 being inner or outer. I don't see anything to this effect in the Cortex-A8 TRM? In fact the only recommendation I could find(section 8.3) is asking to set L2EN to 1 before setting C bit to 1 irrespective of inner/outer? Actually you have a good point. I was thinking about outer caches like PL310 or L220 that Linux enables later during platform initialisation. They could be left enabled or disabled at boot time but they must be cleaned/invalidated (you would need to flush after disabling just to make sure there are no further lines fetched; alternatively, there shouldn't be any cacheable mappings). So I would say in the context of A8, just leave the L2 enabled in the boot loader. As long as the MMU is disabled, it won't be used. The alternative is to enable L2 slightly later in Linux in the platform code to avoid SoC-dependent code in proc-v7.S. So in summary, either option is possible. You could take the approach of the cache-l2x0.c - check whether it is already enabled and, if not, issue an SMC from the platform code to enable the L2 cache. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 31, 2012 at 1:01 PM, Catalin Marinas catalin.mari...@arm.com wrote: On 31 January 2012 05:21, Aneesh V ane...@ti.com wrote: On Friday 27 January 2012 11:00 PM, Catalin Marinas wrote: On Fri, Jan 20, 2012 at 08:57:11AM +, Joe Woodward wrote: So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? Probably not, because it depends on many variables. See below my take on this. I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) This depends on whether the L2 is inner or outer: L2 inner - leave it enabled in the boot loader L2 outer - leave it disabled in the boot loader 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Same as above: L2 inner - don't do anything, it gets used when SCTLR.M is enabled L2 outer - enabled at boot time via the platform code (later, after MMU was enabled). What is the reasoning behind this recommendation? Why the distinction between L2 being inner or outer. I don't see anything to this effect in the Cortex-A8 TRM? In fact the only recommendation I could find(section 8.3) is asking to set L2EN to 1 before setting C bit to 1 irrespective of inner/outer? Actually you have a good point. I was thinking about outer caches like PL310 or L220 that Linux enables later during platform initialisation. They could be left enabled or disabled at boot time but they must be cleaned/invalidated (you would need to flush after disabling just to make sure there are no further lines fetched; alternatively, there shouldn't be any cacheable mappings). So I would say in the context of A8, just leave the L2 enabled in the boot loader. As long as the MMU is disabled, it won't be used. The alternative is to enable L2 slightly later in Linux in the platform code to avoid SoC-dependent code in proc-v7.S. So in summary, either option is possible. You could take the approach of the cache-l2x0.c - check whether it is already enabled and, if not, issue an SMC from the platform code to enable the L2 cache. if done late in kernel, wouldn't it violate set L2EN to 1 before setting C bit to 1 -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
Sorry for the shameless ping, but I'd really like to know the way forward here... -Original Message- From: Joe Woodward j...@terrafix.co.uk To: Shilimkar, Santosh santosh.shilim...@ti.com, Catalin Marinas catalin.mari...@arm.com Cc: Aneesh V ane...@ti.com, Russell King - ARM Linux li...@arm.linux.org.uk, linux-omap@vger.kernel.org, linux-arm linux-arm- ker...@lists.infradead.org Date: Fri, 20 Jan 2012 08:57:11 + Subject: Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)? So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Is there any concensus? Cheers, Joe Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Fri, Jan 20, 2012 at 08:57:11AM +, Joe Woodward wrote: So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? Probably not, because it depends on many variables. See below my take on this. I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) This depends on whether the L2 is inner or outer: L2 inner - leave it enabled in the boot loader L2 outer - leave it disabled in the boot loader 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Same as above: L2 inner - don't do anything, it gets used when SCTLR.M is enabled L2 outer - enabled at boot time via the platform code (later, after MMU was enabled). -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
So I re-iterate that we need to have solution to this problem. ... I don't want to be a pain, but it seems to me that this dicussion didn't reach a full conclussion? I think it was left with the open options being: 1) Leave the L2/outer cache enabled in the bootloader (not ideal and may cause problems with future devices) 2) Turn the L2/outer cache on for OMAP3 later in the kernel boot when the device is known Is there any concensus? Cheers, Joe Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 10:02 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 9:45 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 8:39 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: How about allowing platform hooks for single SOC builds. I mean having this code under !single_zImage or something like that. That way we don't impact the single zImage efforts and also allow socs to have all those critical, vital bits enabled for the SOC specific builds. Absolutely not! Because if we start doing that, people will get lazy and no platform will ever work in a multi-SOC kernel. If your SOC require some fancy setup that is not shared by other platforms then please abstract that into the bootloader, or make sure it can be deferred later on. There is nothing fancy here. It's an ARM security architecture feature which OMAP implements. Have given enough reason about boot-loaders issues. I was not convinced by those reasons. Just push harder on the bootloader side. There is _no_ reason for the bootloader not to take care of this very platform specific issue. You can even do it into a standalone uImage that returns to u-Boot after it is done with its magic. Is OMAP getting beaten up here just because it uses ARM security feature and implements it's mechanics? I don't care if this is OMAP, UX500 or i.MX. There is a line to be drawn and this is just too bad if you are trying to cross it. Well I was trying to get a solution for a genuine problem. Let's take an example. #ifdef CONFIG_ARM_ERRATA_742230 cmp r6, #0x22 @ only present up to r2p2 mrcle p15, 0, r10, c15, c0, 1 @ read diagnostic register orrle r10, r10, #1 4 @ set bit #4 mcrle p15, 0, r10, c15, c0, 1 @ write diagnostic register #endif The SoC's which doesn't have security restriction and the kernel is in secure mode, above errata is applied in it's right place. Before MMU is enabled. There are many bits like this which are patched in kernel. On OMAP, all these can't be set-up in kernel since kernel is in non-secure mode. To set such bit's we need to call some pre-defined security calls. Absolutely. Just don't do it in the early kernel boot code where it is not convenient to do so. Got it. Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
...snip... Fair point. It will be harder to maintain and won't be consistent. Am not sure what you mean because secure API as such isn't a problem. If you mean one standard interface for all the ARM SOC's then that's something won't be easy to handled because it is tied up the security architecture which can vary across SoCs. The latter. This is exactly the kind of pain I forsaw with this security shite, and when I heard about it I was utterly dismayed at ARM Ltd for coming up with such a brain-dead lack of design here. You're having to struggle with the results of that by having lots of SoC specific hooks all around the place to fiddle with this that and the other. Your need to have something called from the early assembly code is just yet more of that disease caused by ARM Ltd's lack of forsight on this matter. I have no solution for you on this I managed use some secure macro kind of code but as you said it will be really hard to maintain. So we are stuck with this issue then. So, is the upshot of this that the kernel isn't going to be in a position to enable the L2/outer cache on OMAP3 (due to the need for hacky/unmaintainable code)? Hence the bootloader/uBoot had better leave it enabled... Cheers, Joe Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
Santosh, Russel, On Monday 16 January 2012 06:52 PM, Shilimkar, Santosh wrote: On Mon, Jan 16, 2012 at 2:13 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Mon, Jan 16, 2012 at 01:43:03PM +0100, Shilimkar, Santosh wrote: This code will be in assembly and that's what I have been using. Not having stack shoudn't be a blocker and can be work-around in this code. And this API has to be anyway called before MMU is enabled. What about SMC on OMAP potentially corrupting most of the integer registers? What if it corrupts an integer register which we depend upon? You don't have a stack here to save those registers. We'd need at least one additional register to save the old 'lr' value. That's right. What about other secure monitors? Some implementations require you to give parameters via memory. How do you obtain that memory that early on in the kernel boot (you haven't parsed anything here.) Not only that but you're asking to make the kernel boot a _lot_ more fragile, when people start stuffing all kinds of utter shite into this hook - and then we're burdened again with the old 'my kernel won't boot and it remains silent, well your debug code is utter shite get rid of it' crap that we used to have in the early 2000s. I really don't want to go anywhere near that situation. Fair point. It will be harder to maintain and won't be consistent. Am not sure what you mean because secure API as such isn't a problem. If you mean one standard interface for all the ARM SOC's then that's something won't be easy to handled because it is tied up the security architecture which can vary across SoCs. The latter. This is exactly the kind of pain I forsaw with this security shite, and when I heard about it I was utterly dismayed at ARM Ltd for coming up with such a brain-dead lack of design here. You're having to struggle with the results of that by having lots of SoC specific hooks all around the place to fiddle with this that and the other. Your need to have something called from the early assembly code is just yet more of that disease caused by ARM Ltd's lack of forsight on this matter. I have no solution for you on this I managed use some secure macro kind of code but as you said it will be really hard to maintain. So we are stuck with this issue then. Can't we delay the L2 initialization a bit and do it when we have a stack, like we do in OMAP4 (early_initcall)? Am I missing something? br, Aneesh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 08:54:44AM +, Joe Woodward wrote: So, is the upshot of this that the kernel isn't going to be in a position to enable the L2/outer cache on OMAP3 (due to the need for hacky/unmaintainable code)? Hence the bootloader/uBoot had better leave it enabled... It could but the Linux decompressor needs to be aware and either flush the L2 (more difficult as it doesn't have all the device information) or set the page table attributes to outer non-cacheable (TEX[2:0] = 0b100). The latter may still not work if there are stale L2 cache lines left by U-Boot (and that's always possible unless U-Boot also uses outer non-cacheable memory attributes). But I would agree with Aneesh - can we not enabled the L2 at a later point? -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
Hi Catalin, On Tuesday 17 January 2012 05:41 PM, Catalin Marinas wrote: On Tue, Jan 17, 2012 at 08:54:44AM +, Joe Woodward wrote: So, is the upshot of this that the kernel isn't going to be in a position to enable the L2/outer cache on OMAP3 (due to the need for hacky/unmaintainable code)? Hence the bootloader/uBoot had better leave it enabled... It could but the Linux decompressor needs to be aware and either flush the L2 (more difficult as it doesn't have all the device information) or Cortex-A8 is aware of L2$ and can flush it, isn't it? set the page table attributes to outer non-cacheable (TEX[2:0] = 0b100). If the above is right, this is not needed right? The latter may still not work if there are stale L2 cache lines left by U-Boot (and that's always possible unless U-Boot also uses outer non-cacheable memory attributes). U-Boot flushes the caches before disabling it. On top of it, it does an invalidate after the disabling the caches to cover some corner case. So, it's guaranteed that there won't be any stale entries in L2 after U-Boot. So, we could as well leave L2$ enabled (and invalidated) and caches disabled globally after U-Boot. But I thought enabling L2$ again in kernel is the cleaner solution. br, Aneesh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 1:27 PM, Aneesh V ane...@ti.com wrote: Hi Catalin, On Tuesday 17 January 2012 05:41 PM, Catalin Marinas wrote: On Tue, Jan 17, 2012 at 08:54:44AM +, Joe Woodward wrote: So, is the upshot of this that the kernel isn't going to be in a position to enable the L2/outer cache on OMAP3 (due to the need for hacky/unmaintainable code)? Hence the bootloader/uBoot had better leave it enabled... It could but the Linux decompressor needs to be aware and either flush the L2 (more difficult as it doesn't have all the device information) or Cortex-A8 is aware of L2$ and can flush it, isn't it? set the page table attributes to outer non-cacheable (TEX[2:0] = 0b100). If the above is right, this is not needed right? Well the L2 can be configured as inner or outer, so above alone won't work. Boot-loader disabling L2 cache ( all caches) is still right thing and that's what kernel expect. Since the early kernel code can't be patches for A8, may be delaying L2 enabled would work. But then on A15, we are back to square one since there is no control to turn ON/OFF l2 cache. On A15 infact there are other CP15 registers which needs to be set before MMU is enabled to have best configuration. So the need of this early hook is pretty much there. May be separate CPU setup code for such cases is a possible way to go. Basically the problem needs to be sorted out, because I already see this is needed for A15 as well Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 12:40:54PM +, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 1:27 PM, Aneesh V ane...@ti.com wrote: Hi Catalin, On Tuesday 17 January 2012 05:41 PM, Catalin Marinas wrote: On Tue, Jan 17, 2012 at 08:54:44AM +, Joe Woodward wrote: So, is the upshot of this that the kernel isn't going to be in a position to enable the L2/outer cache on OMAP3 (due to the need for hacky/unmaintainable code)? Hence the bootloader/uBoot had better leave it enabled... It could but the Linux decompressor needs to be aware and either flush the L2 (more difficult as it doesn't have all the device information) or Cortex-A8 is aware of L2$ and can flush it, isn't it? set the page table attributes to outer non-cacheable (TEX[2:0] = 0b100). If the above is right, this is not needed right? Well the L2 can be configured as inner or outer, so above alone won't work. Boot-loader disabling L2 cache ( all caches) is still right thing and that's what kernel expect. Since the early kernel code can't be patches for A8, may be delaying L2 enabled would work. Ah, I missed the fact that the L2 is an inner cache on OMAP3+A8. In this case, I would argue to just leave it enabled since as long as the MMU is off it won't be accessed. Why does U-Boot need to disable the L2 (I suspect via some auxiliary control register)? But then on A15, we are back to square one since there is no control to turn ON/OFF l2 cache. Well, it's an inner cache, it gets disabled/enabled via the SCTLR.M bit as is the L1 cache. It does not need to be disabled independently. On A15 infact there are other CP15 registers which needs to be set before MMU is enabled to have best configuration. Yes, there are, and my view is that it is up to the platform firmware to set things right. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 12:27:25PM +, Aneesh V wrote: Hi Catalin, On Tuesday 17 January 2012 05:41 PM, Catalin Marinas wrote: On Tue, Jan 17, 2012 at 08:54:44AM +, Joe Woodward wrote: So, is the upshot of this that the kernel isn't going to be in a position to enable the L2/outer cache on OMAP3 (due to the need for hacky/unmaintainable code)? Hence the bootloader/uBoot had better leave it enabled... It could but the Linux decompressor needs to be aware and either flush the L2 (more difficult as it doesn't have all the device information) or Cortex-A8 is aware of L2$ and can flush it, isn't it? As I said to Santosh, I only had the outer cache in mind (e.g. PL310). There is no extra configuration needed in the kernel decompressor in this case. set the page table attributes to outer non-cacheable (TEX[2:0] = 0b100). If the above is right, this is not needed right? Correct, since L2 is inner cache. The latter may still not work if there are stale L2 cache lines left by U-Boot (and that's always possible unless U-Boot also uses outer non-cacheable memory attributes). U-Boot flushes the caches before disabling it. On top of it, it does an invalidate after the disabling the caches to cover some corner case. So, it's guaranteed that there won't be any stale entries in L2 after U-Boot. So, we could as well leave L2$ enabled (and invalidated) and caches disabled globally after U-Boot. Yes. But I thought enabling L2$ again in kernel is the cleaner solution. Why? It gets enabled via SCTLR.M together with L1. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 2:39 PM, Catalin Marinas catalin.mari...@arm.com wrote: On Tue, Jan 17, 2012 at 12:40:54PM +, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 1:27 PM, Aneesh V ane...@ti.com wrote: Hi Catalin, On Tuesday 17 January 2012 05:41 PM, Catalin Marinas wrote: On Tue, Jan 17, 2012 at 08:54:44AM +, Joe Woodward wrote: So, is the upshot of this that the kernel isn't going to be in a position to enable the L2/outer cache on OMAP3 (due to the need for hacky/unmaintainable code)? Hence the bootloader/uBoot had better leave it enabled... It could but the Linux decompressor needs to be aware and either flush the L2 (more difficult as it doesn't have all the device information) or Cortex-A8 is aware of L2$ and can flush it, isn't it? set the page table attributes to outer non-cacheable (TEX[2:0] = 0b100). If the above is right, this is not needed right? Well the L2 can be configured as inner or outer, so above alone won't work. Boot-loader disabling L2 cache ( all caches) is still right thing and that's what kernel expect. Since the early kernel code can't be patches for A8, may be delaying L2 enabled would work. Ah, I missed the fact that the L2 is an inner cache on OMAP3+A8. In this case, I would argue to just leave it enabled since as long as the MMU is off it won't be accessed. Why does U-Boot need to disable the L2 (I suspect via some auxiliary control register)? But then on A15, we are back to square one since there is no control to turn ON/OFF l2 cache. Well, it's an inner cache, it gets disabled/enabled via the SCTLR.M bit as is the L1 cache. It does not need to be disabled independently. On A15 infact there are other CP15 registers which needs to be set before MMU is enabled to have best configuration. Yes, there are, and my view is that it is up to the platform firmware to set things right. Firmware is ROM'ed in SOC's and can't be altered without a new silicon spin. Then we get into situations where Errata A will say turn OFF/ON x bit and so on. Most of these bits are secure write-only registers hence ROM code provides few API's to alter these registers in future if needed. BTW, look at the processor setup functions where we are altering these bits in kernel today (Diagnostic, RR policy etc) and they work on ARM SoC's which either don't use trustzone or kernel is in secure mode. Now same thing if needs to be done for SoC's like OMAP where these bits are under secure control, we need to use the ROM secure API's and kernel don't seems to give any flexibility to patch that. I don't understand how __only__ these devices needs to be patches outside the kernel. Firmware point is actually irrelevant for OMAP since it can't be changed once it is ROMed. Patching in boot-loaders isn't an option either since every customers prefers to use there own boot-loader and then controlling this vital bits is impossible. So I re-iterate that we need to have solution to this problem. Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 3:39 PM, Catalin Marinas catalin.mari...@arm.com wrote: On Tue, Jan 17, 2012 at 12:40:54PM +, Shilimkar, Santosh wrote: Well the L2 can be configured as inner or outer, so above alone won't work. Boot-loader disabling L2 cache ( all caches) is still right thing and that's what kernel expect. Since the early kernel code can't be patches for A8, may be delaying L2 enabled would work. Ah, I missed the fact that the L2 is an inner cache on OMAP3+A8. At least on my OMAP3 board it's not. There is L2 AUX control register L2 inner bit that's always cleared here. I know outer cacheablility bits in page table descriptors have effect here, we have tested performace before.. -- Gražvydas -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
Catalin Marinas catalin.mari...@arm.com writes: On Tue, Jan 17, 2012 at 12:27:25PM +, Aneesh V wrote: But I thought enabling L2$ again in kernel is the cleaner solution. Why? It gets enabled via SCTLR.M together with L1. Not if the L2EN bit of the auxiliary control register was cleared, and u-boot messes with this bit. -- Måns Rullgård m...@mansr.com -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 01:58:18PM +, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 2:39 PM, Catalin Marinas catalin.mari...@arm.com wrote: On Tue, Jan 17, 2012 at 12:40:54PM +, Shilimkar, Santosh wrote: On A15 infact there are other CP15 registers which needs to be set before MMU is enabled to have best configuration. Yes, there are, and my view is that it is up to the platform firmware to set things right. Firmware is ROM'ed in SOC's and can't be altered without a new silicon spin. Then we get into situations where Errata A will say turn OFF/ON x bit and so on. Most of these bits are secure write-only registers hence ROM code provides few API's to alter these registers in future if needed. BTW, does the firmware make any checks on what bits it allows to be set? Some of them may have security implications (and may not even be documented). Anyway, the first step is this API provided by the secure firmware. Since such API may need to be called before the MMU is initialised, Linux would need to have knowledge of the platform type early on. Having some platform hook (asm macro) to call early on wouldn't work with the single zImage configuration. Stack space is not an issue as we already have one for ARMv7 for D-cache flushing (XIP kernels would work but they aren't that many). BTW, look at the processor setup functions where we are altering these bits in kernel today (Diagnostic, RR policy etc) and they work on ARM SoC's which either don't use trustzone or kernel is in secure mode. What we did for many of them was to check whether the bits are already set to avoid a fault when running in NS mode. Now same thing if needs to be done for SoC's like OMAP where these bits are under secure control, we need to use the ROM secure API's and kernel don't seems to give any flexibility to patch that. I don't understand how __only__ these devices needs to be patches outside the kernel. Because there isn't a standard API to set those registers. And such generic API would not even be easily possible given that you can't tell what auxiliary (or even undocumented) registers a new CPU may have in the future (unless the API is part of the TRM). Firmware point is actually irrelevant for OMAP since it can't be changed once it is ROMed. Patching in boot-loaders isn't an option either since every customers prefers to use there own boot-loader and then controlling this vital bits is impossible. So I re-iterate that we need to have solution to this problem. I don't disagree with this, a solution is needed. In the past I was for having platform hooks in the early kernel code path but in the light of latest kernel developments (FDT, single zImage), I no longer see this as acceptable. If you can't change the boot loader, an alternative could be for the SoC vendor to provide an additional binary that is executed by the boot loader before Linux is started. Such binary could either live in some flash or it could be wrapped around the Linux kernel image before the kernel is stored into flash. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 5:27 PM, Catalin Marinas catalin.mari...@arm.com wrote: On Tue, Jan 17, 2012 at 01:58:18PM +, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 2:39 PM, Catalin Marinas catalin.mari...@arm.com wrote: On Tue, Jan 17, 2012 at 12:40:54PM +, Shilimkar, Santosh wrote: On A15 infact there are other CP15 registers which needs to be set before MMU is enabled to have best configuration. Yes, there are, and my view is that it is up to the platform firmware to set things right. Firmware is ROM'ed in SOC's and can't be altered without a new silicon spin. Then we get into situations where Errata A will say turn OFF/ON x bit and so on. Most of these bits are secure write-only registers hence ROM code provides few API's to alter these registers in future if needed. BTW, does the firmware make any checks on what bits it allows to be set? Some of them may have security implications (and may not even be documented). Anyway, the first step is this API provided by the secure firmware. Since such API may need to be called before the MMU is initialised, Linux would need to have knowledge of the platform type early on. Having some platform hook (asm macro) to call early on wouldn't work with the single zImage configuration. Stack space is not an issue as we already have one for ARMv7 for D-cache flushing (XIP kernels would work but they aren't that many). That's true. BTW, look at the processor setup functions where we are altering these bits in kernel today (Diagnostic, RR policy etc) and they work on ARM SoC's which either don't use trustzone or kernel is in secure mode. What we did for many of them was to check whether the bits are already set to avoid a fault when running in NS mode. Yes but then as I mentioned for the secure kernels, they can set from the kernel code. Now same thing if needs to be done for SoC's like OMAP where these bits are under secure control, we need to use the ROM secure API's and kernel don't seems to give any flexibility to patch that. I don't understand how __only__ these devices needs to be patches outside the kernel. Because there isn't a standard API to set those registers. And such generic API would not even be easily possible given that you can't tell what auxiliary (or even undocumented) registers a new CPU may have in the future (unless the API is part of the TRM). Firmware point is actually irrelevant for OMAP since it can't be changed once it is ROMed. Patching in boot-loaders isn't an option either since every customers prefers to use there own boot-loader and then controlling this vital bits is impossible. So I re-iterate that we need to have solution to this problem. I don't disagree with this, a solution is needed. In the past I was for having platform hooks in the early kernel code path but in the light of latest kernel developments (FDT, single zImage), I no longer see this as acceptable. Thanks for acknowledging the problem. Not allowing such hooks for single zImage doesn't seems to be right. Nobody ships kernel build for all socs, and it's usage is really limited to few distro's. But that's different topic. How about allowing platform hooks for single SOC builds. I mean having this code under !single_zImage or something like that. That way we don't impact the single zImage efforts and also allow socs to have all those critical, vital bits enabled for the SOC specific builds. If you can't change the boot loader, an alternative could be for the SoC vendor to provide an additional binary that is executed by the boot loader before Linux is started. Such binary could either live in some flash or it could be wrapped around the Linux kernel image before the kernel is stored into flash. We tried something similar with boot-loaders and failed badly to get that right for all the product trees. So if the single zImage is the blocker I would like to tackle that and have all this important information within kernel code. Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 5:27 PM, Catalin Marinas catalin.mari...@arm.com wrote: BTW, does the firmware make any checks on what bits it allows to be set? Some of them may have security implications (and may not even be documented). Anyway, the first step is this API provided by the secure firmware. Since such API may need to be called before the MMU is initialised, Linux would need to have knowledge of the platform type early on. Having some platform hook (asm macro) to call early on wouldn't work with the single zImage configuration. Stack space is not an issue as we already have one for ARMv7 for D-cache flushing (XIP kernels would work but they aren't that many). That's true. That stack is ugly and _does_ break XIP (even if there aren't that many if at all on ARMv7). Since we're already writing to RAM while setting up the initial page table and therefore we did the necessary work to locate it already, we could as well put a temporary stack for early usage right below the page table. Firmware point is actually irrelevant for OMAP since it can't be changed once it is ROMed. Patching in boot-loaders isn't an option either since every customers prefers to use there own boot-loader and then controlling this vital bits is impossible. So I re-iterate that we need to have solution to this problem. I don't disagree with this, a solution is needed. In the past I was for having platform hooks in the early kernel code path but in the light of latest kernel developments (FDT, single zImage), I no longer see this as acceptable. Thanks for acknowledging the problem. Not allowing such hooks for single zImage doesn't seems to be right. Nobody ships kernel build for all socs, and it's usage is really limited to few distro's. But that's different topic. No, I think this is right on topic. If nobody ships a kernel for multiple SOCs, this is because right now this can't be done. But distros are craving for this ability. How about allowing platform hooks for single SOC builds. I mean having this code under !single_zImage or something like that. That way we don't impact the single zImage efforts and also allow socs to have all those critical, vital bits enabled for the SOC specific builds. Absolutely not! Because if we start doing that, people will get lazy and no platform will ever work in a multi-SOC kernel. If your SOC require some fancy setup that is not shared by other platforms then please abstract that into the bootloader, or make sure it can be deferred later on. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 02:58:18PM +0100, Shilimkar, Santosh wrote: Patching in boot-loaders isn't an option either since every customers prefers to use there own boot-loader and then controlling this vital bits is impossible. So I re-iterate that we need to have solution to this problem. And I'll re-iterate that it'll create an absolute abonimation of code to do this in the kernel - not only will it be extremely fragile, it will be hell to maintain in the longer term. It will also be highly platform specific. Essentially, what you're asking is for us to find some way to call a platform specific function depending on the platform (which we haven't parsed yet), which uses a set of undefined registers, which may corrupt an undefined set of registers. And call that function from a place in the kernel which requires a set of registers to be preserved, with no stack available to save registers, and call the function via a PC relative branch (as its virtual address is useless to us at this point.) If anything goes wrong, there's no way to report that as your only option is to stop dead (if you're lucky) or maybe end up with the processor trying to execute instructions from a non-existent or uninitialized vectors page, etc. This is far too much. This is far too fragile. This is asking for undebuggable problems. This is asking for lots of people with unbootable kernels. No. Thanks. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 04:27:21PM +, Catalin Marinas wrote: Anyway, the first step is this API provided by the secure firmware. Since such API may need to be called before the MMU is initialised, Linux would need to have knowledge of the platform type early on. Having some platform hook (asm macro) to call early on wouldn't work with the single zImage configuration. Stack space is not an issue as we already have one for ARMv7 for D-cache flushing (XIP kernels would work but they aren't that many). It _is_ a problem. How much stack space would these undefined platform specific functions require? 16 words? 32 words? A page? Where does this memory come from? The .data section, or are we going to stuff it into what is supposed to be the read-only text section? -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 8:47 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Tue, Jan 17, 2012 at 04:27:21PM +, Catalin Marinas wrote: Anyway, the first step is this API provided by the secure firmware. Since such API may need to be called before the MMU is initialised, Linux would need to have knowledge of the platform type early on. Having some platform hook (asm macro) to call early on wouldn't work with the single zImage configuration. Stack space is not an issue as we already have one for ARMv7 for D-cache flushing (XIP kernels would work but they aren't that many). It _is_ a problem. How much stack space would these undefined platform specific functions require? 16 words? 32 words? A page? Where does this memory come from? The .data section, or are we going to stuff it into what is supposed to be the read-only text section? Stack requirement is limited to saving and restoring the processor registers. There is no additional need. If this is an issue, we can also use SRAM as a stack for this operation. Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 8:39 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 5:27 PM, Catalin Marinas catalin.mari...@arm.com wrote: BTW, does the firmware make any checks on what bits it allows to be set? Some of them may have security implications (and may not even be documented). Anyway, the first step is this API provided by the secure firmware. Since such API may need to be called before the MMU is initialised, Linux would need to have knowledge of the platform type early on. Having some platform hook (asm macro) to call early on wouldn't work with the single zImage configuration. Stack space is not an issue as we already have one for ARMv7 for D-cache flushing (XIP kernels would work but they aren't that many). That's true. That stack is ugly and _does_ break XIP (even if there aren't that many if at all on ARMv7). Since we're already writing to RAM while setting up the initial page table and therefore we did the necessary work to locate it already, we could as well put a temporary stack for early usage right below the page table. Firmware point is actually irrelevant for OMAP since it can't be changed once it is ROMed. Patching in boot-loaders isn't an option either since every customers prefers to use there own boot-loader and then controlling this vital bits is impossible. So I re-iterate that we need to have solution to this problem. I don't disagree with this, a solution is needed. In the past I was for having platform hooks in the early kernel code path but in the light of latest kernel developments (FDT, single zImage), I no longer see this as acceptable. Thanks for acknowledging the problem. Not allowing such hooks for single zImage doesn't seems to be right. Nobody ships kernel build for all socs, and it's usage is really limited to few distro's. But that's different topic. No, I think this is right on topic. If nobody ships a kernel for multiple SOCs, this is because right now this can't be done. But distros are craving for this ability. How about allowing platform hooks for single SOC builds. I mean having this code under !single_zImage or something like that. That way we don't impact the single zImage efforts and also allow socs to have all those critical, vital bits enabled for the SOC specific builds. Absolutely not! Because if we start doing that, people will get lazy and no platform will ever work in a multi-SOC kernel. If your SOC require some fancy setup that is not shared by other platforms then please abstract that into the bootloader, or make sure it can be deferred later on. There is nothing fancy here. It's an ARM security architecture feature which OMAP implements. Have given enough reason about boot-loaders issues. Is OMAP getting beaten up here just because it uses ARM security feature and implements it's mechanics? Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 8:39 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: How about allowing platform hooks for single SOC builds. I mean having this code under !single_zImage or something like that. That way we don't impact the single zImage efforts and also allow socs to have all those critical, vital bits enabled for the SOC specific builds. Absolutely not! Because if we start doing that, people will get lazy and no platform will ever work in a multi-SOC kernel. If your SOC require some fancy setup that is not shared by other platforms then please abstract that into the bootloader, or make sure it can be deferred later on. There is nothing fancy here. It's an ARM security architecture feature which OMAP implements. Have given enough reason about boot-loaders issues. I was not convinced by those reasons. Just push harder on the bootloader side. There is _no_ reason for the bootloader not to take care of this very platform specific issue. You can even do it into a standalone uImage that returns to u-Boot after it is done with its magic. Is OMAP getting beaten up here just because it uses ARM security feature and implements it's mechanics? I don't care if this is OMAP, UX500 or i.MX. There is a line to be drawn and this is just too bad if you are trying to cross it. Nicolas
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 09:11:37PM +0100, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 8:47 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Tue, Jan 17, 2012 at 04:27:21PM +, Catalin Marinas wrote: Anyway, the first step is this API provided by the secure firmware. Since such API may need to be called before the MMU is initialised, Linux would need to have knowledge of the platform type early on. Having some platform hook (asm macro) to call early on wouldn't work with the single zImage configuration. Stack space is not an issue as we already have one for ARMv7 for D-cache flushing (XIP kernels would work but they aren't that many). It _is_ a problem. How much stack space would these undefined platform specific functions require? 16 words? 32 words? A page? Where does this memory come from? The .data section, or are we going to stuff it into what is supposed to be the read-only text section? Stack requirement is limited to saving and restoring the processor registers. There is no additional need. If this is an issue, we can also use SRAM as a stack for this operation. How do you know that's all that would be required? Let's look at another secure monitor API implementation in the kernel. MSM has one - arch/arm/mach-msm/scm.c. This requires parameters passed through memory - a command buffer consisting of at least four words in memory, followed by a response buffer of at least three words. So where would this be placed if not on the stack? Pick some random address and hope that there's some memory there? This is what I'm saying: while we may know what the requirement is for OMAP, that's not what it is for other SoCs. Every SoC has different requirements for this, and so we have _no_ idea what so ever how much stack space would actually be required. And if we don't know that, we have a real chance of having some implementation overflow the stack, thereby overwriting something (eg, maybe your DT data, maybe your initramfs image) which would be extremely detrimental. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, 17 Jan 2012, Nicolas Pitre wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 8:39 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: How about allowing platform hooks for single SOC builds. I mean having this code under !single_zImage or something like that. That way we don't impact the single zImage efforts and also allow socs to have all those critical, vital bits enabled for the SOC specific builds. Absolutely not! Because if we start doing that, people will get lazy and no platform will ever work in a multi-SOC kernel. If your SOC require some fancy setup that is not shared by other platforms then please abstract that into the bootloader, or make sure it can be deferred later on. There is nothing fancy here. It's an ARM security architecture feature which OMAP implements. Have given enough reason about boot-loaders issues. I was not convinced by those reasons. Just push harder on the bootloader side. There is _no_ reason for the bootloader not to take care of this very platform specific issue. You can even do it into a standalone uImage that returns to u-Boot after it is done with its magic. Alternatively, you may leverage all the infrastructure that was implemented to support CPU offlining i.e. the creation of a 1:1 mapping needed to disable the MMU and so on. That should be possible for you to just have the MMU turned off, then call the secure mode to turn on L2, then restore MMU and normal operation of the kernel. This can be accomplished later during boot rather than right at the beginning. Nicolas
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 9:45 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 8:39 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: How about allowing platform hooks for single SOC builds. I mean having this code under !single_zImage or something like that. That way we don't impact the single zImage efforts and also allow socs to have all those critical, vital bits enabled for the SOC specific builds. Absolutely not! Because if we start doing that, people will get lazy and no platform will ever work in a multi-SOC kernel. If your SOC require some fancy setup that is not shared by other platforms then please abstract that into the bootloader, or make sure it can be deferred later on. There is nothing fancy here. It's an ARM security architecture feature which OMAP implements. Have given enough reason about boot-loaders issues. I was not convinced by those reasons. Just push harder on the bootloader side. There is _no_ reason for the bootloader not to take care of this very platform specific issue. You can even do it into a standalone uImage that returns to u-Boot after it is done with its magic. Is OMAP getting beaten up here just because it uses ARM security feature and implements it's mechanics? I don't care if this is OMAP, UX500 or i.MX. There is a line to be drawn and this is just too bad if you are trying to cross it. Well I was trying to get a solution for a genuine problem. Let's take an example. #ifdef CONFIG_ARM_ERRATA_742230 cmp r6, #0x22 @ only present up to r2p2 mrcle p15, 0, r10, c15, c0, 1 @ read diagnostic register orrle r10, r10, #1 4 @ set bit #4 mcrle p15, 0, r10, c15, c0, 1 @ write diagnostic register #endif The SoC's which doesn't have security restriction and the kernel is in secure mode, above errata is applied in it's right place. Before MMU is enabled. There are many bits like this which are patched in kernel. On OMAP, all these can't be set-up in kernel since kernel is in non-secure mode. To set such bit's we need to call some pre-defined security calls. Regards santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 9:45 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: On Tue, Jan 17, 2012 at 8:39 PM, Nicolas Pitre n...@fluxnic.net wrote: On Tue, 17 Jan 2012, Shilimkar, Santosh wrote: How about allowing platform hooks for single SOC builds. I mean having this code under !single_zImage or something like that. That way we don't impact the single zImage efforts and also allow socs to have all those critical, vital bits enabled for the SOC specific builds. Absolutely not! Because if we start doing that, people will get lazy and no platform will ever work in a multi-SOC kernel. If your SOC require some fancy setup that is not shared by other platforms then please abstract that into the bootloader, or make sure it can be deferred later on. There is nothing fancy here. It's an ARM security architecture feature which OMAP implements. Have given enough reason about boot-loaders issues. I was not convinced by those reasons. Just push harder on the bootloader side. There is _no_ reason for the bootloader not to take care of this very platform specific issue. You can even do it into a standalone uImage that returns to u-Boot after it is done with its magic. Is OMAP getting beaten up here just because it uses ARM security feature and implements it's mechanics? I don't care if this is OMAP, UX500 or i.MX. There is a line to be drawn and this is just too bad if you are trying to cross it. Well I was trying to get a solution for a genuine problem. Let's take an example. #ifdef CONFIG_ARM_ERRATA_742230 cmp r6, #0x22 @ only present up to r2p2 mrcle p15, 0, r10, c15, c0, 1 @ read diagnostic register orrle r10, r10, #1 4 @ set bit #4 mcrle p15, 0, r10, c15, c0, 1 @ write diagnostic register #endif The SoC's which doesn't have security restriction and the kernel is in secure mode, above errata is applied in it's right place. Before MMU is enabled. There are many bits like this which are patched in kernel. On OMAP, all these can't be set-up in kernel since kernel is in non-secure mode. To set such bit's we need to call some pre-defined security calls. Absolutely. Just don't do it in the early kernel boot code where it is not convenient to do so. Nicolas
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Tue, Jan 17, 2012 at 09:27:02PM +0100, Shilimkar, Santosh wrote: There is nothing fancy here. It's an ARM security architecture feature which OMAP implements. Have given enough reason about boot-loaders issues. There is nothing fancy about not being permitted to access registers due to security restrictions. What is fancy is that every SoC vendor out there implements their own private API to provide a method to access these registers without any form of commonality. Is OMAP getting beaten up here just because it uses ARM security feature and implements it's mechanics? To confirm what Nicolas said in reply to this - I really don't care either what kind of SoC this is. This is purely about the technical problems with what is being proposed. And, as I've said several times already, the root cause of this problem is not the SoC vendors, but a lack of standardization about how these services are provided. Imagine this: what if every vendor of a PC out there provided their ACPI data in totally different formats? -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
+ linux-arm, Russell and Catalin On Mon, Jan 16, 2012 at 11:03 AM, Joe Woodward j...@terrafix.co.uk wrote: The latest uBoot release (2011.12) disables the L2/outer cache during boot on OMAP boards. uBoot commit: armv7: disable L2 cache in cleanup_before_linux() on 6th Dec 2011 by Aneesh V adds the following to uBootSources/arch/arm/cpu/armv7/cpu.c:cleanup_before_linux(): ... v7_out_cache_disable(); The commit message implies this change was to make booting reliable on OMAP4 by disabling L2 cache before jumping to Linux. Sadly this commit also affects OMAP3-based boards... Since switching to uBoot 2011.12 I'm seeing about half the performance on my GUSMTIX Overo (OMAP3530). In the kernel sources there seem to be OMAP4 specific functions for re-enabling the L2 cache (omap4-common.c:omap_l2_cache_init()), but none for OMAP3. So my question is... do we expect that Linux should be re-enabling the L2/outer cache during boot after uBoot has disabled it on OMAP3? Yes. Kernel already does enabled L2 on A8 CPU's but OMAP3 needs a secure call and then the patching becomes that much difficult. We need some kind of generic hook in proc*_setup() function where in such secure bits can be turned ON. Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Mon, Jan 16, 2012 at 11:18:21AM +0100, Shilimkar, Santosh wrote: + linux-arm, Russell and Catalin On Mon, Jan 16, 2012 at 11:03 AM, Joe Woodward j...@terrafix.co.uk wrote: The latest uBoot release (2011.12) disables the L2/outer cache during boot on OMAP boards. uBoot commit: armv7: disable L2 cache in cleanup_before_linux() on 6th Dec 2011 by Aneesh V adds the following to uBootSources/arch/arm/cpu/armv7/cpu.c:cleanup_before_linux(): ... v7_out_cache_disable(); The commit message implies this change was to make booting reliable on OMAP4 by disabling L2 cache before jumping to Linux. Sadly this commit also affects OMAP3-based boards... Since switching to uBoot 2011.12 I'm seeing about half the performance on my GUSMTIX Overo (OMAP3530). In the kernel sources there seem to be OMAP4 specific functions for re-enabling the L2 cache (omap4-common.c:omap_l2_cache_init()), but none for OMAP3. So my question is... do we expect that Linux should be re-enabling the L2/outer cache during boot after uBoot has disabled it on OMAP3? Yes. Kernel already does enabled L2 on A8 CPU's but OMAP3 needs a secure call and then the patching becomes that much difficult. We need some kind of generic hook in proc*_setup() function where in such secure bits can be turned ON. You can't have that kind of hook: there is no stack here to call other functions, plus we aren't running with the MMU on at that point so only carefully crafted assembly could be used. Why is it soo difficult to issue the secure call? This just proves my point that the lack of consistent secure monitor API for these kinds of basic service _are_ a big problem, one which ARM Ltd should have addressed when they invented this stuff. My view continues to be that the secure monitor API is a total abortion, one which should never have been allowed to happen. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Mon, Jan 16, 2012 at 11:59 AM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Mon, Jan 16, 2012 at 11:18:21AM +0100, Shilimkar, Santosh wrote: + linux-arm, Russell and Catalin On Mon, Jan 16, 2012 at 11:03 AM, Joe Woodward j...@terrafix.co.uk wrote: The latest uBoot release (2011.12) disables the L2/outer cache during boot on OMAP boards. uBoot commit: armv7: disable L2 cache in cleanup_before_linux() on 6th Dec 2011 by Aneesh V adds the following to uBootSources/arch/arm/cpu/armv7/cpu.c:cleanup_before_linux(): ... v7_out_cache_disable(); The commit message implies this change was to make booting reliable on OMAP4 by disabling L2 cache before jumping to Linux. Sadly this commit also affects OMAP3-based boards... Since switching to uBoot 2011.12 I'm seeing about half the performance on my GUSMTIX Overo (OMAP3530). In the kernel sources there seem to be OMAP4 specific functions for re-enabling the L2 cache (omap4-common.c:omap_l2_cache_init()), but none for OMAP3. So my question is... do we expect that Linux should be re-enabling the L2/outer cache during boot after uBoot has disabled it on OMAP3? Yes. Kernel already does enabled L2 on A8 CPU's but OMAP3 needs a secure call and then the patching becomes that much difficult. We need some kind of generic hook in proc*_setup() function where in such secure bits can be turned ON. You can't have that kind of hook: there is no stack here to call other functions, plus we aren't running with the MMU on at that point so only carefully crafted assembly could be used. This code will be in assembly and that's what I have been using. Not having stack shoudn't be a blocker and can be work-around in this code. And this API has to be anyway called before MMU is enabled. Why is it soo difficult to issue the secure call? Nothing difficult. It is just that there is no place today where platform can call these secure API's Call where ? This just proves my point that the lack of consistent secure monitor API for these kinds of basic service _are_ a big problem, one which ARM Ltd should have addressed when they invented this stuff. My view continues to be that the secure monitor API is a total abortion, one which should never have been allowed to happen. Am not sure what you mean because secure API as such isn't a problem. If you mean one standard interface for all the ARM SOC's then that's something won't be easy to handled because it is tied up the security architecture which can vary across SoCs. Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Mon, Jan 16, 2012 at 01:43:03PM +0100, Shilimkar, Santosh wrote: This code will be in assembly and that's what I have been using. Not having stack shoudn't be a blocker and can be work-around in this code. And this API has to be anyway called before MMU is enabled. What about SMC on OMAP potentially corrupting most of the integer registers? What if it corrupts an integer register which we depend upon? You don't have a stack here to save those registers. We'd need at least one additional register to save the old 'lr' value. What about other secure monitors? Some implementations require you to give parameters via memory. How do you obtain that memory that early on in the kernel boot (you haven't parsed anything here.) Not only that but you're asking to make the kernel boot a _lot_ more fragile, when people start stuffing all kinds of utter shite into this hook - and then we're burdened again with the old 'my kernel won't boot and it remains silent, well your debug code is utter shite get rid of it' crap that we used to have in the early 2000s. I really don't want to go anywhere near that situation. Am not sure what you mean because secure API as such isn't a problem. If you mean one standard interface for all the ARM SOC's then that's something won't be easy to handled because it is tied up the security architecture which can vary across SoCs. The latter. This is exactly the kind of pain I forsaw with this security shite, and when I heard about it I was utterly dismayed at ARM Ltd for coming up with such a brain-dead lack of design here. You're having to struggle with the results of that by having lots of SoC specific hooks all around the place to fiddle with this that and the other. Your need to have something called from the early assembly code is just yet more of that disease caused by ARM Ltd's lack of forsight on this matter. I have no solution for you on this. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OMAP3 L2/outer cache enabled in kernel (after being disabled by uBoot)?
On Mon, Jan 16, 2012 at 2:13 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Mon, Jan 16, 2012 at 01:43:03PM +0100, Shilimkar, Santosh wrote: This code will be in assembly and that's what I have been using. Not having stack shoudn't be a blocker and can be work-around in this code. And this API has to be anyway called before MMU is enabled. What about SMC on OMAP potentially corrupting most of the integer registers? What if it corrupts an integer register which we depend upon? You don't have a stack here to save those registers. We'd need at least one additional register to save the old 'lr' value. That's right. What about other secure monitors? Some implementations require you to give parameters via memory. How do you obtain that memory that early on in the kernel boot (you haven't parsed anything here.) Not only that but you're asking to make the kernel boot a _lot_ more fragile, when people start stuffing all kinds of utter shite into this hook - and then we're burdened again with the old 'my kernel won't boot and it remains silent, well your debug code is utter shite get rid of it' crap that we used to have in the early 2000s. I really don't want to go anywhere near that situation. Fair point. It will be harder to maintain and won't be consistent. Am not sure what you mean because secure API as such isn't a problem. If you mean one standard interface for all the ARM SOC's then that's something won't be easy to handled because it is tied up the security architecture which can vary across SoCs. The latter. This is exactly the kind of pain I forsaw with this security shite, and when I heard about it I was utterly dismayed at ARM Ltd for coming up with such a brain-dead lack of design here. You're having to struggle with the results of that by having lots of SoC specific hooks all around the place to fiddle with this that and the other. Your need to have something called from the early assembly code is just yet more of that disease caused by ARM Ltd's lack of forsight on this matter. I have no solution for you on this I managed use some secure macro kind of code but as you said it will be really hard to maintain. So we are stuck with this issue then. Regards Santosh -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
