Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Tue, Oct 19, 2010 at 04:31:30PM -0700, Daniel Walker wrote: On Mon, 2010-10-18 at 09:44 +0200, Ohad Ben-Cohen wrote: OMAP4 introduces a Spinlock hardware module, which provides hardware assistance for synchronization and mutual exclusion between heterogeneous processors and those not operating under a single, shared operating system (e.g. OMAP4 has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP). The intention of this hardware module is to allow remote processors, that have no alternative mechanism to accomplish synchronization and mutual exclusion operations, to share resources (such as memory and/or any other hardware resource). This patchset adds a new misc driver for this OMAP hwspinlock module. Does this code interface with some hardware unit (other than the other processors) to accomplish this locking ? The reason I ask is because MSM has similar code, and from what I can tell the MSM version has some structures in memory but that's all. It just operates on the structures in memory. It might be worth looking over the two implementation so we aren't both remaking the wheel. Ohad's message to which you replied had: To: linux-omap@vger.kernel.org, linux-ker...@vger.kernel.org, linux-arm-ker...@lists.infradead.org Cc: Ohad Ben-Cohen o...@wizery.com, Hari Kanigeri h-kanige...@ti.com, Benoit Cousson b-cous...@ti.com, Tony Lindgren t...@atomide.com, Greg KH g...@kroah.com, Grant Likely grant.lik...@secretlab.ca, a...@linux-foundation.org, Suman Anna s-a...@ti.com Yours has: To: Ohad Ben-Cohen o...@wizery.com Cc: Hari Kanigeri h-kanige...@ti.com, Benoit Cousson b-cous...@ti.com, Tony Lindgren t...@atomide.com, Greg KH g...@kroah.com, linux-ker...@vger.kernel.org, Grant Likely grant.lik...@secretlab.ca, ma...@codeaurora.org, a...@linux-foundation.org, Suman Anna s-a...@ti.com, ma...@codeaurora.orgmattw, linux-arm-ker...@lists.infradead.org which includes an invalid address ma...@codeaurora.orgmattw. Is there a reason why you're excluding the linux-omap list from your message and subsequent discussion? -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
[resubmitting due to l-o being dropped from this discussion fork. Thanks Russell for catching this] On Wed, Oct 20, 2010 at 1:31 AM, Daniel Walker dwal...@codeaurora.org wrote: On Mon, 2010-10-18 at 09:44 +0200, Ohad Ben-Cohen wrote: OMAP4 introduces a Spinlock hardware module, which provides hardware assistance for synchronization and mutual exclusion between heterogeneous processors and those not operating under a single, shared operating system (e.g. OMAP4 has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP). The intention of this hardware module is to allow remote processors, that have no alternative mechanism to accomplish synchronization and mutual exclusion operations, to share resources (such as memory and/or any other hardware resource). This patchset adds a new misc driver for this OMAP hwspinlock module. Does this code interface with some hardware unit (other than the other processors) to accomplish this locking ? Yes, it's a special-purpose hardware peripheral. The reason I ask is because MSM has similar code, and from what I can tell the MSM version has some structures in memory but that's all. It just operates on the structures in memory. That's interesting. We did have thoughts of making this a generic framework, in the hope that it would be useful for other vendors too, but we didn't find additional users. It might be worth looking over the two implementation so we aren't both remaking the wheel. Indeed. Where is that MSM code ? Thanks, Ohad. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Wed, 2010-10-20 at 10:53 +0100, Russell King - ARM Linux wrote: To: Ohad Ben-Cohen o...@wizery.com Cc: Hari Kanigeri h-kanige...@ti.com, Benoit Cousson b-cous...@ti.com, Tony Lindgren t...@atomide.com, Greg KH g...@kroah.com, linux-ker...@vger.kernel.org, Grant Likely grant.lik...@secretlab.ca, ma...@codeaurora.org, a...@linux-foundation.org, Suman Anna s-a...@ti.com, ma...@codeaurora.orgmattw, linux-arm-ker...@lists.infradead.org which includes an invalid address ma...@codeaurora.orgmattw. Is there a reason why you're excluding the linux-omap list from your message and subsequent discussion? I was trying to add Matt to the CC, but I guess I accidentally deleted some CC's .. So it was purely an accident. Daniel -- Sent by a consultant of the Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
Indeed. Where is that MSM code ? We don't have an equivalent feature on MSM, nor any plans for one. Daniel was thinking of an internal test feature that had been deprecated a while ago. - Bryan -- Sent by an employee of the Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, 2010-10-18 at 09:44 +0200, Ohad Ben-Cohen wrote: OMAP4 introduces a Spinlock hardware module, which provides hardware assistance for synchronization and mutual exclusion between heterogeneous processors and those not operating under a single, shared operating system (e.g. OMAP4 has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP). I just have to ask... was it really easier to build silicon than to agree on a spinlock ABI? -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, Oct 18, 2010 at 02:46:55PM +0200, Peter Zijlstra wrote: On Mon, 2010-10-18 at 09:44 +0200, Ohad Ben-Cohen wrote: OMAP4 introduces a Spinlock hardware module, which provides hardware assistance for synchronization and mutual exclusion between heterogeneous processors and those not operating under a single, shared operating system (e.g. OMAP4 has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP). I just have to ask... was it really easier to build silicon than to agree on a spinlock ABI? I'm not really sure what point you're trying to make, but if you're suggesting that Linux's spinlock should be exposed to these other processors, you're completely off your rocker. Doing so would set the kernels spinlock API in stone, which is really something you don't want to do. Not only that, but it would mean that software written for the M3 and DSP would have to know about the GPL'd spinlock layout, and I suspect that would cause major licencing headaches. In any case, Linux's spinlock API (or more accurately, the ARM exclusive access instructions) relies upon hardware coherency support (a piece of hardware called an exclusive monitor) which isn't present on the M3 nor DSP processors. So there's no way to ensure that updates from the M3 and DSP are atomic wrt the A9 updates. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, 2010-10-18 at 14:35 +0100, Russell King - ARM Linux wrote: On Mon, Oct 18, 2010 at 02:46:55PM +0200, Peter Zijlstra wrote: On Mon, 2010-10-18 at 09:44 +0200, Ohad Ben-Cohen wrote: OMAP4 introduces a Spinlock hardware module, which provides hardware assistance for synchronization and mutual exclusion between heterogeneous processors and those not operating under a single, shared operating system (e.g. OMAP4 has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP). I just have to ask... was it really easier to build silicon than to agree on a spinlock ABI? I'm not really sure what point you're trying to make, but if you're suggesting that Linux's spinlock should be exposed to these other processors, you're completely off your rocker. Of course not, that would indeed be utterly silly, nor would it serve any purpose, the Linux kernel spinlocks are internal spinlocks and need not interact with anything out side of it. But for the purpose of communicating with a heterogeneous CPU/DSP it would make perfect sense to specify a spinlock ABI. Creating specific hardware just to serialise between these components seems like overkill. In any case, Linux's spinlock API (or more accurately, the ARM exclusive access instructions) relies upon hardware coherency support (a piece of hardware called an exclusive monitor) which isn't present on the M3 nor DSP processors. So there's no way to ensure that updates from the M3 and DSP are atomic wrt the A9 updates. Right, so the problem is that there simply is no way to do atomic memory access from these auxiliary processing units wrt the main CPU? Seeing as they operate on the same memory space, wouldn't it make sense to have them cache-coherent and thus provide atomicy guarantees through that? But that's water under the bridge, and your last paragraph does indeed answer my question. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, Oct 18, 2010 at 3:43 PM, Peter Zijlstra pet...@infradead.org wrote: Right, so the problem is that there simply is no way to do atomic memory access from these auxiliary processing units wrt the main CPU? Yes. There are a few relevant system-wide limitations, one of them is that simply the system interconnect does not support those fancy atomic operations. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, 2010-10-18 at 16:28 +0200, Ohad Ben-Cohen wrote: On Mon, Oct 18, 2010 at 3:43 PM, Peter Zijlstra pet...@infradead.org wrote: Right, so the problem is that there simply is no way to do atomic memory access from these auxiliary processing units wrt the main CPU? Yes. There are a few relevant system-wide limitations, one of them is that simply the system interconnect does not support those fancy atomic operations. Does it support full memory coherency though? Otherwise I can see memory based message passing becoming rather interesting. Without coherency everybody needs to be damn sure to flush the relevant bits before unlocking and such. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
Peter Zijlstra pet...@infradead.org wrote: On Mon, 2010-10-18 at 14:35 +0100, Russell King - ARM Linux wrote: In any case, Linux's spinlock API (or more accurately, the ARM exclusive access instructions) relies upon hardware coherency support (a piece of hardware called an exclusive monitor) which isn't present on the M3 nor DSP processors. So there's no way to ensure that updates from the M3 and DSP are atomic wrt the A9 updates. Right, so the problem is that there simply is no way to do atomic memory access from these auxiliary processing units wrt the main CPU? Seeing as they operate on the same memory space, wouldn't it make sense to have them cache-coherent and thus provide atomicy guarantees through that? With cache coherency you may get atomicity of writes or reads but usually not atomic modifications. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, 2010-10-18 at 16:27 +0100, Catalin Marinas wrote: Peter Zijlstra pet...@infradead.org wrote: On Mon, 2010-10-18 at 14:35 +0100, Russell King - ARM Linux wrote: In any case, Linux's spinlock API (or more accurately, the ARM exclusive access instructions) relies upon hardware coherency support (a piece of hardware called an exclusive monitor) which isn't present on the M3 nor DSP processors. So there's no way to ensure that updates from the M3 and DSP are atomic wrt the A9 updates. Right, so the problem is that there simply is no way to do atomic memory access from these auxiliary processing units wrt the main CPU? Seeing as they operate on the same memory space, wouldn't it make sense to have them cache-coherent and thus provide atomicy guarantees through that? With cache coherency you may get atomicity of writes or reads but usually not atomic modifications. Sure, but you can 'easily' extend your coherency protocols with support for things like ll/sc (or larger transactions). Have ll bring the cacheline into exclusive state and tag it, then anything that demotes the cacheline will clear the tag and make sc fail. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, Oct 18, 2010 at 5:32 PM, Peter Zijlstra pet...@infradead.org wrote: Sure, but you can 'easily' extend your coherency protocols with support In our case, there are no coherency protocols supported between the A9, M3 and the DSP. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, 2010-10-18 at 17:35 +0200, Ohad Ben-Cohen wrote: On Mon, Oct 18, 2010 at 5:32 PM, Peter Zijlstra pet...@infradead.org wrote: Sure, but you can 'easily' extend your coherency protocols with support In our case, there are no coherency protocols supported between the A9, M3 and the DSP. Sure, I got that, I was simply commenting on Catalin's statement that cache coherency doesn't (need to) bring atomic modifications. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
Peter Zijlstra pet...@infradead.org wrote: On Mon, 2010-10-18 at 16:27 +0100, Catalin Marinas wrote: Peter Zijlstra pet...@infradead.org wrote: On Mon, 2010-10-18 at 14:35 +0100, Russell King - ARM Linux wrote: In any case, Linux's spinlock API (or more accurately, the ARM exclusive access instructions) relies upon hardware coherency support (a piece of hardware called an exclusive monitor) which isn't present on the M3 nor DSP processors. So there's no way to ensure that updates from the M3 and DSP are atomic wrt the A9 updates. Right, so the problem is that there simply is no way to do atomic memory access from these auxiliary processing units wrt the main CPU? Seeing as they operate on the same memory space, wouldn't it make sense to have them cache-coherent and thus provide atomicy guarantees through that? With cache coherency you may get atomicity of writes or reads but usually not atomic modifications. Sure, but you can 'easily' extend your coherency protocols with support for things like ll/sc (or larger transactions). Have ll bring the cacheline into exclusive state and tag it, then anything that demotes the cacheline will clear the tag and make sc fail. For the ll/sc operations on ARM (exclusive load/store) there is a per-CPU local exclusive monitor and a (virtual) global one. The global one may either be a separate piece of hardware or emulated via cache lines as you said. But if you need synchronisation with a CPU (or DSP) like Cortex-M3 which doesn't have any built-in caches, you can only get atomic operations on the main processor (A9) but not on the M3 (as you can't have a cache line in exclusive state on the M3). The M3 may have a local exclusive monitor (like the main CPU) but it isn't cleared by memory accesses from the main CPU. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] Add OMAP hardware spinlock misc driver
On Mon, 2010-10-18 at 16:51 +0100, Catalin Marinas wrote: Peter Zijlstra pet...@infradead.org wrote: On Mon, 2010-10-18 at 16:27 +0100, Catalin Marinas wrote: Peter Zijlstra pet...@infradead.org wrote: On Mon, 2010-10-18 at 14:35 +0100, Russell King - ARM Linux wrote: In any case, Linux's spinlock API (or more accurately, the ARM exclusive access instructions) relies upon hardware coherency support (a piece of hardware called an exclusive monitor) which isn't present on the M3 nor DSP processors. So there's no way to ensure that updates from the M3 and DSP are atomic wrt the A9 updates. Right, so the problem is that there simply is no way to do atomic memory access from these auxiliary processing units wrt the main CPU? Seeing as they operate on the same memory space, wouldn't it make sense to have them cache-coherent and thus provide atomicy guarantees through that? With cache coherency you may get atomicity of writes or reads but usually not atomic modifications. Right, so you forgot the qualifying part of your stmt: on ARM. Sure, but you can 'easily' extend your coherency protocols with support for things like ll/sc (or larger transactions). Have ll bring the cacheline into exclusive state and tag it, then anything that demotes the cacheline will clear the tag and make sc fail. For the ll/sc operations on ARM (exclusive load/store) there is a per-CPU local exclusive monitor and a (virtual) global one. The global one may either be a separate piece of hardware or emulated via cache lines as you said. But if you need synchronisation with a CPU (or DSP) like Cortex-M3 which doesn't have any built-in caches, you can only get atomic operations on the main processor (A9) but not on the M3 (as you can't have a cache line in exclusive state on the M3). Right, and I take it that modifying the M3 to participate in the full coherency/exclusive monitor thing would have been more work. The M3 may have a local exclusive monitor (like the main CPU) but it isn't cleared by memory accesses from the main CPU. Sounds like asking for trouble if you ask me ;-) -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
