Re: Memory synchronization vs. interrupt handlers
On 26 August 2013 16:49, Alan Stern wrote: > Here's a question that doesn't seem to be answered in > Documentation/memory-barriers.txt. Are memory accesses within an > interrupt handler synchronized with respect to interrupts? > > In more detail, suppose we have an interrupt handler that uses a memory > variable A. The device attached to the IRQ line sends two interrupt > requests, and we get: > > CPU 0 CPU 1 > - - > Receive IRQ > Call the interrupt handler > Write A > Finish IRQ processing > > Receive IRQ > Call the interrupt handler > Read A > Finish IRQ processing > > Is CPU 0's write to A guaranteed to be visible on CPU 1? Given that > interrupts on an IRQ line are serialized, and that IRQ processing must > involve some amount of memory barriers, I would expect the answer to be > Yes. On arm (or arm64), this is not guaranteed. Finishing the IRQ processing usually involves a device write but there is no ordering required with other write accesses. It could easily be fixed in the IRQ controller code (drivers/irqchip/irq-gic.c for newer ARM processors). We have a barrier for SMP cross-calls for the same ordering reason, so I guess we need one for EOI as well. In practice I would think it's nearly impossible to hit this issue, given the time to save the state when taking the interrupt plus some spinlocks, the write from CPU0 would become visible. Also, if the data is accessed by the same driver with the same IRQ, most likely the interrupt goes to the same CPU (unless you had some rebalancing, but this being rarer it makes it less likely). If the data is being accessed between two separate IRQ handlers, a spinlock must be used anyway. -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On 26 August 2013 16:49, Alan Stern st...@rowland.harvard.edu wrote: Here's a question that doesn't seem to be answered in Documentation/memory-barriers.txt. Are memory accesses within an interrupt handler synchronized with respect to interrupts? In more detail, suppose we have an interrupt handler that uses a memory variable A. The device attached to the IRQ line sends two interrupt requests, and we get: CPU 0 CPU 1 - - Receive IRQ Call the interrupt handler Write A Finish IRQ processing Receive IRQ Call the interrupt handler Read A Finish IRQ processing Is CPU 0's write to A guaranteed to be visible on CPU 1? Given that interrupts on an IRQ line are serialized, and that IRQ processing must involve some amount of memory barriers, I would expect the answer to be Yes. On arm (or arm64), this is not guaranteed. Finishing the IRQ processing usually involves a device write but there is no ordering required with other write accesses. It could easily be fixed in the IRQ controller code (drivers/irqchip/irq-gic.c for newer ARM processors). We have a barrier for SMP cross-calls for the same ordering reason, so I guess we need one for EOI as well. In practice I would think it's nearly impossible to hit this issue, given the time to save the state when taking the interrupt plus some spinlocks, the write from CPU0 would become visible. Also, if the data is accessed by the same driver with the same IRQ, most likely the interrupt goes to the same CPU (unless you had some rebalancing, but this being rarer it makes it less likely). If the data is being accessed between two separate IRQ handlers, a spinlock must be used anyway. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On 08/29/2013 04:51 PM, Paul E. McKenney wrote: > On Wed, Aug 28, 2013 at 01:28:08PM -0700, H. Peter Anvin wrote: >> On 08/28/2013 12:16 PM, Alan Stern wrote: >>> Russell, Peter, and Ingo: >>> >>> Can you folks enlighten us regarding this issue for some common >>> architectures? >> >> On x86, IRET is a serializing instruction; it guarantees hard >> serialization of absolutely everything. > > So a second interrupt from this same device could not appear to happen > before the IRET, no matter what device and/or I/O bus? Or is IRET > defined to synchronize all the way out to the whatever device is > generating the next interrupt? The second interrupt from this same device can occur as soon as the EOI cycle is done, which happens before the IRET. The EOI cycle is an I/O operation and since integer operations to memory are strongly ordered that implies all other effects are globally visible. In addition, there is usually synchronization that happens due to reading an interrupt status register or something else. >> I would expect architectures that have weak memory ordering to put >> appropriate barriers in the IRQ entry/exit code. > > Adding a few on CC. Also restating the question as I understand it: > > Suppose that a given device generates an interrupt on CPU 0, > but that before CPU 0's interrupt handler completes, this device > wants to generate a second interrupt on CPU 1. This can happen > as soon as CPU 0's handler does an EOI or equivalent. > > Can CPU 1's interrupt handler assume that all the in-memory effects > of CPU 0's interrupt handler will be visible, even if neither > interrupt handler uses locking or memory barriers? > On x86 it certainly can. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On Thu, 2013-08-29 at 16:51 -0700, Paul E. McKenney wrote: > On Wed, Aug 28, 2013 at 01:28:08PM -0700, H. Peter Anvin wrote: > > On 08/28/2013 12:16 PM, Alan Stern wrote: > > > Russell, Peter, and Ingo: > > > > > > Can you folks enlighten us regarding this issue for some common > > > architectures? > > > > On x86, IRET is a serializing instruction; it guarantees hard > > serialization of absolutely everything. > > So a second interrupt from this same device could not appear to happen > before the IRET, no matter what device and/or I/O bus? Or is IRET > defined to synchronize all the way out to the whatever device is > generating the next interrupt? > > > I would expect architectures that have weak memory ordering to put > > appropriate barriers in the IRQ entry/exit code. Not sure why you would expect that, there is no reason to do such a thing. > Adding a few on CC. Also restating the question as I understand it: > > Suppose that a given device generates an interrupt on CPU 0, > but that before CPU 0's interrupt handler completes, this device > wants to generate a second interrupt on CPU 1. This can happen > as soon as CPU 0's handler does an EOI or equivalent. By "interrupt handler" are you talking about the general handling of all interrupts in the kernel or the device specific interrupt handler ? Typically the EOI is done after the later has run. > Can CPU 1's interrupt handler assume that all the in-memory effects > of CPU 0's interrupt handler will be visible, even if neither > interrupt handler uses locking or memory barriers? We don't formally provide such a guarantee no. There is a spin_lock on the way back from the device handler and before the EOI but not an unlock so we don't have a full ordering here (see handle_irq_event). At least on powerpc, the EOI will generally be an MMIO which will however synchronize everything because we have a sync before the store in our MMIO accessors, but that might not be true when running under the pHyp hypervisor, as we do a hypercall there and I don't think that includes a sync instruction inside the hypervisor. So I'd say that as-is, no, we don't provide that guarantee. Cheers, Ben. > > Thanx, Paul > > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On Wed, Aug 28, 2013 at 01:28:08PM -0700, H. Peter Anvin wrote: > On 08/28/2013 12:16 PM, Alan Stern wrote: > > Russell, Peter, and Ingo: > > > > Can you folks enlighten us regarding this issue for some common > > architectures? > > On x86, IRET is a serializing instruction; it guarantees hard > serialization of absolutely everything. So a second interrupt from this same device could not appear to happen before the IRET, no matter what device and/or I/O bus? Or is IRET defined to synchronize all the way out to the whatever device is generating the next interrupt? > I would expect architectures that have weak memory ordering to put > appropriate barriers in the IRQ entry/exit code. Adding a few on CC. Also restating the question as I understand it: Suppose that a given device generates an interrupt on CPU 0, but that before CPU 0's interrupt handler completes, this device wants to generate a second interrupt on CPU 1. This can happen as soon as CPU 0's handler does an EOI or equivalent. Can CPU 1's interrupt handler assume that all the in-memory effects of CPU 0's interrupt handler will be visible, even if neither interrupt handler uses locking or memory barriers? Thanx, Paul -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On Wed, 28 Aug 2013, H. Peter Anvin wrote: > On 08/28/2013 12:16 PM, Alan Stern wrote: > > Russell, Peter, and Ingo: > > > > Can you folks enlighten us regarding this issue for some common > > architectures? > > > > On x86, IRET is a serializing instruction; it guarantees hard > serialization of absolutely everything. That answers half of the question. What about the other half? Does the CPU automatically serialize everything when it takes an interrupt? > I would expect architectures that have weak memory ordering to put > appropriate barriers in the IRQ entry/exit code. Then would it be acceptable to mention this in the memory-barriers.txt file? Alan Stern -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On Wed, 28 Aug 2013, H. Peter Anvin wrote: On 08/28/2013 12:16 PM, Alan Stern wrote: Russell, Peter, and Ingo: Can you folks enlighten us regarding this issue for some common architectures? On x86, IRET is a serializing instruction; it guarantees hard serialization of absolutely everything. That answers half of the question. What about the other half? Does the CPU automatically serialize everything when it takes an interrupt? I would expect architectures that have weak memory ordering to put appropriate barriers in the IRQ entry/exit code. Then would it be acceptable to mention this in the memory-barriers.txt file? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On Wed, Aug 28, 2013 at 01:28:08PM -0700, H. Peter Anvin wrote: On 08/28/2013 12:16 PM, Alan Stern wrote: Russell, Peter, and Ingo: Can you folks enlighten us regarding this issue for some common architectures? On x86, IRET is a serializing instruction; it guarantees hard serialization of absolutely everything. So a second interrupt from this same device could not appear to happen before the IRET, no matter what device and/or I/O bus? Or is IRET defined to synchronize all the way out to the whatever device is generating the next interrupt? I would expect architectures that have weak memory ordering to put appropriate barriers in the IRQ entry/exit code. Adding a few on CC. Also restating the question as I understand it: Suppose that a given device generates an interrupt on CPU 0, but that before CPU 0's interrupt handler completes, this device wants to generate a second interrupt on CPU 1. This can happen as soon as CPU 0's handler does an EOI or equivalent. Can CPU 1's interrupt handler assume that all the in-memory effects of CPU 0's interrupt handler will be visible, even if neither interrupt handler uses locking or memory barriers? Thanx, Paul -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On Thu, 2013-08-29 at 16:51 -0700, Paul E. McKenney wrote: On Wed, Aug 28, 2013 at 01:28:08PM -0700, H. Peter Anvin wrote: On 08/28/2013 12:16 PM, Alan Stern wrote: Russell, Peter, and Ingo: Can you folks enlighten us regarding this issue for some common architectures? On x86, IRET is a serializing instruction; it guarantees hard serialization of absolutely everything. So a second interrupt from this same device could not appear to happen before the IRET, no matter what device and/or I/O bus? Or is IRET defined to synchronize all the way out to the whatever device is generating the next interrupt? I would expect architectures that have weak memory ordering to put appropriate barriers in the IRQ entry/exit code. Not sure why you would expect that, there is no reason to do such a thing. Adding a few on CC. Also restating the question as I understand it: Suppose that a given device generates an interrupt on CPU 0, but that before CPU 0's interrupt handler completes, this device wants to generate a second interrupt on CPU 1. This can happen as soon as CPU 0's handler does an EOI or equivalent. By interrupt handler are you talking about the general handling of all interrupts in the kernel or the device specific interrupt handler ? Typically the EOI is done after the later has run. Can CPU 1's interrupt handler assume that all the in-memory effects of CPU 0's interrupt handler will be visible, even if neither interrupt handler uses locking or memory barriers? We don't formally provide such a guarantee no. There is a spin_lock on the way back from the device handler and before the EOI but not an unlock so we don't have a full ordering here (see handle_irq_event). At least on powerpc, the EOI will generally be an MMIO which will however synchronize everything because we have a sync before the store in our MMIO accessors, but that might not be true when running under the pHyp hypervisor, as we do a hypercall there and I don't think that includes a sync instruction inside the hypervisor. So I'd say that as-is, no, we don't provide that guarantee. Cheers, Ben. Thanx, Paul -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On 08/29/2013 04:51 PM, Paul E. McKenney wrote: On Wed, Aug 28, 2013 at 01:28:08PM -0700, H. Peter Anvin wrote: On 08/28/2013 12:16 PM, Alan Stern wrote: Russell, Peter, and Ingo: Can you folks enlighten us regarding this issue for some common architectures? On x86, IRET is a serializing instruction; it guarantees hard serialization of absolutely everything. So a second interrupt from this same device could not appear to happen before the IRET, no matter what device and/or I/O bus? Or is IRET defined to synchronize all the way out to the whatever device is generating the next interrupt? The second interrupt from this same device can occur as soon as the EOI cycle is done, which happens before the IRET. The EOI cycle is an I/O operation and since integer operations to memory are strongly ordered that implies all other effects are globally visible. In addition, there is usually synchronization that happens due to reading an interrupt status register or something else. I would expect architectures that have weak memory ordering to put appropriate barriers in the IRQ entry/exit code. Adding a few on CC. Also restating the question as I understand it: Suppose that a given device generates an interrupt on CPU 0, but that before CPU 0's interrupt handler completes, this device wants to generate a second interrupt on CPU 1. This can happen as soon as CPU 0's handler does an EOI or equivalent. Can CPU 1's interrupt handler assume that all the in-memory effects of CPU 0's interrupt handler will be visible, even if neither interrupt handler uses locking or memory barriers? On x86 it certainly can. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On 08/28/2013 12:16 PM, Alan Stern wrote: > Russell, Peter, and Ingo: > > Can you folks enlighten us regarding this issue for some common > architectures? > On x86, IRET is a serializing instruction; it guarantees hard serialization of absolutely everything. I would expect architectures that have weak memory ordering to put appropriate barriers in the IRQ entry/exit code. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
Russell, Peter, and Ingo: Can you folks enlighten us regarding this issue for some common architectures? On Tue, 27 Aug 2013, Paul E. McKenney wrote: > On Mon, Aug 26, 2013 at 11:49:15AM -0400, Alan Stern wrote: > > David and Paul: > > > > Here's a question that doesn't seem to be answered in > > Documentation/memory-barriers.txt. Are memory accesses within an > > interrupt handler synchronized with respect to interrupts? > > > > In more detail, suppose we have an interrupt handler that uses a memory > > variable A. The device attached to the IRQ line sends two interrupt > > requests, and we get: > > > > CPU 0 CPU 1 > > - - > > Receive IRQ > > Call the interrupt handler > > Write A > > Finish IRQ processing > > > > Receive IRQ > > Call the interrupt handler > > Read A > > Finish IRQ processing > > > > Is CPU 0's write to A guaranteed to be visible on CPU 1? Given that > > interrupts on an IRQ line are serialized, and that IRQ processing must > > involve some amount of memory barriers, I would expect the answer to be > > Yes. > > I have no idea. I would hope that it did, but a lot depends on how or > whether the end-of-interrupt processing is handled by the I/O hardware. Isn't this the sort of thing the kernel should guarantee? The low-level code that fields interrupts ought to do the equivalent of an "acquire" barrier at the start and a "release" barrier at the end, so that operations by the higher-level handlers don't "leak out". For example, if the low-level code would lock and unlock a spinlock at the start and end of the interrupt, that would be more than sufficient (provided it always used the same spinlock for the same IRQ line). Of course, drivers needing this behavior can always use their own spinlocks. Still, it seems like this ought to fall under the purview of the core kernel. > I believe that we need to ask the architecture maintainers. And maybe > also someone who knows about the devices in question. The devices aren't relevant here because the variable A resides in main memory, not on the device. What matter is, as you say, the architectural code. Alan Stern -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
Russell, Peter, and Ingo: Can you folks enlighten us regarding this issue for some common architectures? On Tue, 27 Aug 2013, Paul E. McKenney wrote: On Mon, Aug 26, 2013 at 11:49:15AM -0400, Alan Stern wrote: David and Paul: Here's a question that doesn't seem to be answered in Documentation/memory-barriers.txt. Are memory accesses within an interrupt handler synchronized with respect to interrupts? In more detail, suppose we have an interrupt handler that uses a memory variable A. The device attached to the IRQ line sends two interrupt requests, and we get: CPU 0 CPU 1 - - Receive IRQ Call the interrupt handler Write A Finish IRQ processing Receive IRQ Call the interrupt handler Read A Finish IRQ processing Is CPU 0's write to A guaranteed to be visible on CPU 1? Given that interrupts on an IRQ line are serialized, and that IRQ processing must involve some amount of memory barriers, I would expect the answer to be Yes. I have no idea. I would hope that it did, but a lot depends on how or whether the end-of-interrupt processing is handled by the I/O hardware. Isn't this the sort of thing the kernel should guarantee? The low-level code that fields interrupts ought to do the equivalent of an acquire barrier at the start and a release barrier at the end, so that operations by the higher-level handlers don't leak out. For example, if the low-level code would lock and unlock a spinlock at the start and end of the interrupt, that would be more than sufficient (provided it always used the same spinlock for the same IRQ line). Of course, drivers needing this behavior can always use their own spinlocks. Still, it seems like this ought to fall under the purview of the core kernel. I believe that we need to ask the architecture maintainers. And maybe also someone who knows about the devices in question. The devices aren't relevant here because the variable A resides in main memory, not on the device. What matter is, as you say, the architectural code. Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On 08/28/2013 12:16 PM, Alan Stern wrote: Russell, Peter, and Ingo: Can you folks enlighten us regarding this issue for some common architectures? On x86, IRET is a serializing instruction; it guarantees hard serialization of absolutely everything. I would expect architectures that have weak memory ordering to put appropriate barriers in the IRQ entry/exit code. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On Mon, Aug 26, 2013 at 11:49:15AM -0400, Alan Stern wrote: > David and Paul: > > Here's a question that doesn't seem to be answered in > Documentation/memory-barriers.txt. Are memory accesses within an > interrupt handler synchronized with respect to interrupts? > > In more detail, suppose we have an interrupt handler that uses a memory > variable A. The device attached to the IRQ line sends two interrupt > requests, and we get: > > CPU 0 CPU 1 > - - > Receive IRQ > Call the interrupt handler > Write A > Finish IRQ processing > > Receive IRQ > Call the interrupt handler > Read A > Finish IRQ processing > > Is CPU 0's write to A guaranteed to be visible on CPU 1? Given that > interrupts on an IRQ line are serialized, and that IRQ processing must > involve some amount of memory barriers, I would expect the answer to be > Yes. I have no idea. I would hope that it did, but a lot depends on how or whether the end-of-interrupt processing is handled by the I/O hardware. > Does the answer change if the IRQ line is shared? I wouldn't expect > it to be. > > Now, if the handler were bound to multiple IRQ (or MSI) lines, then > there'd be no reason to expect this to work. However, even in this > case, it seems that as long as we restrict our attention to handler > invocations in response to interrupt requests from one particular IRQ > line, the answer should be Yes. (For example, if device X on IRQ I and > device Y on IRQ J both used the same handler, a write to A in response > to an interrupt from device X should be visible the next time X sends > an interrupt.) > > Do you know the answers? I believe that we need to ask the architecture maintainers. And maybe also someone who knows about the devices in question. Thanx, Paul -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Memory synchronization vs. interrupt handlers
On Mon, Aug 26, 2013 at 11:49:15AM -0400, Alan Stern wrote: David and Paul: Here's a question that doesn't seem to be answered in Documentation/memory-barriers.txt. Are memory accesses within an interrupt handler synchronized with respect to interrupts? In more detail, suppose we have an interrupt handler that uses a memory variable A. The device attached to the IRQ line sends two interrupt requests, and we get: CPU 0 CPU 1 - - Receive IRQ Call the interrupt handler Write A Finish IRQ processing Receive IRQ Call the interrupt handler Read A Finish IRQ processing Is CPU 0's write to A guaranteed to be visible on CPU 1? Given that interrupts on an IRQ line are serialized, and that IRQ processing must involve some amount of memory barriers, I would expect the answer to be Yes. I have no idea. I would hope that it did, but a lot depends on how or whether the end-of-interrupt processing is handled by the I/O hardware. Does the answer change if the IRQ line is shared? I wouldn't expect it to be. Now, if the handler were bound to multiple IRQ (or MSI) lines, then there'd be no reason to expect this to work. However, even in this case, it seems that as long as we restrict our attention to handler invocations in response to interrupt requests from one particular IRQ line, the answer should be Yes. (For example, if device X on IRQ I and device Y on IRQ J both used the same handler, a write to A in response to an interrupt from device X should be visible the next time X sends an interrupt.) Do you know the answers? I believe that we need to ask the architecture maintainers. And maybe also someone who knows about the devices in question. Thanx, Paul -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Memory synchronization vs. interrupt handlers
David and Paul: Here's a question that doesn't seem to be answered in Documentation/memory-barriers.txt. Are memory accesses within an interrupt handler synchronized with respect to interrupts? In more detail, suppose we have an interrupt handler that uses a memory variable A. The device attached to the IRQ line sends two interrupt requests, and we get: CPU 0 CPU 1 - - Receive IRQ Call the interrupt handler Write A Finish IRQ processing Receive IRQ Call the interrupt handler Read A Finish IRQ processing Is CPU 0's write to A guaranteed to be visible on CPU 1? Given that interrupts on an IRQ line are serialized, and that IRQ processing must involve some amount of memory barriers, I would expect the answer to be Yes. Does the answer change if the IRQ line is shared? I wouldn't expect it to be. Now, if the handler were bound to multiple IRQ (or MSI) lines, then there'd be no reason to expect this to work. However, even in this case, it seems that as long as we restrict our attention to handler invocations in response to interrupt requests from one particular IRQ line, the answer should be Yes. (For example, if device X on IRQ I and device Y on IRQ J both used the same handler, a write to A in response to an interrupt from device X should be visible the next time X sends an interrupt.) Do you know the answers? Alan Stern -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Memory synchronization vs. interrupt handlers
David and Paul: Here's a question that doesn't seem to be answered in Documentation/memory-barriers.txt. Are memory accesses within an interrupt handler synchronized with respect to interrupts? In more detail, suppose we have an interrupt handler that uses a memory variable A. The device attached to the IRQ line sends two interrupt requests, and we get: CPU 0 CPU 1 - - Receive IRQ Call the interrupt handler Write A Finish IRQ processing Receive IRQ Call the interrupt handler Read A Finish IRQ processing Is CPU 0's write to A guaranteed to be visible on CPU 1? Given that interrupts on an IRQ line are serialized, and that IRQ processing must involve some amount of memory barriers, I would expect the answer to be Yes. Does the answer change if the IRQ line is shared? I wouldn't expect it to be. Now, if the handler were bound to multiple IRQ (or MSI) lines, then there'd be no reason to expect this to work. However, even in this case, it seems that as long as we restrict our attention to handler invocations in response to interrupt requests from one particular IRQ line, the answer should be Yes. (For example, if device X on IRQ I and device Y on IRQ J both used the same handler, a write to A in response to an interrupt from device X should be visible the next time X sends an interrupt.) Do you know the answers? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
