Re: RFC on writel and writel_relaxed

On 3/29/2018 9:40 PM, Benjamin Herrenschmidt wrote: > On Thu, 2018-03-29 at 09:56 -0400, Sinan Kaya wrote: >> On 3/28/2018 11:55 AM, David Miller wrote: >>> From: Benjamin Herrenschmidt >>> Date: Thu, 29 Mar 2018 02:13:16 +1100 >>> Let's fix all archs, it's way

Re: RFC on writel and writel_relaxed

On Thu, 2018-03-29 at 09:56 -0400, Sinan Kaya wrote: > On 3/28/2018 11:55 AM, David Miller wrote: > > From: Benjamin Herrenschmidt > > Date: Thu, 29 Mar 2018 02:13:16 +1100 > > > > > Let's fix all archs, it's way easier than fixing all drivers. Half of > > > the archs

Re: RFC on writel and writel_relaxed

On 3/29/2018 12:29 PM, Arnd Bergmann wrote: > On Thu, Mar 29, 2018 at 3:56 PM, Sinan Kaya wrote: >> On 3/28/2018 11:55 AM, David Miller wrote: >>> From: Benjamin Herrenschmidt >>> Date: Thu, 29 Mar 2018 02:13:16 +1100 >>> Let's fix all archs,

Re: RFC on writel and writel_relaxed

On Thu, Mar 29, 2018 at 02:58:34PM +, David Laight wrote: > From: Jason Gunthorpe > > Sent: 29 March 2018 15:45 > ... > > > > When talking about ordering between the devices, the relevant question > > > > is what happens if the writel(DEVICE_BAR) triggers DEVICE_BAR to DMA > > > > from the

Re: RFC on writel and writel_relaxed

On Thu, Mar 29, 2018 at 3:56 PM, Sinan Kaya wrote: > On 3/28/2018 11:55 AM, David Miller wrote: >> From: Benjamin Herrenschmidt >> Date: Thu, 29 Mar 2018 02:13:16 +1100 >> >>> Let's fix all archs, it's way easier than fixing all drivers. Half of

RE: RFC on writel and writel_relaxed

From: Jason Gunthorpe > Sent: 29 March 2018 15:45 ... > > > When talking about ordering between the devices, the relevant question > > > is what happens if the writel(DEVICE_BAR) triggers DEVICE_BAR to DMA > > > from the DEVICE_FOO. 'ordered' means that in this case > > > writel(DEVICE_FOO) must

Re: RFC on writel and writel_relaxed

On Thu, Mar 29, 2018 at 10:19:41AM +0100, Will Deacon wrote: > On Wed, Mar 28, 2018 at 10:57:32AM -0600, Jason Gunthorpe wrote: > > On Wed, Mar 28, 2018 at 11:13:45AM +0100, Will Deacon wrote: > > > On Wed, Mar 28, 2018 at 09:01:27PM +1100, Benjamin Herrenschmidt wrote: > > > > On Wed, 2018-03-28

Re: RFC on writel and writel_relaxed

From: Sinan Kaya Date: Thu, 29 Mar 2018 09:56:01 -0400 > sparc question sent Sparc never lets physical memory accesses pass MMIO, and vice versa. They are always strongly ordered amongst eachother. Therefore no explicit barrier instructions are necessary.

Re: RFC on writel and writel_relaxed

On 3/28/2018 11:55 AM, David Miller wrote: > From: Benjamin Herrenschmidt > Date: Thu, 29 Mar 2018 02:13:16 +1100 > >> Let's fix all archs, it's way easier than fixing all drivers. Half of >> the archs are unused or dead anyway. > > Agreed. > I pinged most of the

Re: RFC on writel and writel_relaxed

On Thu, Mar 29, 2018 at 08:31:32AM +1100, Benjamin Herrenschmidt wrote: > On Thu, 2018-03-29 at 02:23 +1000, Nicholas Piggin wrote: > > This is a variation on the mandatory write barrier that causes writes to > > weakly > > ordered I/O regions to be partially ordered. Its effects may go

Re: RFC on writel and writel_relaxed

On Wed, Mar 28, 2018 at 10:57:32AM -0600, Jason Gunthorpe wrote: > On Wed, Mar 28, 2018 at 11:13:45AM +0100, Will Deacon wrote: > > On Wed, Mar 28, 2018 at 09:01:27PM +1100, Benjamin Herrenschmidt wrote: > > > On Wed, 2018-03-28 at 11:55 +0200, Arnd Bergmann wrote: > > > > > powerpc and ARM can't

Re: RFC on writel and writel_relaxed

On Thu, 29 Mar 2018 08:31:32 +1100 Benjamin Herrenschmidt wrote: > On Thu, 2018-03-29 at 02:23 +1000, Nicholas Piggin wrote: > > On Wed, 28 Mar 2018 11:55:09 -0400 (EDT) > > David Miller wrote: > > > > > From: Benjamin Herrenschmidt

Re: RFC on writel and writel_relaxed

On Thu, 2018-03-29 at 02:23 +1000, Nicholas Piggin wrote: > On Wed, 28 Mar 2018 11:55:09 -0400 (EDT) > David Miller wrote: > > > From: Benjamin Herrenschmidt > > Date: Thu, 29 Mar 2018 02:13:16 +1100 > > > > > Let's fix all archs, it's way easier

Re: RFC on writel and writel_relaxed

On Wed, Mar 28, 2018 at 11:13:45AM +0100, Will Deacon wrote: > On Wed, Mar 28, 2018 at 09:01:27PM +1100, Benjamin Herrenschmidt wrote: > > On Wed, 2018-03-28 at 11:55 +0200, Arnd Bergmann wrote: > > > > powerpc and ARM can't quite make them synchronous I think, but at least > > > > they should

Re: RFC on writel and writel_relaxed

On Wed, 28 Mar 2018 11:55:09 -0400 (EDT) David Miller wrote: > From: Benjamin Herrenschmidt > Date: Thu, 29 Mar 2018 02:13:16 +1100 > > > Let's fix all archs, it's way easier than fixing all drivers. Half of > > the archs are unused or dead

RE: RFC on writel and writel_relaxed

From: Benjamin Herrenschmidt > Sent: 28 March 2018 16:13 ... > > I've always wondered exactly what the twi;isync were for - always seemed > > very heavy handed for most mmio reads. > > Particularly if you are doing mmio reads from a data fifo. > > If you do that you should use the "s" version of

Re: RFC on writel and writel_relaxed

From: Benjamin Herrenschmidt Date: Thu, 29 Mar 2018 02:13:16 +1100 > Let's fix all archs, it's way easier than fixing all drivers. Half of > the archs are unused or dead anyway. Agreed.

Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 07:41 -0400, ok...@codeaurora.org wrote: > Yes, we want to get there indeed. It is because of some arch not > implementing writel properly. Maintainers want to play safe. > > That is why I asked if IA64 and other well known archs follow the > strongly ordered rule at this

Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 11:30 +, David Laight wrote: > From: Benjamin Herrenschmidt > > Sent: 28 March 2018 10:56 > > ... > > For example, let's say I have a device with a reset bit and the spec > > says the reset bit needs to be set for at least 10us. > > > > This is wrong: > > > >

Re: RFC on writel and writel_relaxed

On 2018-03-28 02:14, Linus Torvalds wrote: On Tue, Mar 27, 2018 at 5:24 PM, Sinan Kaya wrote: Basically changing it to dma_buffer->foo = 1;/* WB */ wmb() writel_relaxed(KICK, DMA_KICK_REGISTER);/* UC */ mmiowb() Why? Why not just remove

RE: RFC on writel and writel_relaxed

From: Benjamin Herrenschmidt > Sent: 28 March 2018 10:56 ... > For example, let's say I have a device with a reset bit and the spec > says the reset bit needs to be set for at least 10us. > > This is wrong: > > writel(1, RESET_REG); > usleep(10); > writel(0, RESET_REG); > >

Re: Aw: Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 12:13 +0200, Lino Sanfilippo wrote: > Hi, > > > > > > Yeah so that other trick I'm talking about is also used for timing > > accuracy. > > > > For example, let's say I have a device with a reset bit and the spec > > says the reset bit needs to be set for at least 10us. >

Aw: Re: RFC on writel and writel_relaxed

Hi, > > Yeah so that other trick I'm talking about is also used for timing > accuracy. > > For example, let's say I have a device with a reset bit and the spec > says the reset bit needs to be set for at least 10us. > > This is wrong: > > writel(1, RESET_REG); > usleep(10); >

Re: RFC on writel and writel_relaxed

On Wed, Mar 28, 2018 at 09:01:27PM +1100, Benjamin Herrenschmidt wrote: > On Wed, 2018-03-28 at 11:55 +0200, Arnd Bergmann wrote: > > > powerpc and ARM can't quite make them synchronous I think, but at least > > > they should have the same semantics as writel. > > > > One thing that ARM does IIRC

Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 11:55 +0200, Arnd Bergmann wrote: > > powerpc and ARM can't quite make them synchronous I think, but at least > > they should have the same semantics as writel. > > One thing that ARM does IIRC is that it only guarantees to order writel() > within > one device, and the

Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 10:09 +0100, Will Deacon wrote: > On Wed, Mar 28, 2018 at 09:00:01AM +, David Laight wrote: > > From: Will Deacon > > > Sent: 28 March 2018 09:54 > > > > ... > > > > > I don't think so. My reading of memory-barriers.txt says that writeX > > > > > might > > > > > expand

Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 10:07 +0100, Will Deacon wrote: > > For arm/arm64 we guarantee ordering for (1) but not for (2) -- you'd need to > add an mb() to make it work. > > Do both of these work on power? Yes. There's even another quirk, see further down ;-) > If so, I guess I can make readl

Re: RFC on writel and writel_relaxed

On Wed, Mar 28, 2018 at 11:50 AM, Benjamin Herrenschmidt wrote: > On Wed, 2018-03-28 at 09:53 +0100, Will Deacon wrote: >> For arm/arm64 these end up behaving exactly the same as readX/writeX, but >> I'm nervous about changing the documentation without understanding why

Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 09:53 +0100, Will Deacon wrote: > For arm/arm64 these end up behaving exactly the same as readX/writeX, but > I'm nervous about changing the documentation without understanding why it's > like it is currently. Maybe another ia64 thing?. I doubt it ... the Intel ancestry here

Re: RFC on writel and writel_relaxed

On Wed, Mar 28, 2018 at 09:00:01AM +, David Laight wrote: > From: Will Deacon > > Sent: 28 March 2018 09:54 > ... > > > > I don't think so. My reading of memory-barriers.txt says that writeX > > > > might > > > > expand to outX, and outX is not ordered with respect to other types of > > > >

Re: RFC on writel and writel_relaxed

On Wed, Mar 28, 2018 at 05:42:56PM +1100, Benjamin Herrenschmidt wrote: > On Tue, 2018-03-27 at 20:26 -1000, Linus Torvalds wrote: > > On Tue, Mar 27, 2018 at 6:33 PM, Benjamin Herrenschmidt > > wrote: > > > > > > This is why, I want (with your agreement) to define

RE: RFC on writel and writel_relaxed

From: Will Deacon > Sent: 28 March 2018 09:54 ... > > > I don't think so. My reading of memory-barriers.txt says that writeX might > > > expand to outX, and outX is not ordered with respect to other types of > > > memory. > > > > Ugh ? > > > > My understanding of HW at least is the exact opposite.

Re: RFC on writel and writel_relaxed

On Wed, Mar 28, 2018 at 08:29:45AM +1100, Benjamin Herrenschmidt wrote: > On Tue, 2018-03-27 at 15:36 +0100, Will Deacon wrote: > > > Can we say the same thing for iowrite32() and iowrite32be(). I also see > > > wmb() > > > in front of these. > > > > I don't think so. My reading of

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018, 20:43 Benjamin Herrenschmidt wrote: > > > > Of course, you'd have to be pretty odd to want to start a DMA with a > > read anyway - partly exactly because it's bad for performance since > > reads will be synchronous and not buffered like a write). >

Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 09:11 +0200, Arnd Bergmann wrote: > On Wed, Mar 28, 2018 at 8:56 AM, Benjamin Herrenschmidt > wrote: > > On Wed, 2018-03-28 at 06:53 +, Linus Torvalds wrote: > > > On Tue, Mar 27, 2018, 20:43 Benjamin Herrenschmidt > > >

Re: RFC on writel and writel_relaxed

On Wed, Mar 28, 2018 at 8:56 AM, Benjamin Herrenschmidt wrote: > On Wed, 2018-03-28 at 06:53 +, Linus Torvalds wrote: >> On Tue, Mar 27, 2018, 20:43 Benjamin Herrenschmidt >> wrote: >> That's why in/out were *so* slow, and why nobody uses

Re: RFC on writel and writel_relaxed

On Wed, 2018-03-28 at 06:53 +, Linus Torvalds wrote: > > > On Tue, Mar 27, 2018, 20:43 Benjamin Herrenschmidt ing.org> wrote: > > > > > > Of course, you'd have to be pretty odd to want to start a DMA > > with a > > > read anyway - partly exactly because it's bad for

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 20:26 -1000, Linus Torvalds wrote: > On Tue, Mar 27, 2018 at 6:33 PM, Benjamin Herrenschmidt > wrote: > > > > This is why, I want (with your agreement) to define clearly and once > > and for all, that the Linux semantics of writel are that it is

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 6:33 PM, Benjamin Herrenschmidt wrote: > > This is why, I want (with your agreement) to define clearly and once > and for all, that the Linux semantics of writel are that it is ordered > with previous writes to coherent memory (*) Honestly, I

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 5:24 PM, Sinan Kaya wrote: > > Basically changing it to > > dma_buffer->foo = 1;/* WB */ > wmb() > writel_relaxed(KICK, DMA_KICK_REGISTER);/* UC */ > mmiowb() Why? Why not just remove the wmb(), and keep the barrier in

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 23:24 -0400, Sinan Kaya wrote: > On 3/27/2018 10:51 PM, Linus Torvalds wrote: > > > The discussion at hand is about > > > > > > dma_buffer->foo = 1;/* WB */ > > > writel(KICK, DMA_KICK_REGISTER);/* UC */ > > > > Yes. That

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 16:51 -1000, Linus Torvalds wrote: > On Tue, Mar 27, 2018 at 3:03 PM, Benjamin Herrenschmidt > wrote: > > > > The discussion at hand is about > > > > dma_buffer->foo = 1;/* WB */ > > writel(KICK,

Re: RFC on writel and writel_relaxed

On 3/27/2018 10:51 PM, Linus Torvalds wrote: >> The discussion at hand is about >> >> dma_buffer->foo = 1;/* WB */ >> writel(KICK, DMA_KICK_REGISTER);/* UC */ > Yes. That certainly is ordered on x86. In fact, afaik it's ordered > even if that writel()

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 3:03 PM, Benjamin Herrenschmidt wrote: > > The discussion at hand is about > > dma_buffer->foo = 1;/* WB */ > writel(KICK, DMA_KICK_REGISTER);/* UC */ Yes. That certainly is ordered on x86. In fact,

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 16:10 +0100, Will Deacon wrote: > To clarify: are you saying that on x86 you need a wmb() prior to a writel > if you want that writel to be ordered after prior writes to memory? Is this > specific to WC memory or some other non-standard attribute? > > The only reason we have

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 14:39 -1000, Linus Torvalds wrote: > On Tue, Mar 27, 2018 at 11:33 AM, Benjamin Herrenschmidt > wrote: > > > > Well, we need to clarify that once and for all, because as I wrote > > earlier, it was decreed by Linus more than a decade ago that

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 11:33 AM, Benjamin Herrenschmidt wrote: > > Well, we need to clarify that once and for all, because as I wrote > earlier, it was decreed by Linus more than a decade ago that writel > would be fully ordered by itself vs. previous memory stores (at

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 14:54 -0700, Alexander Duyck wrote: > On Tue, Mar 27, 2018 at 2:35 PM, Benjamin Herrenschmidt > wrote: > > On Tue, 2018-03-27 at 10:46 -0400, Sinan Kaya wrote: > > > combined buffers. > > > > > > Alex: > > > "Don't bother. I can tell you right now

Re: RFC on writel and writel_relaxed

On 3/27/2018 5:54 PM, Alexander Duyck wrote: > I view the wmb() + writel_relaxed() as more of a driver owning and > handling this itself. Besides in the Intel Ethernet driver case it is > better performance as our wmb() placement for us also provides a > secondary barrier so we don't need to add a

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 2:35 PM, Benjamin Herrenschmidt wrote: > On Tue, 2018-03-27 at 10:46 -0400, Sinan Kaya wrote: >> combined buffers. >> >> Alex: >> "Don't bother. I can tell you right now that for x86 you have to have a >> wmb() before the writel(). > > No, this

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 10:46 -0400, Sinan Kaya wrote: > combined buffers. > > Alex: > "Don't bother. I can tell you right now that for x86 you have to have a > wmb() before the writel(). No, this isn't the semantics of writel. You shouldn't need it unless something changed and we need to revisit

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 11:54 -0700, Alexander Duyck wrote: > As far as I know the code has been this way for a while, something > like 2002, when the barrier was already present in e1000. However > there it was calling out weakly ordered models "such as IA-64". Since > then pretty much all the

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 15:36 +0100, Will Deacon wrote: > > Can we say the same thing for iowrite32() and iowrite32be(). I also see > > wmb() > > in front of these. > > I don't think so. My reading of memory-barriers.txt says that writeX might > expand to outX, and outX is not ordered with respect

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 08:12 -0600, Jason Gunthorpe wrote: > > I have been converting wmb+writel to wmb+writel_relaxed. (About 30 patches) > > > > I will have to just remove the wmb and keep writel, then repost. > > Okay, but before you do that, can we get a statement how this works > for WC? >

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 09:46 -0400, Sinan Kaya wrote: > On 3/27/2018 7:02 AM, Will Deacon wrote: > > - See Documentation/DMA-API.txt for more information on consistent > > memory. > > + can see it now has ownership. Note that, when using writel(), a prior > > + wmb() is not needed to

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 9:54 PM, Arnd Bergmann wrote: > On Tue, Mar 27, 2018 at 8:54 PM, Alexander Duyck > wrote: >> On Tue, Mar 27, 2018 at 8:10 AM, Will Deacon wrote: > > 11.10 STORE BUFFER > Intel 64 and IA-32 processors

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 8:54 PM, Alexander Duyck wrote: > On Tue, Mar 27, 2018 at 8:10 AM, Will Deacon wrote: >>> >>> Sinan >>> "We are being told that if you use writel(), then you don't need a wmb() on >>> all architectures." >>> >>> Alex: >>>

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 8:10 AM, Will Deacon wrote: > Hi Alex, > > On Tue, Mar 27, 2018 at 10:46:58AM -0400, Sinan Kaya wrote: >> +netdev, +Alex >> >> On 3/26/2018 6:00 PM, Benjamin Herrenschmidt wrote: >> > On Mon, 2018-03-26 at 23:30 +0200, Arnd Bergmann wrote: >> >> Most

Re: RFC on writel and writel_relaxed

Hi, On 27-03-2018 15:46, Sinan Kaya wrote: > > Sinan > "We are being told that if you use writel(), then you don't need a wmb() on > all architectures." > > Alex: > "I'm not sure who told you that but that is incorrect, at least for > x86. If you attempt to use writel() without the wmb() we will

Re: RFC on writel and writel_relaxed

Hi Alex, On Tue, Mar 27, 2018 at 10:46:58AM -0400, Sinan Kaya wrote: > +netdev, +Alex > > On 3/26/2018 6:00 PM, Benjamin Herrenschmidt wrote: > > On Mon, 2018-03-26 at 23:30 +0200, Arnd Bergmann wrote: > >> Most of the drivers have a unwound loop with writeq() or something to > >>> do it. > >>

Re: RFC on writel and writel_relaxed

+netdev, +Alex On 3/26/2018 6:00 PM, Benjamin Herrenschmidt wrote: > On Mon, 2018-03-26 at 23:30 +0200, Arnd Bergmann wrote: >> Most of the drivers have a unwound loop with writeq() or something to >>> do it. >> >> But isn't the writeq() barrier much more expensive than anything you'd >> do in

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 09:46:51AM -0400, Sinan Kaya wrote: > On 3/27/2018 7:02 AM, Will Deacon wrote: > > - See Documentation/DMA-API.txt for more information on consistent > > memory. > > + can see it now has ownership. Note that, when using writel(), a prior > > + wmb() is not

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 10:42:00AM +0100, Will Deacon wrote: > On Tue, Mar 27, 2018 at 09:56:47AM +0200, Arnd Bergmann wrote: > > On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: > > > On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin Herrenschmidt wrote: > > >> On Mon,

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 09:56:47AM +0200, Arnd Bergmann wrote: > On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: > > On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin Herrenschmidt wrote: > >> On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > > > I even see patches

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 08:22:55AM -0400, ok...@codeaurora.org wrote: > On 2018-03-27 07:23, Benjamin Herrenschmidt wrote: > >On Tue, 2018-03-27 at 11:44 +0200, Arnd Bergmann wrote: > >>> The interesting thing is that we do seem to have a whole LOT of these > >>> spurrious wmb before writel all

Re: RFC on writel and writel_relaxed

On 3/27/2018 7:02 AM, Will Deacon wrote: > - See Documentation/DMA-API.txt for more information on consistent memory. > + can see it now has ownership. Note that, when using writel(), a prior > + wmb() is not needed to guarantee that the cache coherent memory writes > + have

RE: RFC on writel and writel_relaxed

> Fair enough. I'd rather people used _relaxed by default, but I have to admit > that it will probably just result in them getting things wrong... Certainly requiring the driver writes use explicit barriers should make them understand when and why they are needed - and then put in the correct

Re: RFC on writel and writel_relaxed

On 2018-03-27 07:23, Benjamin Herrenschmidt wrote: On Tue, 2018-03-27 at 11:44 +0200, Arnd Bergmann wrote: > The interesting thing is that we do seem to have a whole LOT of these > spurrious wmb before writel all over the tree, I suspect because of > that incorrect recommendation in

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 10:57 +0100, Will Deacon wrote: > > > > The interesting thing is that we do seem to have a whole LOT of these > > spurrious wmb before writel all over the tree, I suspect because of > > that incorrect recommendation in memory-barriers.txt. > > > > We should fix that. > >

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 12:02 +0100, Will Deacon wrote: > can see it now has ownership. Note that, when using writel(), a prior > > wmb() is not needed to guarantee that the cache coherent memory writes > > have completed before writing to the cache incoherent MMIO region. > >

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 10:20:02PM +1100, Benjamin Herrenschmidt wrote: > On Tue, 2018-03-27 at 10:42 +0100, Will Deacon wrote: > > > > > > This example adds a wmb() between two writes to a coherent DMA > > > area, it is definitely required there. I'm pretty sure I've never seen > > > any bug

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 11:44 +0200, Arnd Bergmann wrote: > > The interesting thing is that we do seem to have a whole LOT of these > > spurrious wmb before writel all over the tree, I suspect because of > > that incorrect recommendation in memory-barriers.txt. > > > > We should fix that. > >

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 10:42 +0100, Will Deacon wrote: > > > > This example adds a wmb() between two writes to a coherent DMA > > area, it is definitely required there. I'm pretty sure I've never seen > > any bug reports pointing to a missing wmb() between memory > > and MMIO write accesses, but

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 1:02 PM, Will Deacon wrote: > On Tue, Mar 27, 2018 at 12:53:49PM +0200, Arnd Bergmann wrote: >> On Tue, Mar 27, 2018 at 12:09 PM, Will Deacon wrote: >> > On Tue, Mar 27, 2018 at 12:05:06PM +0200, Arnd Bergmann wrote: > diff --git

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 12:53:49PM +0200, Arnd Bergmann wrote: > On Tue, Mar 27, 2018 at 12:09 PM, Will Deacon wrote: > > On Tue, Mar 27, 2018 at 12:05:06PM +0200, Arnd Bergmann wrote: > >> > - > >> > - See Documentation/DMA-API.txt for more information on consistent >

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 12:09 PM, Will Deacon wrote: > On Tue, Mar 27, 2018 at 12:05:06PM +0200, Arnd Bergmann wrote: >> > - >> > - See Documentation/DMA-API.txt for more information on consistent >> > memory. >> > + can see it now has ownership. Note that, when

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 12:05:06PM +0200, Arnd Bergmann wrote: > On Tue, Mar 27, 2018 at 11:57 AM, Will Deacon wrote: > > > > > From db0daeaf94f0f6232f8206fc07a74211324b11d9 Mon Sep 17 00:00:00 2001 > > From: Will Deacon > > Date: Tue, 27 Mar 2018

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 11:57 AM, Will Deacon wrote: > > From db0daeaf94f0f6232f8206fc07a74211324b11d9 Mon Sep 17 00:00:00 2001 > From: Will Deacon > Date: Tue, 27 Mar 2018 10:49:58 +0100 > Subject: [PATCH] docs/memory-barriers.txt: Fix broken DMA vs

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 11:44:22AM +0200, Arnd Bergmann wrote: > On Tue, Mar 27, 2018 at 10:56 AM, Benjamin Herrenschmidt > wrote: > > On Tue, 2018-03-27 at 09:56 +0200, Arnd Bergmann wrote: > >> On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: >

Re: RFC on writel and writel_relaxed

[+ locking/ordering/docs people] On Tue, Mar 27, 2018 at 07:56:59PM +1100, Benjamin Herrenschmidt wrote: > On Tue, 2018-03-27 at 09:56 +0200, Arnd Bergmann wrote: > > On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: > > > On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 10:56 AM, Benjamin Herrenschmidt wrote: > On Tue, 2018-03-27 at 09:56 +0200, Arnd Bergmann wrote: >> On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: >> >> I'm pretty sure I've never seen >> any bug reports pointing to a

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 09:56:47AM +0200, Arnd Bergmann wrote: > On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: > > On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin Herrenschmidt wrote: > >> On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > > > I even see patches

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 09:56 +0200, Arnd Bergmann wrote: > On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: > > On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin Herrenschmidt wrote: > > > On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > > > I even see patches

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: > On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin Herrenschmidt wrote: >> On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > I even see patches adding wmb() based on actual observed memory > corruption during testing

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 10:59:40AM +1100, Benjamin Herrenschmidt wrote: > On Mon, 2018-03-26 at 16:50 -0600, Jason Gunthorpe wrote: > > On Tue, Mar 27, 2018 at 09:36:11AM +1100, Benjamin Herrenschmidt wrote: > > > On Mon, 2018-03-26 at 16:27 -0600, Jason Gunthorpe wrote: > > > > > Otherwise almost

Re: RFC on writel and writel_relaxed

On Mon, 2018-03-26 at 16:50 -0600, Jason Gunthorpe wrote: > On Tue, Mar 27, 2018 at 09:36:11AM +1100, Benjamin Herrenschmidt wrote: > > On Mon, 2018-03-26 at 16:27 -0600, Jason Gunthorpe wrote: > > > > Otherwise almost all drivers out there are broken which I very much > > > > doubt :-) > > > > >

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 09:36:11AM +1100, Benjamin Herrenschmidt wrote: > On Mon, 2018-03-26 at 16:27 -0600, Jason Gunthorpe wrote: > > > Otherwise almost all drivers out there are broken which I very much > > > doubt :-) > > > > But.. Sinan is right, you look anywhere in the driver tree and you

Re: RFC on writel and writel_relaxed

On Tue, 2018-03-27 at 09:36 +1100, Benjamin Herrenschmidt wrote: > I don't kow, it used to be the case, at least that's what drove us to > define things the way we did. > > Maybe things changed, but if that's the case, nobody knows for sure, > and we probably want to get Linus POV on the matter.

Re: RFC on writel and writel_relaxed

On Mon, 2018-03-26 at 16:27 -0600, Jason Gunthorpe wrote: > > Otherwise almost all drivers out there are broken which I very much > > doubt :-) > > But.. Sinan is right, you look anywhere in the driver tree and you > find stuff like this: > > drivers/net/ethernet/intel/i40e/i40e_txrx.c > >

Re: RFC on writel and writel_relaxed

On Mon, 2018-03-26 at 18:08 -0400, Sinan Kaya wrote: > On 3/26/2018 6:01 PM, Benjamin Herrenschmidt wrote: > > On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > > On 3/26/2018 5:30 PM, Arnd Bergmann wrote: > > > > > But that was never a requirement of writel(), > > > > >

Re: RFC on writel and writel_relaxed

On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin Herrenschmidt wrote: > On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > On 3/26/2018 5:30 PM, Arnd Bergmann wrote: > > > > But that was never a requirement of writel(), > > > > Documentation/memory-barriers.txt gives an explicit example

Re: RFC on writel and writel_relaxed

On 3/26/2018 6:01 PM, Benjamin Herrenschmidt wrote: > On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: >> On 3/26/2018 5:30 PM, Arnd Bergmann wrote: But that was never a requirement of writel(), Documentation/memory-barriers.txt gives an explicit example demanding the wmb()

Re: RFC on writel and writel_relaxed

On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > On 3/26/2018 5:30 PM, Arnd Bergmann wrote: > > > But that was never a requirement of writel(), > > > Documentation/memory-barriers.txt gives an explicit example demanding > > > the wmb() before writel() for ordering system memory against

Re: RFC on writel and writel_relaxed

On Mon, 2018-03-26 at 23:30 +0200, Arnd Bergmann wrote: > Most of the drivers have a unwound loop with writeq() or something to > > do it. > > But isn't the writeq() barrier much more expensive than anything you'd > do in function calls? It is for us, and will break any write combining. > > >

Re: RFC on writel and writel_relaxed

On 3/26/2018 5:30 PM, Arnd Bergmann wrote: >> But that was never a requirement of writel(), >> Documentation/memory-barriers.txt gives an explicit example demanding >> the wmb() before writel() for ordering system memory against writel. > Indeed, but it's in an example for when to use dma_wmb(),

Re: RFC on writel and writel_relaxed

On Mon, 2018-03-26 at 10:54 -0600, Jason Gunthorpe wrote: > On Mon, Mar 26, 2018 at 11:08:45AM +, David Laight wrote: > > > > This is a super performance critical operation for most drivers and > > > > directly impacts network performance. > > > > Perhaps there ought to be writel_nobarrier()

Re: RFC on writel and writel_relaxed

On Mon, Mar 26, 2018 at 11:09 PM, Jason Gunthorpe wrote: > On Mon, Mar 26, 2018 at 10:43:43PM +0200, Arnd Bergmann wrote: >> On Mon, Mar 26, 2018 at 10:25 PM, Jason Gunthorpe wrote: >> > On Mon, Mar 26, 2018 at 09:44:15PM +0200, Arnd Bergmann wrote: >> >> On Mon,

Re: RFC on writel and writel_relaxed

On Mon, Mar 26, 2018 at 10:43:43PM +0200, Arnd Bergmann wrote: > On Mon, Mar 26, 2018 at 10:25 PM, Jason Gunthorpe wrote: > > On Mon, Mar 26, 2018 at 09:44:15PM +0200, Arnd Bergmann wrote: > >> On Mon, Mar 26, 2018 at 6:54 PM, Jason Gunthorpe wrote: > >> > On Mon,

Re: RFC on writel and writel_relaxed

On Mon, Mar 26, 2018 at 10:25 PM, Jason Gunthorpe wrote: > On Mon, Mar 26, 2018 at 09:44:15PM +0200, Arnd Bergmann wrote: >> On Mon, Mar 26, 2018 at 6:54 PM, Jason Gunthorpe wrote: >> > On Mon, Mar 26, 2018 at 11:08:45AM +, David Laight wrote: >> >> > > This is

Re: RFC on writel and writel_relaxed

On Mon, Mar 26, 2018 at 09:44:15PM +0200, Arnd Bergmann wrote: > On Mon, Mar 26, 2018 at 6:54 PM, Jason Gunthorpe wrote: > > On Mon, Mar 26, 2018 at 11:08:45AM +, David Laight wrote: > >> > > This is a super performance critical operation for most drivers and > >> > > directly

  1   2   >