Re: RFC on writel and writel_relaxed

2018-04-02 Thread Sinan Kaya
On 3/29/2018 9:40 PM, Benjamin Herrenschmidt wrote: > On Thu, 2018-03-29 at 09:56 -0400, Sinan Kaya wrote: >> On 3/28/2018 11:55 AM, David Miller wrote: >>> From: Benjamin Herrenschmidt >>> Date: Thu, 29 Mar 2018 02:13:16 +1100 >>> Let's fix all archs, it's way easier than fixing all drivers.

Re: RFC on writel and writel_relaxed

2018-03-29 Thread Benjamin Herrenschmidt
On Thu, 2018-03-29 at 09:56 -0400, Sinan Kaya wrote: > On 3/28/2018 11:55 AM, David Miller wrote: > > From: Benjamin Herrenschmidt > > Date: Thu, 29 Mar 2018 02:13:16 +1100 > > > > > Let's fix all archs, it's way easier than fixing all drivers. Half of > > > the archs are unused or dead anyway. >

Re: RFC on writel and writel_relaxed

2018-03-29 Thread Sinan Kaya
On 3/29/2018 12:29 PM, Arnd Bergmann wrote: > On Thu, Mar 29, 2018 at 3:56 PM, Sinan Kaya wrote: >> On 3/28/2018 11:55 AM, David Miller wrote: >>> From: Benjamin Herrenschmidt >>> Date: Thu, 29 Mar 2018 02:13:16 +1100 >>> Let's fix all archs, it's way easier than fixing all drivers. Half of

Re: RFC on writel and writel_relaxed

2018-03-29 Thread Arnd Bergmann
On Thu, Mar 29, 2018 at 3:56 PM, Sinan Kaya wrote: > On 3/28/2018 11:55 AM, David Miller wrote: >> From: Benjamin Herrenschmidt >> Date: Thu, 29 Mar 2018 02:13:16 +1100 >> >>> Let's fix all archs, it's way easier than fixing all drivers. Half of >>> the archs are unused or dead anyway. >> >> Agre

Re: RFC on writel and writel_relaxed

2018-03-29 Thread David Miller
From: Sinan Kaya Date: Thu, 29 Mar 2018 09:56:01 -0400 > sparc question sent Sparc never lets physical memory accesses pass MMIO, and vice versa. They are always strongly ordered amongst eachother. Therefore no explicit barrier instructions are necessary.

Re: RFC on writel and writel_relaxed

2018-03-29 Thread Sinan Kaya
On 3/28/2018 11:55 AM, David Miller wrote: > From: Benjamin Herrenschmidt > Date: Thu, 29 Mar 2018 02:13:16 +1100 > >> Let's fix all archs, it's way easier than fixing all drivers. Half of >> the archs are unused or dead anyway. > > Agreed. > I pinged most of the maintainers yesterday. Which a

Re: RFC on writel and writel_relaxed

2018-03-29 Thread Will Deacon
On Thu, Mar 29, 2018 at 08:31:32AM +1100, Benjamin Herrenschmidt wrote: > On Thu, 2018-03-29 at 02:23 +1000, Nicholas Piggin wrote: > > This is a variation on the mandatory write barrier that causes writes to > > weakly > > ordered I/O regions to be partially ordered. Its effects may go beyon

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Nicholas Piggin
On Thu, 29 Mar 2018 08:31:32 +1100 Benjamin Herrenschmidt wrote: > On Thu, 2018-03-29 at 02:23 +1000, Nicholas Piggin wrote: > > On Wed, 28 Mar 2018 11:55:09 -0400 (EDT) > > David Miller wrote: > > > > > From: Benjamin Herrenschmidt > > > Date: Thu, 29 Mar 2018 02:13:16 +1100 > > > > > >

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Benjamin Herrenschmidt
On Thu, 2018-03-29 at 02:23 +1000, Nicholas Piggin wrote: > On Wed, 28 Mar 2018 11:55:09 -0400 (EDT) > David Miller wrote: > > > From: Benjamin Herrenschmidt > > Date: Thu, 29 Mar 2018 02:13:16 +1100 > > > > > Let's fix all archs, it's way easier than fixing all drivers. Half of > > > the archs

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Nicholas Piggin
On Wed, 28 Mar 2018 11:55:09 -0400 (EDT) David Miller wrote: > From: Benjamin Herrenschmidt > Date: Thu, 29 Mar 2018 02:13:16 +1100 > > > Let's fix all archs, it's way easier than fixing all drivers. Half of > > the archs are unused or dead anyway. > > Agreed. While we're making decrees her

RE: RFC on writel and writel_relaxed

2018-03-28 Thread David Laight
From: Benjamin Herrenschmidt > Sent: 28 March 2018 16:13 ... > > I've always wondered exactly what the twi;isync were for - always seemed > > very heavy handed for most mmio reads. > > Particularly if you are doing mmio reads from a data fifo. > > If you do that you should use the "s" version of t

Re: RFC on writel and writel_relaxed

2018-03-28 Thread David Miller
From: Benjamin Herrenschmidt Date: Thu, 29 Mar 2018 02:13:16 +1100 > Let's fix all archs, it's way easier than fixing all drivers. Half of > the archs are unused or dead anyway. Agreed.

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Benjamin Herrenschmidt
On Wed, 2018-03-28 at 07:41 -0400, ok...@codeaurora.org wrote: > Yes, we want to get there indeed. It is because of some arch not > implementing writel properly. Maintainers want to play safe. > > That is why I asked if IA64 and other well known archs follow the > strongly ordered rule at this m

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Benjamin Herrenschmidt
On Wed, 2018-03-28 at 11:30 +, David Laight wrote: > From: Benjamin Herrenschmidt > > Sent: 28 March 2018 10:56 > > ... > > For example, let's say I have a device with a reset bit and the spec > > says the reset bit needs to be set for at least 10us. > > > > This is wrong: > > > > writel

Re: RFC on writel and writel_relaxed

2018-03-28 Thread okaya
On 2018-03-28 02:14, Linus Torvalds wrote: On Tue, Mar 27, 2018 at 5:24 PM, Sinan Kaya wrote: Basically changing it to dma_buffer->foo = 1;/* WB */ wmb() writel_relaxed(KICK, DMA_KICK_REGISTER);/* UC */ mmiowb() Why? Why not just remove the wmb(), and keep the

RE: RFC on writel and writel_relaxed

2018-03-28 Thread David Laight
From: Benjamin Herrenschmidt > Sent: 28 March 2018 10:56 ... > For example, let's say I have a device with a reset bit and the spec > says the reset bit needs to be set for at least 10us. > > This is wrong: > > writel(1, RESET_REG); > usleep(10); > writel(0, RESET_REG); > > Bec

Re: Aw: Re: RFC on writel and writel_relaxed

2018-03-28 Thread Benjamin Herrenschmidt
On Wed, 2018-03-28 at 12:13 +0200, Lino Sanfilippo wrote: > Hi, > > > > > > Yeah so that other trick I'm talking about is also used for timing > > accuracy. > > > > For example, let's say I have a device with a reset bit and the spec > > says the reset bit needs to be set for at least 10us. > >

Aw: Re: RFC on writel and writel_relaxed

2018-03-28 Thread Lino Sanfilippo
Hi, > > Yeah so that other trick I'm talking about is also used for timing > accuracy. > > For example, let's say I have a device with a reset bit and the spec > says the reset bit needs to be set for at least 10us. > > This is wrong: > > writel(1, RESET_REG); > usleep(10); >

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Benjamin Herrenschmidt
On Wed, 2018-03-28 at 10:07 +0100, Will Deacon wrote: > > For arm/arm64 we guarantee ordering for (1) but not for (2) -- you'd need to > add an mb() to make it work. > > Do both of these work on power? Yes. There's even another quirk, see further down ;-) > If so, I guess I can make readl even

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Will Deacon
On Wed, Mar 28, 2018 at 05:42:56PM +1100, Benjamin Herrenschmidt wrote: > On Tue, 2018-03-27 at 20:26 -1000, Linus Torvalds wrote: > > On Tue, Mar 27, 2018 at 6:33 PM, Benjamin Herrenschmidt > > wrote: > > > > > > This is why, I want (with your agreement) to define clearly and once > > > and for

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Benjamin Herrenschmidt
On Wed, 2018-03-28 at 09:11 +0200, Arnd Bergmann wrote: > On Wed, Mar 28, 2018 at 8:56 AM, Benjamin Herrenschmidt > wrote: > > On Wed, 2018-03-28 at 06:53 +, Linus Torvalds wrote: > > > On Tue, Mar 27, 2018, 20:43 Benjamin Herrenschmidt > > > wrote: > > > That's why in/out were *so* slow, an

Re: RFC on writel and writel_relaxed

2018-03-28 Thread Arnd Bergmann
On Wed, Mar 28, 2018 at 8:56 AM, Benjamin Herrenschmidt wrote: > On Wed, 2018-03-28 at 06:53 +, Linus Torvalds wrote: >> On Tue, Mar 27, 2018, 20:43 Benjamin Herrenschmidt >> wrote: >> That's why in/out were *so* slow, and why nobody uses them any more >> (well, the address size limitations

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Wed, 2018-03-28 at 06:53 +, Linus Torvalds wrote: > > > On Tue, Mar 27, 2018, 20:43 Benjamin Herrenschmidt ing.org> wrote: > > > > > > Of course, you'd have to be pretty odd to want to start a DMA > > with a > > > read anyway - partly exactly because it's bad for performance > > since > >

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Tue, 2018-03-27 at 20:26 -1000, Linus Torvalds wrote: > On Tue, Mar 27, 2018 at 6:33 PM, Benjamin Herrenschmidt > wrote: > > > > This is why, I want (with your agreement) to define clearly and once > > and for all, that the Linux semantics of writel are that it is ordered > > with previous wri

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Linus Torvalds
On Tue, Mar 27, 2018 at 6:33 PM, Benjamin Herrenschmidt wrote: > > This is why, I want (with your agreement) to define clearly and once > and for all, that the Linux semantics of writel are that it is ordered > with previous writes to coherent memory (*) Honestly, I think those are the sane seman

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Linus Torvalds
On Tue, Mar 27, 2018 at 5:24 PM, Sinan Kaya wrote: > > Basically changing it to > > dma_buffer->foo = 1;/* WB */ > wmb() > writel_relaxed(KICK, DMA_KICK_REGISTER);/* UC */ > mmiowb() Why? Why not just remove the wmb(), and keep the barrier in the writel()? The above

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Tue, 2018-03-27 at 23:24 -0400, Sinan Kaya wrote: > On 3/27/2018 10:51 PM, Linus Torvalds wrote: > > > The discussion at hand is about > > > > > > dma_buffer->foo = 1;/* WB */ > > > writel(KICK, DMA_KICK_REGISTER);/* UC */ > > > > Yes. That certainly

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Tue, 2018-03-27 at 16:51 -1000, Linus Torvalds wrote: > On Tue, Mar 27, 2018 at 3:03 PM, Benjamin Herrenschmidt > wrote: > > > > The discussion at hand is about > > > > dma_buffer->foo = 1;/* WB */ > > writel(KICK, DMA_KICK_REGISTER);/* UC */ > > Y

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Sinan Kaya
On 3/27/2018 10:51 PM, Linus Torvalds wrote: >> The discussion at hand is about >> >> dma_buffer->foo = 1;/* WB */ >> writel(KICK, DMA_KICK_REGISTER);/* UC */ > Yes. That certainly is ordered on x86. In fact, afaik it's ordered > even if that writel() mig

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Linus Torvalds
On Tue, Mar 27, 2018 at 3:03 PM, Benjamin Herrenschmidt wrote: > > The discussion at hand is about > > dma_buffer->foo = 1;/* WB */ > writel(KICK, DMA_KICK_REGISTER);/* UC */ Yes. That certainly is ordered on x86. In fact, afaik it's ordered even if tha

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Tue, 2018-03-27 at 16:10 +0100, Will Deacon wrote: > To clarify: are you saying that on x86 you need a wmb() prior to a writel > if you want that writel to be ordered after prior writes to memory? Is this > specific to WC memory or some other non-standard attribute? > > The only reason we have

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Tue, 2018-03-27 at 14:39 -1000, Linus Torvalds wrote: > On Tue, Mar 27, 2018 at 11:33 AM, Benjamin Herrenschmidt > wrote: > > > > Well, we need to clarify that once and for all, because as I wrote > > earlier, it was decreed by Linus more than a decade ago that writel > > would be fully ordere

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Linus Torvalds
On Tue, Mar 27, 2018 at 11:33 AM, Benjamin Herrenschmidt wrote: > > Well, we need to clarify that once and for all, because as I wrote > earlier, it was decreed by Linus more than a decade ago that writel > would be fully ordered by itself vs. previous memory stores (at least > on UC memory). Yes

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Tue, 2018-03-27 at 14:54 -0700, Alexander Duyck wrote: > On Tue, Mar 27, 2018 at 2:35 PM, Benjamin Herrenschmidt > wrote: > > On Tue, 2018-03-27 at 10:46 -0400, Sinan Kaya wrote: > > > combined buffers. > > > > > > Alex: > > > "Don't bother. I can tell you right now that for x86 you have to h

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Sinan Kaya
On 3/27/2018 5:54 PM, Alexander Duyck wrote: > I view the wmb() + writel_relaxed() as more of a driver owning and > handling this itself. Besides in the Intel Ethernet driver case it is > better performance as our wmb() placement for us also provides a > secondary barrier so we don't need to add a

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Alexander Duyck
On Tue, Mar 27, 2018 at 2:35 PM, Benjamin Herrenschmidt wrote: > On Tue, 2018-03-27 at 10:46 -0400, Sinan Kaya wrote: >> combined buffers. >> >> Alex: >> "Don't bother. I can tell you right now that for x86 you have to have a >> wmb() before the writel(). > > No, this isn't the semantics of write

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Tue, 2018-03-27 at 10:46 -0400, Sinan Kaya wrote: > combined buffers. > > Alex: > "Don't bother. I can tell you right now that for x86 you have to have a > wmb() before the writel(). No, this isn't the semantics of writel. You shouldn't need it unless something changed and we need to revisit

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Benjamin Herrenschmidt
On Tue, 2018-03-27 at 11:54 -0700, Alexander Duyck wrote: > As far as I know the code has been this way for a while, something > like 2002, when the barrier was already present in e1000. However > there it was calling out weakly ordered models "such as IA-64". Since > then pretty much all the hardw

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Arnd Bergmann
On Tue, Mar 27, 2018 at 9:54 PM, Arnd Bergmann wrote: > On Tue, Mar 27, 2018 at 8:54 PM, Alexander Duyck > wrote: >> On Tue, Mar 27, 2018 at 8:10 AM, Will Deacon wrote: > > 11.10 STORE BUFFER > Intel 64 and IA-32 processors temporarily store each write (store) to > memory in a store buffer. The

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Arnd Bergmann
On Tue, Mar 27, 2018 at 8:54 PM, Alexander Duyck wrote: > On Tue, Mar 27, 2018 at 8:10 AM, Will Deacon wrote: >>> >>> Sinan >>> "We are being told that if you use writel(), then you don't need a wmb() on >>> all architectures." >>> >>> Alex: >>> "I'm not sure who told you that but that is incorr

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Alexander Duyck
On Tue, Mar 27, 2018 at 8:10 AM, Will Deacon wrote: > Hi Alex, > > On Tue, Mar 27, 2018 at 10:46:58AM -0400, Sinan Kaya wrote: >> +netdev, +Alex >> >> On 3/26/2018 6:00 PM, Benjamin Herrenschmidt wrote: >> > On Mon, 2018-03-26 at 23:30 +0200, Arnd Bergmann wrote: >> >> Most of the drivers have a

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Will Deacon
Hi Alex, On Tue, Mar 27, 2018 at 10:46:58AM -0400, Sinan Kaya wrote: > +netdev, +Alex > > On 3/26/2018 6:00 PM, Benjamin Herrenschmidt wrote: > > On Mon, 2018-03-26 at 23:30 +0200, Arnd Bergmann wrote: > >> Most of the drivers have a unwound loop with writeq() or something to > >>> do it. > >> >

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Jose Abreu
Hi, On 27-03-2018 15:46, Sinan Kaya wrote: > > Sinan > "We are being told that if you use writel(), then you don't need a wmb() on > all architectures." > > Alex: > "I'm not sure who told you that but that is incorrect, at least for > x86. If you attempt to use writel() without the wmb() we will h

Re: RFC on writel and writel_relaxed

2018-03-27 Thread Sinan Kaya
+netdev, +Alex On 3/26/2018 6:00 PM, Benjamin Herrenschmidt wrote: > On Mon, 2018-03-26 at 23:30 +0200, Arnd Bergmann wrote: >> Most of the drivers have a unwound loop with writeq() or something to >>> do it. >> >> But isn't the writeq() barrier much more expensive than anything you'd >> do in fu