RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Pravin Bathija
Stefan Roese wrote: Correct. IIRC, some PATA driver as Pravin already mentioned. Cheers, Stefan Thanks Stefan. The whole intention of the patch/hack (or whatever one might call it :) ) was to avoid rogue drivers from setting pci_cache_line_size to non-zero value even though the underlying

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Mikhail Zolotaryov
Benjamin Herrenschmidt wrote: On Wed, 2009-09-09 at 17:40 +0300, Mikhail Zolotaryov wrote: Hi Tom, In my case __dma_sync() calls flush_dcache_range() (it's due to alignment) from a tasklet - no OOPS. It uses dcbf instruction instead of dcbi - that's the difference as dcbf is not

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Benjamin Herrenschmidt
On Fri, 2009-09-11 at 10:17 +0300, Mikhail Zolotaryov wrote: Benjamin Herrenschmidt wrote: On Wed, 2009-09-09 at 17:40 +0300, Mikhail Zolotaryov wrote: Hi Tom, In my case __dma_sync() calls flush_dcache_range() (it's due to alignment) from a tasklet - no OOPS. It uses dcbf

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Tom Burns
Hi Ben, Benjamin Herrenschmidt wrote: On Wed, 2009-09-09 at 09:43 -0400, Tom Burns wrote: Hi, With the default config for the Sequoia board on 2.6.24, calling pci_dma_sync_sg_for_cpu() results in executing invalidate_dcache_range() in arch/ppc/kernel/misc.S from __dma_sync(). This

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Prodyut Hazarika
I tried, using our JTAG debugger (BDI3000), to pause operation after calling dma_alloc_coherent to examine the TLB entry for the memory returned by the call (which was just past CONFIG_CONSISTENT_START=0xff10). The TLB list loaded at the time that I paused operation did not show a

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Pravin Bathija
Benjamin Herrenschmidt wrote: Do you know many drivers that do config space accesses without using the config space accessors ? Such drivers should be banned to oblivion. Cheers, Ben. I'm not aware of such drivers in the 2.6.30+ kernel. Thanks, Pravin On Thu, 2009-09-10 at 22:35 -0700,

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Tom Burns
Hi, Thank you everyone for your help. I've been looking into the other dma/pci API calls (dma_alloc_coherent, pci_alloc_consistent). I don't see how either of these return memory mapped to a TLB with the I bit set to 1 in kernel 2.6.24. In our kernel code, the only use of the PPC44x_TLB_I

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Pravin Bathija
Tom Burns wrote Hi, Thank you everyone for your help. I've been looking into the other dma/pci API calls (dma_alloc_coherent, pci_alloc_consistent). I don't see how either of these return memory mapped to a TLB with the I bit set to 1 in kernel 2.6.24. In our kernel code, the only

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Wed, 2009-09-09 at 10:10 -0400, Tom Burns wrote: Hi Mikhail, Sorry, this DMA code is in a tasklet. Are you suggesting the processor is in supervisor mode at that time? Calling pci_dma_sync_sg_for_cpu() from the tasklet context is what generates the OOPS. The entire oops is as

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Wed, 2009-09-09 at 17:40 +0300, Mikhail Zolotaryov wrote: Hi Tom, In my case __dma_sync() calls flush_dcache_range() (it's due to alignment) from a tasklet - no OOPS. It uses dcbf instruction instead of dcbi - that's the difference as dcbf is not privileged. What it calls depends on

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Thu, 2009-09-10 at 15:53 -0400, Tom Burns wrote: Hi, Thank you everyone for your help. I've been looking into the other dma/pci API calls (dma_alloc_coherent, pci_alloc_consistent). I don't see how either of these return memory mapped to a TLB with the I bit set to 1 in kernel

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Thu, 2009-09-10 at 13:30 -0700, Pravin Bathija wrote: There is also a patch that was submitted for 440EPX a couple of years back. The 440EPX SOC causes hangs with Memory Read Multiple (MRM) commands. Whether MRM is used or not depends on the value of PCI_CACHE_LINE_SIZE register. I see

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Stefan Roese
On Friday 11 September 2009 04:44:34 Benjamin Herrenschmidt wrote: On Thu, 2009-09-10 at 13:30 -0700, Pravin Bathija wrote: There is also a patch that was submitted for 440EPX a couple of years back. The 440EPX SOC causes hangs with Memory Read Multiple (MRM) commands. Whether MRM is used

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Fri, 2009-09-11 at 07:12 +0200, Stefan Roese wrote: It's already there. See commit: 5ce4b59653b2c2053cd9a011918ac1e4747f24cc powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata Ok, that's another way to do it. Will catch nasty drivers who try to write directly rather than clear

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Stefan Roese
On Friday 11 September 2009 07:17:50 Benjamin Herrenschmidt wrote: On Fri, 2009-09-11 at 07:12 +0200, Stefan Roese wrote: It's already there. See commit: 5ce4b59653b2c2053cd9a011918ac1e4747f24cc powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata Ok, that's another way to do it.

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Thu, 2009-09-10 at 22:35 -0700, Pravin Bathija wrote: Thanks Stefan. The whole intention of the patch/hack (or whatever one might call it :) ) was to avoid rogue drivers from setting pci_cache_line_size to non-zero value even though the underlying hardware doesn't support MRM calls.

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Mikhail Zolotaryov
Hi, Why manage cache lines manually, if appropriate code is a part of __dma_sync / dma_sync_single_for_device of DMA API ? (implies CONFIG_NOT_COHERENT_CACHE enabled, as default for Sequoia Board) Prodyut Hazarika wrote: Hi Adam, Yes, I am using the 440EPx (same as the sequoia board).

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Mikhail Zolotaryov
Hi Tom, possible solution could be to use tasklet to perform DMA-related job (as in most cases DMA transfer is interrupt driven - makes sense). Tom Burns wrote: Hi, With the default config for the Sequoia board on 2.6.24, calling pci_dma_sync_sg_for_cpu() results in executing

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Tom Burns
Hi, With the default config for the Sequoia board on 2.6.24, calling pci_dma_sync_sg_for_cpu() results in executing invalidate_dcache_range() in arch/ppc/kernel/misc.S from __dma_sync(). This OOPses on PPC440 since it tries to call directly the assembly instruction dcbi, which can only be

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Tom Burns
Hi Mikhail, Sorry, this DMA code is in a tasklet. Are you suggesting the processor is in supervisor mode at that time? Calling pci_dma_sync_sg_for_cpu() from the tasklet context is what generates the OOPS. The entire oops is as follows, if it's relevant: Oops: kernel access of bad area,

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Adam Zilkie
Hi Prodyut, Yes, I am using the 440EPx (same as the sequoia board). Our ideDriver is DMA'ing blocks of 192-byte data over the PCI bus (using the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on timing) end up being partially corrupted when we try to parse the data in the virtual page.

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Prodyut Hazarika
Hi Adam, Yes, I am using the 440EPx (same as the sequoia board). Our ideDriver is DMA'ing blocks of 192-byte data over the PCI bus (using the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on timing) end up being partially corrupted when we try to parse the data in the virtual page.

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Prodyut Hazarika
We have found that using flush_dcache_range() after each DMA solves the problem. Ideally, we'd like to be able to allocate the virtual page in cache inhibited memory to avoid the performance loss from all the flush calls. To do this, we'd have to change our TLB sizes and reserve a TLB in

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Benjamin Herrenschmidt
On Tue, 2009-09-08 at 14:01 -0400, Adam Zilkie wrote: Hi Prodyut, Yes, I am using the 440EPx (same as the sequoia board). Our ideDriver is DMA'ing blocks of 192-byte data over the PCI bus (using the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on timing) end up being partially

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Benjamin Herrenschmidt
On Tue, 2009-09-08 at 15:30 -0400, Adam Zilkie wrote: All, We have found that using flush_dcache_range() after each DMA solves the problem. Ideally, we'd like to be able to allocate the virtual page in cache inhibited memory to avoid the performance loss from all the flush calls. To do

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Adam Zilkie
We are using pci_alloc_consistent() Adam On Tue, 2009-09-08 at 12:56 -0700, Prodyut Hazarika wrote: We have found that using flush_dcache_range() after each DMA solves the problem. Ideally, we'd like to be able to allocate the virtual page in cache inhibited memory to avoid the

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Adam Zilkie
All, We have found that using flush_dcache_range() after each DMA solves the problem. Ideally, we'd like to be able to allocate the virtual page in cache inhibited memory to avoid the performance loss from all the flush calls. To do this, we'd have to change our TLB sizes and reserve a TLB in

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Benjamin Herrenschmidt
On Tue, 2009-09-08 at 16:00 -0400, Adam Zilkie wrote: We are using pci_alloc_consistent() Then your flush should have no effect since pci_alloc_consistent will return I=1 mapped memory, unless you don't have CONFIG_NOT_COHERENT_CACHE for some reason. Cheers, Ben. Adam On Tue, 2009-09-08 at

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-06 Thread Benjamin Herrenschmidt
; linuxppc-dev@lists.ozlabs.org Subject: Re: AW: PowerPC PCI DMA issues (prefetch/coherency?) Hi Adam, If you have a look in include/asm-ppc/pgtable.h for the following section: #ifdef CONFIG_44x #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_GUARDED) #else #define

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Chris Pringle
Hi Adam, If you have a look in include/asm-ppc/pgtable.h for the following section: #ifdef CONFIG_44x #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_GUARDED) #else #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED) #endif Try adding _PAGE_COHERENT to the appropriate line

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Benjamin Herrenschmidt
On Thu, 2009-09-03 at 09:05 +0100, Chris Pringle wrote: Hi Adam, If you have a look in include/asm-ppc/pgtable.h for the following section: #ifdef CONFIG_44x #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_GUARDED) #else #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED)

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Chris Pringle
; linuxppc-dev@lists.ozlabs.org Subject: Re: AW: PowerPC PCI DMA issues (prefetch/coherency?) Hi Adam, If you have a look in include/asm-ppc/pgtable.h for the following section: #ifdef CONFIG_44x #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_GUARDED) #else #define _PAGE_BASE

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Wrobel Heinz-R39252
+heinz.wrobel=freescale@lists.ozlabs.org [mailto:linuxppc-dev-bounces+heinz.wrobel=freescale@lists.ozlabs.org ] On Behalf Of Chris Pringle Sent: Donnerstag, 3. September 2009 10:05 To: azil...@datacast.com Cc: Tom Burns; Andrea Zypchen; linuxppc-dev@lists.ozlabs.org Subject: Re: AW: PowerPC PCI

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Adam Zilkie
Chris, I noticed the following comment in pgtable.h: * - CACHE COHERENT bit (M) has no effect on PPC440 core, because it * doesn't support SMP. So we can use this as software bit, like * DIRTY. And _PAGE_COHERENT is not defined for the 44x (giving a compile error when I add it the

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Adam Zilkie
Ben, Thanks for your info. Are you sure there is L2 cache on the 440? I am seeing this problem with our custom IDE driver which is based on pretty old code. Our driver uses pci_alloc_consistent() to allocate the physical DMA memory and alloc_pages() to allocate a virtual page. It then uses

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Josh Boyer
On Thu, Sep 03, 2009 at 12:04:50PM -0400, Adam Zilkie wrote: Ben, Thanks for your info. Are you sure there is L2 cache on the 440? It depends on which 440 SoC you have. It also depends on that being configured in the kernel even if it does exist. I am seeing this problem with our custom IDE

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Prodyut Hazarika
Hi Adam, Are you sure there is L2 cache on the 440? It depends on the SoC you are using. SoC like 460EX (Canyonlands board) have L2Cache. It seems you are using a Sequoia board, which has a 440EPx SoC. 440EPx has a 440 cpu core, but no L2Cache. Could you please tell me which SoC you are using?

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-02 Thread Adam Zilkie
Hi Chris, I am having a problem similar to what you described in this discussion. We are using the ppc arch with 2.6.24 with CONFIG_SEQUOIA with compiles arch/ppc/kernel/head_44x.c (quite different from /arch/powerpc/kernel/head_32.S). I would like to apply your backporting patch to this

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-06-29 Thread Chris Pringle
Hi Sergej, I've attached the patch used to fix this issue. Both the patch to pgtable32.h and head_32.S are required in order to make it work. The change to pgtable32.h ensures that all pages are marked cache coherent (results in setting the M bit). The change to head_32.S ensures that the M