Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-10-01 Thread Christoph Hellwig
On Wed, Sep 27, 2017 at 10:13:51AM -0700, Dan Williams wrote: > As far as I can see "Offset can be greater than PAGE_SIZE" is the only > safe assumption for core code. It seems completely bogus to me, but if it is the current assumption we'll have to document it. But this brings me back to that o

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-28 Thread Harsh Jain
On 28-09-2017 18:35, Raj, Ashok wrote: > Thanks for trying that Harsh. > > sp_off turns of super page support. Which this mode, do you still see offsets > greater than 4k? Yes, offset greater than 4k is still there. Refer below. [56732.774872] offset 4110 len 76 dma addr 3a531200e dma len 76 [56

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-28 Thread Raj, Ashok
Thanks for trying that Harsh. sp_off turns of super page support. Which this mode, do you still see offsets greater than 4k? On Thu, Sep 28, 2017 at 07:08:21PM +0530, Harsh Jain wrote: > > > Today I tried with "Intel_iommu=sp_off" boot option. Traffic runs without any > error for more than 1

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-28 Thread Harsh Jain
On 28-09-2017 02:59, Casey Leedom wrote: > Hey Raj, > > Let us know if you need help in gathering more debugging information. For > the time being we've decided to ERRATA the use of the Intel I/O MMU with > IPsec till we Root Cause the issue. But this is still at the top of Harsh's > bug list

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-28 Thread Herbert Xu
On Wed, Sep 27, 2017 at 10:13:04PM +, Casey Leedom wrote: > | From: Raj, Ashok > | Sent: Wednesday, September 27, 2017 12:07 PM > | > | looking at the debug output i got from Harsh it still looks like a bug in > | the code. > | > | [ 538.284589] __domain_mapping nr_pages 0x1 > | [ 538.284600]

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Harsh Jain
On 28-09-2017 03:43, Casey Leedom wrote: > | From: Raj, Ashok > | Sent: Wednesday, September 27, 2017 12:07 PM > | > | looking at the debug output i got from Harsh it still looks like a bug in > | the code. > | > | [ 538.284589] __domain_mapping nr_pages 0x1 > | [ 538.284600] __domain_mapping sg

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Casey Leedom
| From: Raj, Ashok | Sent: Wednesday, September 27, 2017 12:07 PM | | looking at the debug output i got from Harsh it still looks like a bug in | the code. | | [ 538.284589] __domain_mapping nr_pages 0x1 | [ 538.284600] __domain_mapping sg_res 0x1 sg->dma_address 0xf291000e dma len | 0x38 pteval

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Raj, Ashok
Hi Casey looking at the debug output i got from Harsh it still looks like a bug in the code. [ 538.284589] __domain_mapping nr_pages 0x1 [ 538.284600] __domain_mapping sg_res 0x1 sg->dma_address 0xf291000e dma len 0x38 pteval 0x3cbce3003 phys_pfn 0x3cbce3 [ 538.284604] chelsio driver - offse

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Casey Leedom
Hey Raj, Let us know if you need help in gathering more debugging information. For the time being we've decided to ERRATA the use of the Intel I/O MMU with IPsec till we Root Cause the issue. But this is still at the top of Harsh's bug list. With Robin's comments, I'm almost sure that the:

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Raj, Ashok
Hi Robin On Wed, Sep 27, 2017 at 06:18:02PM +0100, Robin Murphy wrote: > On Wed, 27 Sep 2017 16:31:04 + > Casey Leedom wrote: > > > | From: Dan Williams > > | Sent: Tuesday, September 26, 2017 9:10 AM > > | > > | On Tue, Sep 26, 2017 at 9:06 AM, Casey Leedom > > wrote: | > | From: Robin

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Casey Leedom
| From: Robin Murphy | Sent: Wednesday, September 27, 2017 10:18 AM | | From my experience, in general terms each scatterlist segment | represents some contiguous quantity of pages, of which sg->page is the | first, while sg->length and sg->offset describe the specific bounds of | that segment's

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Robin Murphy
On Wed, 27 Sep 2017 16:31:04 + Casey Leedom wrote: > | From: Dan Williams > | Sent: Tuesday, September 26, 2017 9:10 AM > | > | On Tue, Sep 26, 2017 at 9:06 AM, Casey Leedom > wrote: | > | From: Robin Murphy > | > | Sent: Tuesday, September 26, 2017 7:22 AM > | > |... > | > ... > | >

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Dan Williams
On Wed, Sep 27, 2017 at 9:31 AM, Casey Leedom wrote: > | From: Dan Williams > | Sent: Tuesday, September 26, 2017 9:10 AM > | > | On Tue, Sep 26, 2017 at 9:06 AM, Casey Leedom wrote: > | > | From: Robin Murphy > | > | Sent: Tuesday, September 26, 2017 7:22 AM > | > |... > | > ... > | > Regard

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-27 Thread Casey Leedom
| From: Dan Williams | Sent: Tuesday, September 26, 2017 9:10 AM | | On Tue, Sep 26, 2017 at 9:06 AM, Casey Leedom wrote: | > | From: Robin Murphy | > | Sent: Tuesday, September 26, 2017 7:22 AM | > |... | > ... | > Regardless, it seems that you agree that there's an issue with the Intel |

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Casey Leedom
So just to be 100% sure I understand the patch you're proposing, you got the first use of VTD_PAGE_SHIFT wrong; it should have been VTD_PAGE_MASK? I.e. diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 6784a05..d43b566 100644 --- a/drivers/iommu/intel-iommu.c +++ b/dr

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Robin Murphy
On 26/09/17 15:34, Raj, Ashok wrote: > On Tue, Sep 26, 2017 at 03:22:47PM +0100, Robin Murphy wrote: >> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c >> index 6784a05dd6b2..d7f7def81613 100644 >> --- a/drivers/iommu/intel-iommu.c >> +++ b/drivers/iommu/intel-iommu.c >> @@ -

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Raj, Ashok
Oops..minor typo.. VTD_PAGE_SHIFT instead of VTD_PAGE_MASK On Tue, Sep 26, 2017 at 07:34:41AM -0700, Ashok Raj wrote: > On Tue, Sep 26, 2017 at 03:22:47PM +0100, Robin Murphy wrote: > > diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c > > index 6784a05dd6b2..d7f7def81613 100

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Casey Leedom
| From: Robin Murphy | Sent: Tuesday, September 26, 2017 7:22 AM | | On 26/09/17 13:21, Harsh Jain wrote: | > Find attached new set of log. After repeated tries it panics. | | Thanks, that makes things a bit clearer - looks like fixing the physical | address/pteval calculation to not be off by a p

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Raj, Ashok
On Tue, Sep 26, 2017 at 03:22:47PM +0100, Robin Murphy wrote: > diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c > index 6784a05dd6b2..d7f7def81613 100644 > --- a/drivers/iommu/intel-iommu.c > +++ b/drivers/iommu/intel-iommu.c > @@ -2254,10 +2254,12 @@ static int __domain_mapp

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Dan Williams
On Tue, Sep 26, 2017 at 9:06 AM, Casey Leedom wrote: > | From: Robin Murphy > | Sent: Tuesday, September 26, 2017 7:22 AM > > | > | On 26/09/17 13:21, Harsh Jain wrote: > | > Find attached new set of log. After repeated tries it panics. > | > | Thanks, that makes things a bit clearer - looks like

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Robin Murphy
On 26/09/17 13:21, Harsh Jain wrote: > Find attached new set of log. After repeated tries it panics. Thanks, that makes things a bit clearer - looks like fixing the physical address/pteval calculation to not be off by a page in one direction wasn't helping much because the returned DMA address is

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Harsh Jain
On 26-09-2017 01:41, Dan Williams wrote: > On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom wrote: >> | From: Dan Williams >> | Sent: Monday, September 25, 2017 12:31 PM >> | ... >> | IIUC it looks like this has been broken ever since commit e1605495c716 >> | "intel-iommu: Introduce domain_sg_mapp

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Harsh Jain
Find attached new set of log. After repeated tries it panics. On 26-09-2017 09:16, Harsh Jain wrote: > On 26-09-2017 00:16, Casey Leedom wrote: >> | From: Raj, Ashok >> | Sent: Monday, September 25, 2017 8:54 AM >> | >> | Not sure how the page->offset would end up being greater than page-size? >

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-26 Thread Harsh Jain
On 26-09-2017 00:15, David Woodhouse wrote: > On Wed, 2017-09-20 at 16:01 +0800, Herbert Xu wrote: >> Harsh Jain wrote: >>>   >>> While debugging DMA mapping error in chelsio crypto driver we >> observed that when scatter/gather list received by driver has some >> entry with page->offset > 4096

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Harsh Jain
On 26-09-2017 00:16, Casey Leedom wrote: > | From: Raj, Ashok > | Sent: Monday, September 25, 2017 8:54 AM > | > | Not sure how the page->offset would end up being greater than page-size? Refer below > | > | If you have additional traces, please send them by. > | > | Is this a new driver? wonderi

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Casey Leedom
| From: Raj, Ashok | Sent: Monday, September 25, 2017 12:03 PM | | On Mon, Sep 25, 2017 at 01:11:04PM -0700, Dan Williams wrote: | > On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom wrote: | > > | From: Dan Williams | > > | Sent: Monday, September 25, 2017 12:31 PM | > > | ... | > > | IIUC it lo

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Raj, Ashok
Hi On Mon, Sep 25, 2017 at 01:11:04PM -0700, Dan Williams wrote: > On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom wrote: > > | From: Dan Williams > > | Sent: Monday, September 25, 2017 12:31 PM > > | ... > > | IIUC it looks like this has been broken ever since commit e1605495c716 > > | "intel-iom

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Casey Leedom
| From: David Woodhouse | Sent: Monday, September 25, 2017 11:45 AM | | On Wed, 2017-09-20 at 16:01 +0800, Herbert Xu wrote: | > Harsh Jain wrote: | > > | > > While debugging DMA mapping error in chelsio crypto driver we | > observed that when scatter/gather list received by driver has some | > e

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Dan Williams
On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom wrote: > | From: Dan Williams > | Sent: Monday, September 25, 2017 12:31 PM > | ... > | IIUC it looks like this has been broken ever since commit e1605495c716 > | "intel-iommu: Introduce domain_sg_mapping() to speed up > | intel_map_sg()". I.e. it loo

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Casey Leedom
| From: Dan Williams | Sent: Monday, September 25, 2017 12:31 PM | ... | IIUC it looks like this has been broken ever since commit e1605495c716 | "intel-iommu: Introduce domain_sg_mapping() to speed up | intel_map_sg()". I.e. it looks like the calculation for pte_val should | be: | |    pteval =

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Dan Williams
On Mon, Sep 25, 2017 at 10:46 AM, Casey Leedom wrote: > | From: Robin Murphy > | Sent: Wednesday, September 20, 2017 3:12 AM > | > | On 20/09/17 09:01, Herbert Xu wrote: > | > > | > Harsh Jain wrote: > | >> > | >> While debugging DMA mapping error in chelsio crypto driver we > | >> observed that

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread David Woodhouse
On Wed, 2017-09-20 at 16:01 +0800, Herbert Xu wrote: > Harsh Jain wrote: > >  > > While debugging DMA mapping error in chelsio crypto driver we > observed that when scatter/gather list received by driver has some > entry with page->offset > 4096 (PAGE_SIZE). It starts giving DMA > error.  Without

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Casey Leedom
| From: Raj, Ashok | Sent: Monday, September 25, 2017 8:54 AM | | Not sure how the page->offset would end up being greater than page-size? | | If you have additional traces, please send them by. | | Is this a new driver? wondering how we didn't run into this? According to Herbert Xu and one of

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Raj, Ashok
Hi Casey Sorry, somehow didn't see this one come by. On Mon, Sep 25, 2017 at 05:46:40PM +, Casey Leedom wrote: > | From: Robin Murphy > | Sent: Wednesday, September 20, 2017 3:12 AM > | > | On 20/09/17 09:01, Herbert Xu wrote: > | > > | > Harsh Jain wrote: > | >> > | >> While debugging DMA

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-25 Thread Casey Leedom
| From: Robin Murphy | Sent: Wednesday, September 20, 2017 3:12 AM | | On 20/09/17 09:01, Herbert Xu wrote: | > | > Harsh Jain wrote: | >> | >> While debugging DMA mapping error in chelsio crypto driver we | >> observed that when scatter/gather list received by driver has | >> some entry with pag

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-20 Thread Harsh Jain
On 20-09-2017 13:31, Herbert Xu wrote: > Harsh Jain wrote: >> While debugging DMA mapping error in chelsio crypto driver we observed that >> when scatter/gather list received by driver has some entry with page->offset >> > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works f

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-20 Thread Harsh Jain
On 20-09-2017 15:42, Robin Murphy wrote: > On 20/09/17 09:01, Herbert Xu wrote: >> Harsh Jain wrote: >>> While debugging DMA mapping error in chelsio crypto driver we observed that >>> when scatter/gather list received by driver has some entry with >>> page->offset > 4096 (PAGE_SIZE). It start

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-20 Thread Robin Murphy
On 20/09/17 09:01, Herbert Xu wrote: > Harsh Jain wrote: >> >> While debugging DMA mapping error in chelsio crypto driver we observed that >> when scatter/gather list received by driver has some entry with page->offset >> > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works fi

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-20 Thread Herbert Xu
Harsh Jain wrote: > > While debugging DMA mapping error in chelsio crypto driver we observed that > when scatter/gather list received by driver has some entry with page->offset > > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works fine. This is not a bug. The network stack

DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

2017-09-15 Thread Harsh Jain
Hi, While debugging DMA mapping error in chelsio crypto driver we observed that when scatter/gather list received by driver has some entry with page->offset > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works fine. Before reaching to chelsio crypto driver(driver/crypto/chels