Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On Wed, Sep 27, 2017 at 10:13:51AM -0700, Dan Williams wrote: > As far as I can see "Offset can be greater than PAGE_SIZE" is the only > safe assumption for core code. It seems completely bogus to me, but if it is the current assumption we'll have to document it. But this brings me back to that

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 28-09-2017 18:35, Raj, Ashok wrote: > Thanks for trying that Harsh. > > sp_off turns of super page support. Which this mode, do you still see offsets > greater than 4k? Yes, offset greater than 4k is still there. Refer below. [56732.774872] offset 4110 len 76 dma addr 3a531200e dma len 76

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Thanks for trying that Harsh. sp_off turns of super page support. Which this mode, do you still see offsets greater than 4k? On Thu, Sep 28, 2017 at 07:08:21PM +0530, Harsh Jain wrote: > > > Today I tried with "Intel_iommu=sp_off" boot option. Traffic runs without any > error for more than

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 28-09-2017 02:59, Casey Leedom wrote: > Hey Raj, > > Let us know if you need help in gathering more debugging information. For > the time being we've decided to ERRATA the use of the Intel I/O MMU with > IPsec till we Root Cause the issue. But this is still at the top of Harsh's > bug

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 28-09-2017 16:03, Herbert Xu wrote: > On Wed, Sep 27, 2017 at 10:13:04PM +, Casey Leedom wrote: >> | From: Raj, Ashok >> | Sent: Wednesday, September 27, 2017 12:07 PM >> | >> | looking at the debug output i got from Harsh it still looks like a bug in >> | the code.

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On Wed, Sep 27, 2017 at 10:13:04PM +, Casey Leedom wrote: > | From: Raj, Ashok > | Sent: Wednesday, September 27, 2017 12:07 PM > | > | looking at the debug output i got from Harsh it still looks like a bug in > | the code. > | > | [ 538.284589] __domain_mapping nr_pages

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 28-09-2017 03:43, Casey Leedom wrote: > | From: Raj, Ashok > | Sent: Wednesday, September 27, 2017 12:07 PM > | > | looking at the debug output i got from Harsh it still looks like a bug in > | the code. > | > | [ 538.284589] __domain_mapping nr_pages 0x1 > | [

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Raj, Ashok | Sent: Wednesday, September 27, 2017 12:07 PM | | looking at the debug output i got from Harsh it still looks like a bug in | the code. | | [ 538.284589] __domain_mapping nr_pages 0x1 | [ 538.284600] __domain_mapping sg_res 0x1 sg->dma_address 0xf291000e

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Hi Casey looking at the debug output i got from Harsh it still looks like a bug in the code. [ 538.284589] __domain_mapping nr_pages 0x1 [ 538.284600] __domain_mapping sg_res 0x1 sg->dma_address 0xf291000e dma len 0x38 pteval 0x3cbce3003 phys_pfn 0x3cbce3 [ 538.284604] chelsio driver -

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Hey Raj, Let us know if you need help in gathering more debugging information. For the time being we've decided to ERRATA the use of the Intel I/O MMU with IPsec till we Root Cause the issue. But this is still at the top of Harsh's bug list. With Robin's comments, I'm almost sure that

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Hi Robin On Wed, Sep 27, 2017 at 06:18:02PM +0100, Robin Murphy wrote: > On Wed, 27 Sep 2017 16:31:04 + > Casey Leedom wrote: > > > | From: Dan Williams > > | Sent: Tuesday, September 26, 2017 9:10 AM > > | > > | On Tue, Sep 26, 2017 at 9:06

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Robin Murphy | Sent: Wednesday, September 27, 2017 10:18 AM | | From my experience, in general terms each scatterlist segment | represents some contiguous quantity of pages, of which sg->page is the | first, while sg->length and sg->offset describe the specific

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On Wed, 27 Sep 2017 16:31:04 + Casey Leedom wrote: > | From: Dan Williams > | Sent: Tuesday, September 26, 2017 9:10 AM > | > | On Tue, Sep 26, 2017 at 9:06 AM, Casey Leedom > wrote: | > | From: Robin Murphy

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On Wed, Sep 27, 2017 at 9:31 AM, Casey Leedom wrote: > | From: Dan Williams > | Sent: Tuesday, September 26, 2017 9:10 AM > | > | On Tue, Sep 26, 2017 at 9:06 AM, Casey Leedom wrote: > | > | From: Robin Murphy

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Dan Williams | Sent: Tuesday, September 26, 2017 9:10 AM | | On Tue, Sep 26, 2017 at 9:06 AM, Casey Leedom wrote: | > | From: Robin Murphy | > | Sent: Tuesday, September 26, 2017 7:22 AM | > |... | > ... | >

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

So just to be 100% sure I understand the patch you're proposing, you got the first use of VTD_PAGE_SHIFT wrong; it should have been VTD_PAGE_MASK? I.e. diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 6784a05..d43b566 100644 --- a/drivers/iommu/intel-iommu.c +++

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 26/09/17 15:34, Raj, Ashok wrote: > On Tue, Sep 26, 2017 at 03:22:47PM +0100, Robin Murphy wrote: >> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c >> index 6784a05dd6b2..d7f7def81613 100644 >> --- a/drivers/iommu/intel-iommu.c >> +++ b/drivers/iommu/intel-iommu.c >> @@

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Robin Murphy | Sent: Tuesday, September 26, 2017 7:22 AM | | On 26/09/17 13:21, Harsh Jain wrote: | > Find attached new set of log. After repeated tries it panics. | | Thanks, that makes things a bit clearer - looks like fixing the physical | address/pteval

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Oops..minor typo.. VTD_PAGE_SHIFT instead of VTD_PAGE_MASK On Tue, Sep 26, 2017 at 07:34:41AM -0700, Ashok Raj wrote: > On Tue, Sep 26, 2017 at 03:22:47PM +0100, Robin Murphy wrote: > > diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c > > index 6784a05dd6b2..d7f7def81613

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On Tue, Sep 26, 2017 at 03:22:47PM +0100, Robin Murphy wrote: > diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c > index 6784a05dd6b2..d7f7def81613 100644 > --- a/drivers/iommu/intel-iommu.c > +++ b/drivers/iommu/intel-iommu.c > @@ -2254,10 +2254,12 @@ static int

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Robin Murphy | Sent: Tuesday, September 26, 2017 7:22 AM | | On 26/09/17 13:21, Harsh Jain wrote: | > Find attached new set of log. After repeated tries it panics. | | Thanks, that makes things a bit clearer - looks like fixing the physical | address/pteval

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 26/09/17 13:21, Harsh Jain wrote: > Find attached new set of log. After repeated tries it panics. Thanks, that makes things a bit clearer - looks like fixing the physical address/pteval calculation to not be off by a page in one direction wasn't helping much because the returned DMA address is

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 26-09-2017 01:41, Dan Williams wrote: > On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom wrote: >> | From: Dan Williams >> | Sent: Monday, September 25, 2017 12:31 PM >> | ... >> | IIUC it looks like this has been broken ever since commit

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Find attached new set of log. After repeated tries it panics. On 26-09-2017 09:16, Harsh Jain wrote: > On 26-09-2017 00:16, Casey Leedom wrote: >> | From: Raj, Ashok >> | Sent: Monday, September 25, 2017 8:54 AM >> | >> | Not sure how the page->offset would end up being

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 26-09-2017 00:15, David Woodhouse wrote: > On Wed, 2017-09-20 at 16:01 +0800, Herbert Xu wrote: >> Harsh Jain wrote: >>>   >>> While debugging DMA mapping error in chelsio crypto driver we >> observed that when scatter/gather list received by driver has some >> entry with

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 26-09-2017 00:16, Casey Leedom wrote: > | From: Raj, Ashok > | Sent: Monday, September 25, 2017 8:54 AM > | > | Not sure how the page->offset would end up being greater than page-size? Refer below > | > | If you have additional traces, please send them by. > | > | Is this

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Raj, Ashok | Sent: Monday, September 25, 2017 12:03 PM | | On Mon, Sep 25, 2017 at 01:11:04PM -0700, Dan Williams wrote: | > On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom wrote: | > > | From: Dan Williams | > > |

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Hi On Mon, Sep 25, 2017 at 01:11:04PM -0700, Dan Williams wrote: > On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom wrote: > > | From: Dan Williams > > | Sent: Monday, September 25, 2017 12:31 PM > > | ... > > | IIUC it looks like this has been broken

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: David Woodhouse | Sent: Monday, September 25, 2017 11:45 AM | | On Wed, 2017-09-20 at 16:01 +0800, Herbert Xu wrote: | > Harsh Jain wrote: | > > | > > While debugging DMA mapping error in chelsio crypto driver we | > observed that when

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom wrote: > | From: Dan Williams > | Sent: Monday, September 25, 2017 12:31 PM > | ... > | IIUC it looks like this has been broken ever since commit e1605495c716 > | "intel-iommu: Introduce

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Dan Williams | Sent: Monday, September 25, 2017 12:31 PM | ... | IIUC it looks like this has been broken ever since commit e1605495c716 | "intel-iommu: Introduce domain_sg_mapping() to speed up | intel_map_sg()". I.e. it looks like the calculation for pte_val

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On Mon, Sep 25, 2017 at 10:46 AM, Casey Leedom wrote: > | From: Robin Murphy > | Sent: Wednesday, September 20, 2017 3:12 AM > | > | On 20/09/17 09:01, Herbert Xu wrote: > | > > | > Harsh Jain wrote: > | >> > | >> While debugging DMA

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Raj, Ashok | Sent: Monday, September 25, 2017 8:54 AM | | Not sure how the page->offset would end up being greater than page-size? | | If you have additional traces, please send them by. | | Is this a new driver? wondering how we didn't run into this? According to

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On Wed, 2017-09-20 at 16:01 +0800, Herbert Xu wrote: > Harsh Jain wrote: > >  > > While debugging DMA mapping error in chelsio crypto driver we > observed that when scatter/gather list received by driver has some > entry with page->offset > 4096 (PAGE_SIZE). It starts giving

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Hi Casey Sorry, somehow didn't see this one come by. On Mon, Sep 25, 2017 at 05:46:40PM +, Casey Leedom wrote: > | From: Robin Murphy > | Sent: Wednesday, September 20, 2017 3:12 AM > | > | On 20/09/17 09:01, Herbert Xu wrote: > | > > | > Harsh Jain

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

| From: Robin Murphy | Sent: Wednesday, September 20, 2017 3:12 AM | | On 20/09/17 09:01, Herbert Xu wrote: | > | > Harsh Jain wrote: | >> | >> While debugging DMA mapping error in chelsio crypto driver we | >> observed that when scatter/gather list

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 20-09-2017 13:31, Herbert Xu wrote: > Harsh Jain wrote: >> While debugging DMA mapping error in chelsio crypto driver we observed that >> when scatter/gather list received by driver has some entry with page->offset >> > 4096 (PAGE_SIZE). It starts giving DMA error. 

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 20-09-2017 15:42, Robin Murphy wrote: > On 20/09/17 09:01, Herbert Xu wrote: >> Harsh Jain wrote: >>> While debugging DMA mapping error in chelsio crypto driver we observed that >>> when scatter/gather list received by driver has some entry with >>> page->offset > 4096

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

On 20/09/17 09:01, Herbert Xu wrote: > Harsh Jain wrote: >> >> While debugging DMA mapping error in chelsio crypto driver we observed that >> when scatter/gather list received by driver has some entry with page->offset >> > 4096 (PAGE_SIZE). It starts giving DMA error. 

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Harsh Jain wrote: > > While debugging DMA mapping error in chelsio crypto driver we observed that > when scatter/gather list received by driver has some entry with page->offset > > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works fine. This is not a

DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

Hi, While debugging DMA mapping error in chelsio crypto driver we observed that when scatter/gather list received by driver has some entry with page->offset > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works fine. Before reaching to chelsio crypto