Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-12 Thread Oliver O'Halloran
On Wed, Jun 12, 2019 at 4:53 PM Christoph Hellwig wrote: > > On Wed, Jun 12, 2019 at 04:35:22PM +1000, Oliver O'Halloran wrote: > > Setting a 48 bit DMA mask doesn't work today because we only allocate > > IOMMU tables to cover the 0..2GB range of PCI bus addresses. > > I don't think that is true

Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-12 Thread Benjamin Herrenschmidt
On Wed, 2019-06-12 at 09:25 +0300, Oded Gabbay wrote: > > > You can't. Your device is broken. Devices that don't support DMAing to > > the full 64-bit deserve to be added to the trash pile. > > > > Hmm... right know they are added to customers data-centers but what do I know > ;) Well, some cu

Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-12 Thread Benjamin Herrenschmidt
On Wed, 2019-06-12 at 15:45 +1000, Oliver O'Halloran wrote: > > Also, are you sure about the MSI thing? The IODA3 spec says the only > important bits for a 64bit MSI are bits 61:60 (to hit the window) and > the lower bits that determine what IVE to use. Everything in between > is ignored so ORing

Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-11 Thread Christoph Hellwig
On Wed, Jun 12, 2019 at 04:35:22PM +1000, Oliver O'Halloran wrote: > Setting a 48 bit DMA mask doesn't work today because we only allocate > IOMMU tables to cover the 0..2GB range of PCI bus addresses. I don't think that is true upstream, and if it is we need to fix bug in the powerpc code. power

Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-11 Thread Oliver O'Halloran
On Wed, Jun 12, 2019 at 3:25 AM Oded Gabbay wrote: > > On Tue, Jun 11, 2019 at 8:03 PM Oded Gabbay wrote: > > > > On Tue, Jun 11, 2019 at 6:26 PM Greg KH wrote: > > > *snip* > > > > Now, when I tried to integrate Goya into a POWER9 machine, I got a > > reject from the call to pci_set_dma_mask(pd

Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-11 Thread Oded Gabbay
On Wed, Jun 12, 2019 at 1:53 AM Benjamin Herrenschmidt wrote: > > On Tue, 2019-06-11 at 20:22 +0300, Oded Gabbay wrote: > > > > > So, to summarize: > > > If I call pci_set_dma_mask with 48, then it fails on POWER9. However, > > > in runtime, I don't know if its POWER9 or not, so upon failure I wil

Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-11 Thread Oliver O'Halloran
On Wed, Jun 12, 2019 at 8:54 AM Benjamin Herrenschmidt wrote: > > On Tue, 2019-06-11 at 20:22 +0300, Oded Gabbay wrote: > > > > > So, to summarize: > > > If I call pci_set_dma_mask with 48, then it fails on POWER9. However, > > > in runtime, I don't know if its POWER9 or not, so upon failure I wil

Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-11 Thread Benjamin Herrenschmidt
On Tue, 2019-06-11 at 20:22 +0300, Oded Gabbay wrote: > > > So, to summarize: > > If I call pci_set_dma_mask with 48, then it fails on POWER9. However, > > in runtime, I don't know if its POWER9 or not, so upon failure I will > > call it again with 32, which makes our device pretty much unusable.

Re: [PATCH v2 8/8] habanalabs: enable 64-bit DMA mask in POWER9

2019-06-11 Thread Oded Gabbay
On Tue, Jun 11, 2019 at 8:03 PM Oded Gabbay wrote: > > On Tue, Jun 11, 2019 at 6:26 PM Greg KH wrote: > > > > On Tue, Jun 11, 2019 at 08:17:53AM -0700, Christoph Hellwig wrote: > > > On Tue, Jun 11, 2019 at 11:58:57AM +0200, Greg KH wrote: > > > > That feels like a big hack. ppc doesn't have any