Re: Requirement to get BAR pci_bus_address in user space

2018-06-15 Thread Srinath Mannam
Thank you very much Alex and Ben for all the details. Please find my comments in lined.. On Fri, Jun 15, 2018 at 1:34 AM, Walker, Benjamin wrote: > On Thu, 2018-06-14 at 08:50 -0600, Alex Williamson wrote: >> On Thu, 14 Jun 2018 16:18:15 +0530 >> Srinath Mannam wrote: >> >> > Hi Sinan Kaya,

Re: Requirement to get BAR pci_bus_address in user space

2018-06-15 Thread Srinath Mannam
Thank you very much Alex and Ben for all the details. Please find my comments in lined.. On Fri, Jun 15, 2018 at 1:34 AM, Walker, Benjamin wrote: > On Thu, 2018-06-14 at 08:50 -0600, Alex Williamson wrote: >> On Thu, 14 Jun 2018 16:18:15 +0530 >> Srinath Mannam wrote: >> >> > Hi Sinan Kaya,

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Walker, Benjamin
On Thu, 2018-06-14 at 08:50 -0600, Alex Williamson wrote: > On Thu, 14 Jun 2018 16:18:15 +0530 > Srinath Mannam wrote: > > > Hi Sinan Kaya, > > > > Here are the details, > > > > The issue is, For CMB cards SQs are allocated inside device BAR memory > > which is different from normal cards. > >

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Walker, Benjamin
On Thu, 2018-06-14 at 08:50 -0600, Alex Williamson wrote: > On Thu, 14 Jun 2018 16:18:15 +0530 > Srinath Mannam wrote: > > > Hi Sinan Kaya, > > > > Here are the details, > > > > The issue is, For CMB cards SQs are allocated inside device BAR memory > > which is different from normal cards. > >

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Christoph Hellwig
On Thu, Jun 14, 2018 at 08:50:15AM -0600, Alex Williamson wrote: > I don't understand the CQ vs CMB, but I think I gather that there's some > sort of buffer that's allocated from within the devices MMIO BAR and > some programming of the device needs to reference that buffer. > Wouldn't you

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Christoph Hellwig
On Thu, Jun 14, 2018 at 08:50:15AM -0600, Alex Williamson wrote: > I don't understand the CQ vs CMB, but I think I gather that there's some > sort of buffer that's allocated from within the devices MMIO BAR and > some programming of the device needs to reference that buffer. > Wouldn't you

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Alex Williamson
On Thu, 14 Jun 2018 16:18:15 +0530 Srinath Mannam wrote: > Hi Sinan Kaya, > > Here are the details, > > The issue is, For CMB cards SQs are allocated inside device BAR memory > which is different from normal cards. > In Normal cards SQ memory allocated at host side. > In both the cases

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Alex Williamson
On Thu, 14 Jun 2018 16:18:15 +0530 Srinath Mannam wrote: > Hi Sinan Kaya, > > Here are the details, > > The issue is, For CMB cards SQs are allocated inside device BAR memory > which is different from normal cards. > In Normal cards SQ memory allocated at host side. > In both the cases

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Keith Busch
On Thu, Jun 14, 2018 at 04:18:15PM +0530, Srinath Mannam wrote: > The issue is, For CMB cards SQs are allocated inside device BAR memory > which is different from normal cards. > In Normal cards SQ memory allocated at host side. > In both the cases physical address of CQ memory is programmed in

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Keith Busch
On Thu, Jun 14, 2018 at 04:18:15PM +0530, Srinath Mannam wrote: > The issue is, For CMB cards SQs are allocated inside device BAR memory > which is different from normal cards. > In Normal cards SQ memory allocated at host side. > In both the cases physical address of CQ memory is programmed in

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Srinath Mannam
Hi Sinan Kaya, Here are the details, The issue is, For CMB cards SQs are allocated inside device BAR memory which is different from normal cards. In Normal cards SQ memory allocated at host side. In both the cases physical address of CQ memory is programmed in NVMe controller register. This

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Srinath Mannam
Hi Sinan Kaya, Here are the details, The issue is, For CMB cards SQs are allocated inside device BAR memory which is different from normal cards. In Normal cards SQ memory allocated at host side. In both the cases physical address of CQ memory is programmed in NVMe controller register. This

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread okaya
On 2018-06-14 06:29, Srinath Mannam wrote: ++ Alex Williamson, kvm, Hi Christoph, Thank you for quick reply. If we want to add this in vfio then I think we need to do the same in uio case also. As I mentioned in previous mail, in the current implementation resource information (address and

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread okaya
On 2018-06-14 06:29, Srinath Mannam wrote: ++ Alex Williamson, kvm, Hi Christoph, Thank you for quick reply. If we want to add this in vfio then I think we need to do the same in uio case also. As I mentioned in previous mail, in the current implementation resource information (address and

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Srinath Mannam
++ Alex Williamson, kvm, Hi Christoph, Thank you for quick reply. If we want to add this in vfio then I think we need to do the same in uio case also. As I mentioned in previous mail, in the current implementation resource information (address and size) is gathering from resource named file

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Srinath Mannam
++ Alex Williamson, kvm, Hi Christoph, Thank you for quick reply. If we want to add this in vfio then I think we need to do the same in uio case also. As I mentioned in previous mail, in the current implementation resource information (address and size) is gathering from resource named file

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Christoph Hellwig
The only safe way to use PCI(e) devices in userspace is through vfio. I think that is where you need to take your inquiries.

Re: Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Christoph Hellwig
The only safe way to use PCI(e) devices in userspace is through vfio. I think that is where you need to take your inquiries.

Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Srinath Mannam
Hi Bjorn, We observed an issue in NVMe CMD cards in SPDK NVMe libraries. Similar issue fixed with "nvme-pci: Use PCI bus address for data/queues in CMB" commit in nvme kernel driver. As per the fix, pci_bus_address of CMB bar is required to program to NVMe cards. We need to implement similar

Requirement to get BAR pci_bus_address in user space

2018-06-14 Thread Srinath Mannam
Hi Bjorn, We observed an issue in NVMe CMD cards in SPDK NVMe libraries. Similar issue fixed with "nvme-pci: Use PCI bus address for data/queues in CMB" commit in nvme kernel driver. As per the fix, pci_bus_address of CMB bar is required to program to NVMe cards. We need to implement similar