Skip to site navigation (Press enter)

[linuxkernelnewbies] gmane.linux.kernel.pci: Query related to pci_alloc_consistent()

Peter Teoh Tue, 03 Feb 2009 07:28:48 -0800

http://comments.gmane.org/gmane.linux.kernel.pci/2842

Query related to pci_alloc_consistent()

Hi all,

If there are multiple PCIe cards of the same type, e.g. 3 or 4 network
controllers, in a node.
Now, if one allocates a buffer using pci_alloc_consistent() using
pci_dev struct of one of the card.


1) Whether that buffer can be used by remaining cards also ?
OR
Whether DMA addresses returned by pci_alloc_consistent() can be used
by all the cards explained in above situation ?

2) What will be the case for multiple platforms/architectures ?

3) There can be multiple PCIe bridges giving multiple slots on some machines.
What will happen, when above situation is considered in such machines ?

Any link of reference can be helpful.

Thanking you,
Yogeshwar

Permalink | Reply |

Bjorn Helgaas | 22 Jan 17:09

Re: Query related to pci_alloc_consistent()

On Thursday 22 January 2009 04:26:22 am yogeshwar sonawane wrote:
> If there are multiple PCIe cards of the same type, e.g. 3 or 4 network
> controllers, in a node.
> Now, if one allocates a buffer using pci_alloc_consistent() using
> pci_dev struct of one of the card.
> 
> 1) Whether that buffer can be used by remaining cards also ?

Hi Yogeshwar,

No.  When you use pci_alloc_consistent() or any of the other DMA
mapping services, the results are only valid for the device you
specified.

The other similar devices might be behind different IOMMUs,
so you have to do separate calls for each device.

Generally drivers are written to deal with each device completely
separately, so there's little if any state shared between multiple
cards of the same type.

Bjorn

> Whether DMA addresses returned by pci_alloc_consistent() can be used
> by all the cards explained in above situation ?
> 
> 2) What will be the case for multiple platforms/architectures ?
> 
> 3) There can be multiple PCIe bridges giving multiple slots on some machines.
> What will happen, when above situation is considered in such machines ?

(Continue reading)

Permalink | Reply |

Grant Grundler | 22 Jan 20:02

Re: Query related to pci_alloc_consistent()

On Thu, Jan 22, 2009 at 04:56:22PM +0530, yogeshwar sonawane wrote:
> Hi all,
> 
> If there are multiple PCIe cards of the same type, e.g. 3 or 4 network
> controllers, in a node.
> Now, if one allocates a buffer using pci_alloc_consistent() using
> pci_dev struct of one of the card.
> 
> 1) Whether that buffer can be used by remaining cards also ?
> OR
> Whether DMA addresses returned by pci_alloc_consistent() can be used
> by all the cards explained in above situation ?

What bjorn said.

> 2) What will be the case for multiple platforms/architectures ?

pci_alloc_consistent() should conform to Documentation/DMA-API.txt
and Documentation/DMA-mapping.txt. This is true for any platform or
architecture.

BTW, the latter should be moved to Documentation/PCI....Jesse?

> 3) There can be multiple PCIe bridges giving multiple slots on some machines.
> What will happen, when above situation is considered in such machines ?

What Bjorn said.

hth,
grant

(Continue reading)

Permalink | Reply |

Jesse Barnes | 22 Jan 20:20

Re: Query related to pci_alloc_consistent()

On Thursday, January 22, 2009 11:02 am Grant Grundler wrote:
> On Thu, Jan 22, 2009 at 04:56:22PM +0530, yogeshwar sonawane wrote:
> > Hi all,
> >
> > If there are multiple PCIe cards of the same type, e.g. 3 or 4 network
> > controllers, in a node.
> > Now, if one allocates a buffer using pci_alloc_consistent() using
> > pci_dev struct of one of the card.
> >
> > 1) Whether that buffer can be used by remaining cards also ?
> > OR
> > Whether DMA addresses returned by pci_alloc_consistent() can be used
> > by all the cards explained in above situation ?
>
> What bjorn said.
>
> > 2) What will be the case for multiple platforms/architectures ?
>
> pci_alloc_consistent() should conform to Documentation/DMA-API.txt
> and Documentation/DMA-mapping.txt. This is true for any platform or
> architecture.
>
> BTW, the latter should be moved to Documentation/PCI....Jesse?

Yeah, I thought Randy had a patch to do that...

--

-- 
Jesse Barnes, Intel Open Source Technology Center

Permalink | Reply |

Randy Dunlap | 22 Jan 20:27

Re: Query related to pci_alloc_consistent()

Jesse Barnes wrote:
> On Thursday, January 22, 2009 11:02 am Grant Grundler wrote:
>> On Thu, Jan 22, 2009 at 04:56:22PM +0530, yogeshwar sonawane wrote:
>>> Hi all,
>>>
>>> If there are multiple PCIe cards of the same type, e.g. 3 or 4 network
>>> controllers, in a node.
>>> Now, if one allocates a buffer using pci_alloc_consistent() using
>>> pci_dev struct of one of the card.
>>>
>>> 1) Whether that buffer can be used by remaining cards also ?
>>> OR
>>> Whether DMA addresses returned by pci_alloc_consistent() can be used
>>> by all the cards explained in above situation ?
>> What bjorn said.
>>
>>> 2) What will be the case for multiple platforms/architectures ?
>> pci_alloc_consistent() should conform to Documentation/DMA-API.txt
>> and Documentation/DMA-mapping.txt. This is true for any platform or
>> architecture.
>>
>> BTW, the latter should be moved to Documentation/PCI....Jesse?
> 
> Yeah, I thought Randy had a patch to do that...

I do/did.  I wonder how/where/when it got lost.  :(

I found it, will update it.

Thanks,

(Continue reading)

Permalink | Reply |

Randy Dunlap | 23 Jan 00:41

[PATCH] docs: move DMA-mapping.txt to Documentation/PCI/

From: Randy Dunlap <randy.dunlap <at> oracle.com>

Move DMA-mapping.txt to Documentation/PCI/.

DMA-mapping.txt was supposed to be moved from Documentation/ to
Documentation/PCI/.  The 00-INDEX files in those two directories
were updated, along with a few other text files, but the file
itself somehow escaped being moved, so move it and update more
text files and source files with its new location.

Signed-off-by: Randy Dunlap <randy.dunlap <at> oracle.com>
cc:	Greg Kroah-Hartman <gregkh <at> suse.de> # for staging/
cc:	Jesse Barnes <jbarnes <at> virtuousgeek.org>
---
 Documentation/DMA-API.txt                   |    2 
 Documentation/DMA-mapping.txt               |  766 ----------------------------
 Documentation/IO-mapping.txt                |    4 
 Documentation/PCI/PCI-DMA-mapping.txt       |  766 ++++++++++++++++++++++++++++
 Documentation/block/biodoc.txt              |    5 
 Documentation/usb/dma.txt                   |   11 
 arch/ia64/hp/common/sba_iommu.c             |   12 
 arch/parisc/include/asm/dma-mapping.h       |    2 
 arch/parisc/kernel/pci-dma.c                |    2 
 arch/x86/include/asm/dma-mapping.h          |    4 
 arch/x86/kernel/pci-gart_64.c               |    2 
 drivers/parisc/sba_iommu.c                  |   18 
 drivers/staging/altpciechdma/altpciechdma.c |    4 
 include/media/videobuf-dma-sg.h             |    2 
 14 files changed, 801 insertions(+), 799 deletions(-)

(Continue reading)

Permalink | Reply |

Greg KH | 23 Jan 01:02

Re: [PATCH] docs: move DMA-mapping.txt to Documentation/PCI/

On Thu, Jan 22, 2009 at 03:41:24PM -0800, Randy Dunlap wrote:
> From: Randy Dunlap <randy.dunlap <at> oracle.com>
> 
> Move DMA-mapping.txt to Documentation/PCI/.
> 
> DMA-mapping.txt was supposed to be moved from Documentation/ to
> Documentation/PCI/.  The 00-INDEX files in those two directories
> were updated, along with a few other text files, but the file
> itself somehow escaped being moved, so move it and update more
> text files and source files with its new location.
> 
> Signed-off-by: Randy Dunlap <randy.dunlap <at> oracle.com>
> cc:	Greg Kroah-Hartman <gregkh <at> suse.de> # for staging/

Acked-by: Greg Kroah-Hartman <gregkh <at> suse.de>

Permalink | Reply |

yogeshwar sonawane | 27 Jan 14:55

Re: Query related to pci_alloc_consistent()

Hi,

Continuing the discussion, I have some more query.

We are planning to design a PCIe based card which will function as an
accelerator.
Now, there can be more than 1 cards in a node. In some cases, i may
want to give same input data to both the cards.
I have stored my input data to one dma'ble buffer. Then i am asking
both the cards to do READ DMA to that buffer.

1) Can this be achievable & How ?

2) As we discussed, pci_alloc_consistent() may not be helpful. In case
of streaming mapping ,If I have allocated one kernel buffer(using
kmalloc or get_free_pages) and mapped it for card 1 using
pci_map_single, then can I map the
same buffer at same time for card 2 without any unmapping ?

Thanks,
Yogeshwar

On Fri, Jan 23, 2009 at 12:32 AM, Grant Grundler
<grundler <at> parisc-linux.org> wrote:
> On Thu, Jan 22, 2009 at 04:56:22PM +0530, yogeshwar sonawane wrote:
>> Hi all,
>>
>> If there are multiple PCIe cards of the same type, e.g. 3 or 4 network
>> controllers, in a node.
>> Now, if one allocates a buffer using pci_alloc_consistent() using

(Continue reading)

Permalink | Reply |

Bjorn Helgaas | 27 Jan 17:29

Re: Query related to pci_alloc_consistent()

On Tuesday 27 January 2009 06:55:43 am yogeshwar sonawane wrote:
> We are planning to design a PCIe based card which will function as an
> accelerator.
> Now, there can be more than 1 cards in a node. In some cases, i may
> want to give same input data to both the cards.
> I have stored my input data to one dma'ble buffer. Then i am asking
> both the cards to do READ DMA to that buffer.
> 
> 1) Can this be achievable & How ?

Sure.  In this case, you already have the buffer, so you don't want
pci_alloc_consistent() because that would allocate a new buffer.

You should call pci_map_single(), supplying the device and the buffer.
You can call pci_map_single() for as many devices as you want, using
the same buffer.  If the devices are just reading the buffer, there's
no synchronization between devices to worry about.

> 2) As we discussed, pci_alloc_consistent() may not be helpful. In case
> of streaming mapping ,If I have allocated one kernel buffer(using
> kmalloc or get_free_pages) and mapped it for card 1 using
> pci_map_single, then can I map the
> same buffer at same time for card 2 without any unmapping ?

Yes.  Just call pci_map_single() again using the same buffer but a
different device.  If the devices can write to the buffer, you're
responsible for synchronizing so one device doesn't overwrite another
device's data.  And of course, you have to worry about the lifetimes
of the mappings and the buffer itself.  Each pci_map_single() allocates
IOMMU resources, so be sure each call has a corresponding pci_unmap_single()

Re: Query related to pci_alloc_consistent()

On Tuesday 27 January 2009 06:55:43 am yogeshwar sonawane wrote:
> We are planning to design a PCIe based card which will function as an
> accelerator.
> Now, there can be more than 1 cards in a node. In some cases, i may
> want to give same input data to both the cards.
> I have stored my input data to one dma'ble buffer. Then i am asking
> both the cards to do READ DMA to that buffer.
> 
> 1) Can this be achievable & How ?

Sure.  In this case, you already have the buffer, so you don't want
pci_alloc_consistent() because that would allocate a new buffer.

You should call pci_map_single(), supplying the device and the buffer.
You can call pci_map_single() for as many devices as you want, using
the same buffer.  If the devices are just reading the buffer, there's
no synchronization between devices to worry about.

> 2) As we discussed, pci_alloc_consistent() may not be helpful. In case
> of streaming mapping ,If I have allocated one kernel buffer(using
> kmalloc or get_free_pages) and mapped it for card 1 using
> pci_map_single, then can I map the
> same buffer at same time for card 2 without any unmapping ?

Yes.  Just call pci_map_single() again using the same buffer but a
different device.  If the devices can write to the buffer, you're
responsible for synchronizing so one device doesn't overwrite another
device's data.  And of course, you have to worry about the lifetimes
of the mappings and the buffer itself.  Each pci_map_single() allocates
IOMMU resources, so be sure each call has a corresponding pci_unmap_single()
to deallocate them.  And make sure the buffer itself is deallocated
after all the mappings are gone.

Bjorn

> On Fri, Jan 23, 2009 at 12:32 AM, Grant Grundler
> <grundler <at> parisc-linux.org> wrote:
> > On Thu, Jan 22, 2009 at 04:56:22PM +0530, yogeshwar sonawane wrote:
> >> Hi all,
> >>
> >> If there are multiple PCIe cards of the same type, e.g. 3 or 4 network
> >> controllers, in a node.
> >> Now, if one allocates a buffer using pci_alloc_consistent() using
> >> pci_dev struct of one of the card.
> >>
> >> 1) Whether that buffer can be used by remaining cards also ?
> >> OR
> >> Whether DMA addresses returned by pci_alloc_consistent() can be used
> >> by all the cards explained in above situation ?
> >
> > What bjorn said.
> >
> >> 2) What will be the case for multiple platforms/architectures ?
> >
> > pci_alloc_consistent() should conform to Documentation/DMA-API.txt
> > and Documentation/DMA-mapping.txt. This is true for any platform or
> > architecture.
> >
> > BTW, the latter should be moved to Documentation/PCI....Jesse?
> >
> >
> >> 3) There can be multiple PCIe bridges giving multiple slots on some machines.
> >> What will happen, when above situation is considered in such machines ?
> >
> > What Bjorn said.
> >
> > hth,
> > grant
> >
>