Philippe Gerum wrote:
> On Wed, 2011-01-12 at 23:14 +0100, Gilles Chanteperdrix wrote:
>> Philippe Gerum wrote:
>>> On Wed, 2011-01-12 at 16:57 -0500, Herrera-Bendezu, Luis wrote:
>>>>> -----Original Message-----
>>>>> From: Gilles Chanteperdrix [mailto:[email protected]]
>>>>> Sent: Wednesday, January 12, 2011 3:37 PM
>>>>> To: Herrera-Bendezu, Luis
>>>>> Cc: [email protected]
>>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA
>>>>> memory
>>>>>
>>>>> Herrera-Bendezu, Luis wrote:
>>>>>>> From: Gilles Chanteperdrix [mailto:[email protected]]
>>>>>>> Sent: Wednesday, January 12, 2011 11:26 AM
>>>>>>> To: Herrera-Bendezu, Luis
>>>>>>> Cc: [email protected]
>>>>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA
>>>>>>> memory
>>>>>>>
>>>>>>> Herrera-Bendezu, Luis wrote:
>>>>>>>>> From: Gilles Chanteperdrix [mailto:[email protected]]
>>>>>>>>> Sent: Wednesday, January 12, 2011 8:35 AM
>>>>>>>>> To: Herrera-Bendezu, Luis
>>>>>>>>> Cc: [email protected]
>>>>>>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA
>>>>>>>>> memory
>>>>>>>>>
>>>>>>>>> Herrera-Bendezu, Luis wrote:
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Gilles Chanteperdrix [mailto:[email protected]]
>>>>>>>>>>> Sent: Tuesday, January 11, 2011 6:28 PM
>>>>>>>>>>> To: Herrera-Bendezu, Luis
>>>>>>>>>>> Cc: [email protected]
>>>>>>>>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to
>>>>>>>>>>> DMA memory
>>>>>>>>>>>
>>>>>>>>>>> Gilles Chanteperdrix wrote:
>>>>>>>>>>>> Herrera-Bendezu, Luis wrote:
>>>>>>>>>>>>> On 01/05/2011 5:29 PM Gilles Chanteperdrix wrote:
>>>>>>>>>>>>>> Steven A. Falco wrote:
>>>>>>>>>>>>>>> On 01/05/2011 04:33 PM, Gilles Chanteperdrix wrote:
>>>>>>>>>>>>>> Ok. Could you try to do the same operation with the native API?
>>>>>>>>>>>>>> You just
>>>>>>>>>>>>>> have to pass H_SHARED | H_DMA | H_NONCACHED as flags to
>>>>>>>>>>>>>> rt_heap_create
>>>>>>>>>>>>>> to get the same effect as pci_dma_alloc_coherent.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Just to see if the error lies in RTDM implementation or in
>>>>>>>>>>>>>> Xenomai
>>>>>>>>>>>>>> generic code.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>> (...)
>>>>>>>>>>>> What about the other thing I asked you to test?
>>>>>>>>>>>>
>>>>>>>>>>> Ping? Any news about this test?
>>>>>>>>>> rt_heap_create() with flags H_SHARED | H_DMA | H_NONCACHED report
>>>>>>>>>> -EINVAL.
>>>>>>>>>> Documentation indicates that H_NONCACHE is not compatible with H_DMA.
>>>>>>>>> Right, supporting H_NONCACHED | H_DMA would mean that we would have to
>>>>>>>>> use kmalloc/get_free_pages then establish a non-cached mapping in
>>>>>>>>> kernel-space with vm_map_ram.
>>>>>>>>>
>>>>>>>>> Could you try with H_NONCACHED, but without H_DMA? Of course, the
>>>>>>>>> mapping can not be used for DMA, as it will probably not be physically
>>>>>>>>> contiguous, but at least you can try whether accessing in user-space a
>>>>>>>>> non-cached mapping works.
>>>>>>>> It does work but unless an external unit writes to heap memory it would
>>>>>>>> be difficult to verify that the non-cached memory actually works, i.e.
>>>>>>>> access from user space gets expected value(s).
>>>>>>> I am quite confident this works.
>>>>>>>
>>>>>>>>> In any case, we see a defect in the RTDM interface here: we can not
>>>>>>>>> ask
>>>>>>>>> the mapping to be mapped non-cacheable, which somewhat defeats the
>>>>>>>>> purpose of pci_alloc_consistent. I do not know enough the powerpc
>>>>>>>>> architecture to know whether this could be the cause of your issue
>>>>>>>>> (the
>>>>>>>>> same physical area mapped twice, once cached, once non-cached).
>>>>>>>>> However,
>>>>>>>>> on the ARM architecture, for instance, it is bad.
>>>>>>>>>
>>>>>>>> Let me summarize the ideas discussed so far to allocate consistent
>>>>>>>> memory
>>>>>>>> suitable for DMA and corresponding mapping to user space:
>>>>>>>> * rt_heap cannot be created with both H_NONCACHED and H_DMA. But it is
>>>>>>>> automatically mapped to user space with rt_heap_bind().
>>>>>>>>
>>>>>>>> A solution can be to allocate heap with H_SHARED | H_DMA, somehow
>>>>>>>> get bus address of single block of memory (rt_heap { sba } ?) to
>>>>>>>> used for DMA operations.
>>>>>>> The test with rt_heap was just a test to have a comparison between
>>>>>>> xnheap code and rtdm_mmap. But we may indeed want to fix mmappable
>>>>>>> xnheaps later on.
>>>>>>>
>>>>>>>> * RTDM interface rtdm_mmap/unmap cannot handle non-cacheable memory.
>>>>>>>>
>>>>>>>> A solution can be to allocate GFP_DMA memory with kmalloc(), use
>>>>>>>> rtdm_mmap to map to user space. Programmatically make memory
>>>>>>>> consistent before/after DMA transfer to/from external unit, e.g.
>>>>>>>> dma_sync_single_for_device()/dma_sync_single_for_cpu(). This is
>>>>>>>> similar to what streaming DMA mappings do.
>>>>>>>>
>>>>>>>> * Ideally, it should be possible to take memory allocated from
>>>>>>>> dma_alloc_coherent()/pci_alloc_consistent() and mapped it to
>>>>>>>> user space using rtdm_mmap().
>>>>>>> To check that the non-cacheable mapping is indeed the problem, here is a
>>>>>>> patch which adds the possibility to add non-cacheable mappings:
>>>>>>>
>>>>>>> diff --git a/ksrc/skins/rtdm/drvlib.c b/ksrc/skins/rtdm/drvlib.c
>>>>>>> index 3495e63..ec0ddae 100644
>>>>>>> --- a/ksrc/skins/rtdm/drvlib.c
>>>>>>> +++ b/ksrc/skins/rtdm/drvlib.c
>>>>>>> @@ -1791,6 +1791,7 @@ void rtdm_nrtsig_pend(rtdm_nrtsig_t *nrt_sig);
>>>>>>>
>>>>>>> #if defined(CONFIG_XENO_OPT_PERVASIVE) || defined(DOXYGEN_CPP)
>>>>>>> struct rtdm_mmap_data {
>>>>>>> + int noncached;
>>>>>>> void *src_vaddr;
>>>>>>> phys_addr_t src_paddr;
>>>>>>> struct vm_operations_struct *vm_ops;
>>>>>>> @@ -1815,6 +1816,9 @@ static int rtdm_mmap_buffer(struct file *filp,
>>>>>>> struct vm_area_struct *vma)
>>>>>>> maddr = vma->vm_start;
>>>>>>> size = vma->vm_end - vma->vm_start;
>>>>>>>
>>>>>>> + if (mmap_data->noncached)
>>>>>>> + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
>>>>>>> +
>>>>>>> #ifdef CONFIG_MMU
>>>>>>> /* Catch vmalloc memory (vaddr is 0 for I/O mapping) */
>>>>>>> if ((vaddr >= VMALLOC_START) && (vaddr < VMALLOC_END)) {
>>>>>>> @@ -1975,6 +1979,24 @@ int rtdm_mmap_to_user(rtdm_user_info_t
>>>>>>> *user_info,
>>>>>>> void *vm_private_data)
>>>>>>> {
>>>>>>> struct rtdm_mmap_data mmap_data = {
>>>>>>> + .noncached = 0,
>>>>>>> + .src_vaddr = src_addr,
>>>>>>> + .src_paddr = 0,
>>>>>>> + .vm_ops = vm_ops,
>>>>>>> + .vm_private_data = vm_private_data
>>>>>>> + };
>>>>>>> +
>>>>>>> + return rtdm_do_mmap(user_info, &mmap_data, len, prot, pptr);
>>>>>>> +}
>>>>>>> +
>>>>>>> +int rtdm_mmap_noncached_to_user(rtdm_user_info_t *user_info,
>>>>>>> + void *src_addr, size_t len,
>>>>>>> + int prot, void **pptr,
>>>>>>> + struct vm_operations_struct *vm_ops,
>>>>>>> + void *vm_private_data)
>>>>>>> +{
>>>>>>> + struct rtdm_mmap_data mmap_data = {
>>>>>>> + .noncached = 1,
>>>>>>> .src_vaddr = src_addr,
>>>>>>> .src_paddr = 0,
>>>>>>> .vm_ops = vm_ops,
>>>>>>> @@ -2043,6 +2065,7 @@ int rtdm_iomap_to_user(rtdm_user_info_t
>>>>>>> *user_info,
>>>>>>> void *vm_private_data)
>>>>>>> {
>>>>>>> struct rtdm_mmap_data mmap_data = {
>>>>>>> + .noncached = 0,
>>>>>>> .src_vaddr = NULL,
>>>>>>> .src_paddr = src_addr,
>>>>>>> .vm_ops = vm_ops,
>>>>>>>
>>>>>>> Simply call rtdm_mmap_noncached_to_user instead of rtdm_mmap_to_user.
>>>>>>>
>>>>>> Get same kernel panic error as before with this patch. I need to look
>>>>>> closer
>>>>>> at what are the memory allocation functions used by pci_ and dma_.
>>>>> Could you try rtdm_iomap_to_user (but beware, pass the physical address
>>>>> returned by pointer by pci_alloc_consistent)? virt_to_page, or __pa will
>>>>> not work with vmalloc/ioremap addresses. And pci_alloc_consistent
>>>>> probably returns a vmalloc mapping due to the fact that the mapping
>>>>> needs to be non-cacheable, which will happen if your powerpc does not
>>>>> support cache snooping, and define CONFIG_NOT_COHERENT_CACHE.
>>>>>
>>>> Your suggestion works. Can you highlight why it fails when using
>>>> rtdm_mmap_to_user()?
>>> Because unlike the former, it does not get the right platform-dependent
>>> protection bits for the vma from phys_mem_access_prot() for this kind of
>>> memory.
>> There is another issue: the memory returned by pci_alloc_consistent
>> should be handled as vmalloc memory, but it is allocated in a separated
>> area, the CONSISTENT_BASE, CONSISTENT_END area. So the test:
>>
>> if ((vaddr >= VMALLOC_START) && (vaddr < VMALLOC_END))
>>
>> returns false, and the memory is handled as if it were kmalloced memory.
>>
>
> This is the reason why calling rtdm_mmap_to_user for mapping coherent
> memory is not appropriate in the first place, because as the doc states,
> it is for mapping kernel memory pages, but DMA/PCI coherent memory may
> have platform-dependent specifics which make it a different kind of
> beast. Perhaps the documentation should stress this aspect, so that
> rtdm_iomap_to_user() is considered instead.
Yes, OK, I was trying to find a way to get rtdm_mmap_to_user working in
that case. But that is kind of useless, since rtdm_iomap_to_user is
working anyway.
We still have the issue with noncacheable mappings with RTDM though,
does adding two new services:
rtdm_mmap_noncacheable_to_user() and rtdm_iomap_noncacheable_to_user()
look reasonable, or is there a better alternative?
>
--
Gilles.
_______________________________________________
Xenomai-help mailing list
[email protected]
https://mail.gna.org/listinfo/xenomai-help
