Gilles Chanteperdrix wrote:
> Philippe Gerum wrote:
>> On Wed, 2011-01-12 at 16:57 -0500, Herrera-Bendezu, Luis wrote:
>>>> -----Original Message-----
>>>> From: Gilles Chanteperdrix [mailto:[email protected]]
>>>> Sent: Wednesday, January 12, 2011 3:37 PM
>>>> To: Herrera-Bendezu, Luis
>>>> Cc: [email protected]
>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA memory
>>>>
>>>> Herrera-Bendezu, Luis wrote:
>>>>>> From: Gilles Chanteperdrix [mailto:[email protected]]
>>>>>> Sent: Wednesday, January 12, 2011 11:26 AM
>>>>>> To: Herrera-Bendezu, Luis
>>>>>> Cc: [email protected]
>>>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA
>>>>>> memory
>>>>>>
>>>>>> Herrera-Bendezu, Luis wrote:
>>>>>>>> From: Gilles Chanteperdrix [mailto:[email protected]]
>>>>>>>> Sent: Wednesday, January 12, 2011 8:35 AM
>>>>>>>> To: Herrera-Bendezu, Luis
>>>>>>>> Cc: [email protected]
>>>>>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA
>>>>>>>> memory
>>>>>>>>
>>>>>>>> Herrera-Bendezu, Luis wrote:
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Gilles Chanteperdrix [mailto:[email protected]]
>>>>>>>>>> Sent: Tuesday, January 11, 2011 6:28 PM
>>>>>>>>>> To: Herrera-Bendezu, Luis
>>>>>>>>>> Cc: [email protected]
>>>>>>>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA
>>>>>>>>>> memory
>>>>>>>>>>
>>>>>>>>>> Gilles Chanteperdrix wrote:
>>>>>>>>>>> Herrera-Bendezu, Luis wrote:
>>>>>>>>>>>> On 01/05/2011 5:29 PM Gilles Chanteperdrix wrote:
>>>>>>>>>>>>> Steven A. Falco wrote:
>>>>>>>>>>>>>> On 01/05/2011 04:33 PM, Gilles Chanteperdrix wrote:
>>>>>>>>>>>>> Ok. Could you try to do the same operation with the native API?
>>>>>>>>>>>>> You just
>>>>>>>>>>>>> have to pass H_SHARED | H_DMA | H_NONCACHED as flags to
>>>>>>>>>>>>> rt_heap_create
>>>>>>>>>>>>> to get the same effect as pci_dma_alloc_coherent.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Just to see if the error lies in RTDM implementation or in Xenomai
>>>>>>>>>>>>> generic code.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>> (...)
>>>>>>>>>>> What about the other thing I asked you to test?
>>>>>>>>>>>
>>>>>>>>>> Ping? Any news about this test?
>>>>>>>>> rt_heap_create() with flags H_SHARED | H_DMA | H_NONCACHED report
>>>>>>>>> -EINVAL.
>>>>>>>>> Documentation indicates that H_NONCACHE is not compatible with H_DMA.
>>>>>>>> Right, supporting H_NONCACHED | H_DMA would mean that we would have to
>>>>>>>> use kmalloc/get_free_pages then establish a non-cached mapping in
>>>>>>>> kernel-space with vm_map_ram.
>>>>>>>>
>>>>>>>> Could you try with H_NONCACHED, but without H_DMA? Of course, the
>>>>>>>> mapping can not be used for DMA, as it will probably not be physically
>>>>>>>> contiguous, but at least you can try whether accessing in user-space a
>>>>>>>> non-cached mapping works.
>>>>>>> It does work but unless an external unit writes to heap memory it would
>>>>>>> be difficult to verify that the non-cached memory actually works, i.e.
>>>>>>> access from user space gets expected value(s).
>>>>>> I am quite confident this works.
>>>>>>
>>>>>>>> In any case, we see a defect in the RTDM interface here: we can not ask
>>>>>>>> the mapping to be mapped non-cacheable, which somewhat defeats the
>>>>>>>> purpose of pci_alloc_consistent. I do not know enough the powerpc
>>>>>>>> architecture to know whether this could be the cause of your issue (the
>>>>>>>> same physical area mapped twice, once cached, once non-cached).
>>>>>>>> However,
>>>>>>>> on the ARM architecture, for instance, it is bad.
>>>>>>>>
>>>>>>> Let me summarize the ideas discussed so far to allocate consistent
>>>>>>> memory
>>>>>>> suitable for DMA and corresponding mapping to user space:
>>>>>>> * rt_heap cannot be created with both H_NONCACHED and H_DMA. But it is
>>>>>>> automatically mapped to user space with rt_heap_bind().
>>>>>>>
>>>>>>> A solution can be to allocate heap with H_SHARED | H_DMA, somehow
>>>>>>> get bus address of single block of memory (rt_heap { sba } ?) to
>>>>>>> used for DMA operations.
>>>>>> The test with rt_heap was just a test to have a comparison between
>>>>>> xnheap code and rtdm_mmap. But we may indeed want to fix mmappable
>>>>>> xnheaps later on.
>>>>>>
>>>>>>> * RTDM interface rtdm_mmap/unmap cannot handle non-cacheable memory.
>>>>>>>
>>>>>>> A solution can be to allocate GFP_DMA memory with kmalloc(), use
>>>>>>> rtdm_mmap to map to user space. Programmatically make memory
>>>>>>> consistent before/after DMA transfer to/from external unit, e.g.
>>>>>>> dma_sync_single_for_device()/dma_sync_single_for_cpu(). This is
>>>>>>> similar to what streaming DMA mappings do.
>>>>>>>
>>>>>>> * Ideally, it should be possible to take memory allocated from
>>>>>>> dma_alloc_coherent()/pci_alloc_consistent() and mapped it to
>>>>>>> user space using rtdm_mmap().
>>>>>> To check that the non-cacheable mapping is indeed the problem, here is a
>>>>>> patch which adds the possibility to add non-cacheable mappings:
>>>>>>
>>>>>> diff --git a/ksrc/skins/rtdm/drvlib.c b/ksrc/skins/rtdm/drvlib.c
>>>>>> index 3495e63..ec0ddae 100644
>>>>>> --- a/ksrc/skins/rtdm/drvlib.c
>>>>>> +++ b/ksrc/skins/rtdm/drvlib.c
>>>>>> @@ -1791,6 +1791,7 @@ void rtdm_nrtsig_pend(rtdm_nrtsig_t *nrt_sig);
>>>>>>
>>>>>> #if defined(CONFIG_XENO_OPT_PERVASIVE) || defined(DOXYGEN_CPP)
>>>>>> struct rtdm_mmap_data {
>>>>>> + int noncached;
>>>>>> void *src_vaddr;
>>>>>> phys_addr_t src_paddr;
>>>>>> struct vm_operations_struct *vm_ops;
>>>>>> @@ -1815,6 +1816,9 @@ static int rtdm_mmap_buffer(struct file *filp,
>>>>>> struct vm_area_struct *vma)
>>>>>> maddr = vma->vm_start;
>>>>>> size = vma->vm_end - vma->vm_start;
>>>>>>
>>>>>> + if (mmap_data->noncached)
>>>>>> + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
>>>>>> +
>>>>>> #ifdef CONFIG_MMU
>>>>>> /* Catch vmalloc memory (vaddr is 0 for I/O mapping) */
>>>>>> if ((vaddr >= VMALLOC_START) && (vaddr < VMALLOC_END)) {
>>>>>> @@ -1975,6 +1979,24 @@ int rtdm_mmap_to_user(rtdm_user_info_t *user_info,
>>>>>> void *vm_private_data)
>>>>>> {
>>>>>> struct rtdm_mmap_data mmap_data = {
>>>>>> + .noncached = 0,
>>>>>> + .src_vaddr = src_addr,
>>>>>> + .src_paddr = 0,
>>>>>> + .vm_ops = vm_ops,
>>>>>> + .vm_private_data = vm_private_data
>>>>>> + };
>>>>>> +
>>>>>> + return rtdm_do_mmap(user_info, &mmap_data, len, prot, pptr);
>>>>>> +}
>>>>>> +
>>>>>> +int rtdm_mmap_noncached_to_user(rtdm_user_info_t *user_info,
>>>>>> + void *src_addr, size_t len,
>>>>>> + int prot, void **pptr,
>>>>>> + struct vm_operations_struct *vm_ops,
>>>>>> + void *vm_private_data)
>>>>>> +{
>>>>>> + struct rtdm_mmap_data mmap_data = {
>>>>>> + .noncached = 1,
>>>>>> .src_vaddr = src_addr,
>>>>>> .src_paddr = 0,
>>>>>> .vm_ops = vm_ops,
>>>>>> @@ -2043,6 +2065,7 @@ int rtdm_iomap_to_user(rtdm_user_info_t *user_info,
>>>>>> void *vm_private_data)
>>>>>> {
>>>>>> struct rtdm_mmap_data mmap_data = {
>>>>>> + .noncached = 0,
>>>>>> .src_vaddr = NULL,
>>>>>> .src_paddr = src_addr,
>>>>>> .vm_ops = vm_ops,
>>>>>>
>>>>>> Simply call rtdm_mmap_noncached_to_user instead of rtdm_mmap_to_user.
>>>>>>
>>>>> Get same kernel panic error as before with this patch. I need to look
>>>>> closer
>>>>> at what are the memory allocation functions used by pci_ and dma_.
>>>> Could you try rtdm_iomap_to_user (but beware, pass the physical address
>>>> returned by pointer by pci_alloc_consistent)? virt_to_page, or __pa will
>>>> not work with vmalloc/ioremap addresses. And pci_alloc_consistent
>>>> probably returns a vmalloc mapping due to the fact that the mapping
>>>> needs to be non-cacheable, which will happen if your powerpc does not
>>>> support cache snooping, and define CONFIG_NOT_COHERENT_CACHE.
>>>>
>>> Your suggestion works. Can you highlight why it fails when using
>>> rtdm_mmap_to_user()?
>> Because unlike the former, it does not get the right platform-dependent
>> protection bits for the vma from phys_mem_access_prot() for this kind of
>> memory.
>
> There is another issue: the memory returned by pci_alloc_consistent
> should be handled as vmalloc memory, but it is allocated in a separated
> area, the CONSISTENT_BASE, CONSISTENT_END area. So the test:
>
> if ((vaddr >= VMALLOC_START) && (vaddr < VMALLOC_END))
>
> returns false, and the memory is handled as if it were kmalloced memory.
Replacing this test with if (!virt_addr_valid(vaddr)) should do the
trick. Remains to see whethe this macro existed in older versions.
--
Gilles.
_______________________________________________
Xenomai-help mailing list
[email protected]
https://mail.gna.org/listinfo/xenomai-help
