I found the bug. The ammasso driver was passing in the pointer c2_dev->ibdev.dma_device to dma_map_single(). However the driver hadn't yet registered with the ib core, so that field is zero. The dma map code uses bounce buffers if a map is done with a NULL device ptr passed in...
The simple fix is to pass in the correct pointer, &c2dev->pcidev->dev. But I guess I should still change this all to use dma_alloc_coherent()... Roland, Can we get this fix into 2.6.19? Steve. On Fri, 2006-10-27 at 11:15 -0700, Roland Dreier wrote: > > I must be misusing dma_map_single(). What I'm doing is allocating a > > verb message reply queue for the adapter to DMA verb replies into. It > > never gets unmapped. I kmalloc() it, then map it. I could use > > dma_alloc_coherent() or something, and maybe that's what I need to do? > > Yeah, if you want to leave something mapped and have the device DMA > into it, and the CPU look into the buffer too, then you need > consistent/coherent memory -- either pci_alloc_consistent() or > dma_alloc_coherent(). The dma_ variant is slightly better because you > can pass in a GFP_ mask rather than having the kernel pick GFP_ATOMIC > for you. > > > You're saying I must unmap it before the data is valid (cuz of the > > bounce buffering). If that's true, then how in sam hill does user mode > > RDMA work since the user's memory isn't unmapped before the user looks > > like memory that is the target of RDMA???? The uverbs code calls > > dma_map_sg() which is roughly the same as dma_map_single, eh? > > It's a good point. We're kind of counting on the IOMMU situation not > being too wacky, and the device being able to DMA to arbitrary > addresses. So swiotlb won't work in this case actually -- but we > assume any RDMA device can do 64-bit DMA so it doesn't hurt us in > practice. But yes, DMA to userspace is slightly risky and won't work > everywhere. > > - R. _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
