Hi, Le jeudi 11 octobre 2012 à 13:44 -0700, Roland Dreier a écrit : > On Thu, Oct 11, 2012 at 8:34 AM, Animesh K Trivedi1 <[email protected]> wrote: > > > > During memory memory registration, userspace buffers also go through same > > API calls (dma_map_sg_attrs(...)). > > What I am confused about why no such synchronization primitives are > > required in userspace before accessing > > an RDMA data buffer just after when an incoming write/recv (DMA on it) is > > finished? Who guarantees data > > freshness? > > No one has really ever tried to deal with the issue of userspace RDMA on > a cache-incoherent architecture. Basically if you try the current stack, the > in-kernel users (IPoIB etc) should be OK but libibverbs etc. will be > completely > broken. >
With the current ARMv7 Cortex-A9 / Cortex-A15 MPCore and the upcoming ARM 64 bits architecture eg ARMv8 aka Aarch64, one might want in the near future use RDMA (InfiniBand/RoCE) with them to create highly parallel system with low-power consumption. But this question and some reading about ARM memory management makes me feel pretty unsure of the ability to use RDMA (InfiniBand) on ARM. In this article: "ARM, DMA, and memory management" http://lwn.net/Articles/440221/"; it is said that a memory page must not be mapped multiple time with different caching attributes. This article take the point of Linaro's developers who want to upload texture to the GPU without holding them in caches. This behavior might also be applicable to RDMA as well: writing to a memory zone to be either local IBV_WR_SEND, local IBV_WR_RDMA_WRITE or remote IBV_WR_RDMA_READ, there's probably no need to keep it in cache. You could also read this other article "CMA [contiguous memory allocator] and ARM" http://lwn.net/Articles/450286/ and "ARM's multiply-mapped memory mess" http://lwn.net/Articles/409689/ After reading this, and not being an ARM expert, I'm asking myself about a possible RDMA (InfiniBand) support on ARM. Regards. -- Yann Droneaud OPTEYA -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
