We are running into an unexpected problem while registering memory for IB operation. Ibv_reg_mr() call return EFAULT - bad address.
The pointer passed into ibv_reg_mr call is mmap back from memory allocated in our kernel driver. Our test code is able to use this mapped back virtual address to peek and poke without issue; the size is expected. This memory is a fixed size FIFO buffer designed to be used as our FPGA device buffer and IB transport buffer. There is host software to manage buffer usage. If we just malloc memory from app and pass it into ibv_reg_mr call, memory register call return ok. It seems like for some reason, ibv_reg_mr doesn't like the pointer we passed in. Here is how we allocate and map the buffer inside the driver, . 4M memory is created and allocated using the kmem_cache_create, kmem_cache_alloc call during driver initialization. . When mmap is called from user application at run-time, remap_pfn_range is called to map the buffer into the process's virtual address space. page_to_pfn(virt_to_page(buffer) is used to convert kernel virtual address into page and pfn. We tried to change flags during allocation or mapping, but none of them work. Any suggestion how to solve this problem? Here is version information of our test chassis, 64-bit Suse Linux: 3.0.35-rt58-0.5-rt OFED Info: MLNX_OFED_LINUX-1.5.3-3.1.0 (OFED-1.5.3-3.1.0) Regards, Hank Liu -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
