On Thu, 2006-09-14 at 09:36 +0300, Or Gerlitz wrote: > Ralph Campbell wrote: > > On Wed, 2006-09-13 at 12:00 +0300, Or Gerlitz wrote: > >> Ralph Campbell wrote: > > > Well, the other parts of the kernel might not need a kernel virtual > > address but the ib_ipath driver still does. > > So you agree there is a need to kmap/kunamp pages which the user wants > to use with IB and are not mapped into the kernel virt address space?
Yes, I agree for systems which have high memory pages. > > I don't understand what you are talking about. There is an IB > > wire protocol for RDMA, SEND, etc. That doesn't change depending > > on the HCA. > > The InfiniPath HCA has a ring buffer of receive buffers and all > > incoming IB packets are DMA'ed into one of these buffers. > > The ib_ipath software driver examines the packet and > > copies it to the appropriate address. For a packet received with > > a RC_RDMA_WRITE_FIRST, the RKEY and IB address are used to convert > > that into a kernel virtual address and the data is copied. > > The same happens for RC_SEND_FIRST but the KV address comes from > > the LKEY and address in the work request posted by ib_post_recv(). > > OK, this make sense. > > Lets see if i follow: you say that the Infinipath HCA is RX DMA-able but > it does RX DMA to the ipath driver private RX buffers and then the > driver copies from these buffers to the user buffer. My guess is that > you do that to support both recv and rdma read on this QP since if you > would only need to support recv you can have the hca dma-ing to the user > posted rx buffer. You mostly understand. The hardware doesn't have separate receive queues for each QP. All packets go into a single (or at most 4 currently) receive queues and the driver figures out which QP, RDMA memory region, etc. to copy them to. _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
