On Thu, Apr 25, 2013 at 11:56:16PM +0300, Or Gerlitz wrote: > > Ah, this seems contrary to the IPoIB specification? Someone should > > probably talk about how sending from the wrong QPN is acceptable..
> AFAIK the IPoIB specification doesn't mandate the QPN of the sender I'd have to read it again very carefully.. However, checking the src QPN of every UD packet is the only way to detect if the packet was generated by the authentic kernel or from an unprivileged user space process, so there is a certainly importance in the value. But I don't follow why the send QPNs have to be sequential for IPoIB. It looks like this is being motivated by RSS and RSS QPNs are just being reused for TSS? > > As I said, that is ugly. 'TSS' that changes the on-the-wire packet > > is not TSS. It is just ganging QPs together. > > > > Allocating sequential TSS QPNs is an awful hack, what we really > > need is a way to force a UD QP's outgoing QPN. > > INDEED, but this must be supported by the HW. The patch set is already > supporting the case of HW the knows to do that forcing, quoting ---> > IB_DEVICE_UD_TSS which is set to indicate that the device supports "HW > TSS" which means that the HW is capable of over-riding the source UD > QPN present in sent IB datagram header (DTH) with the parent's QPN > <--- where over such HW the on-the-wire IPoIB header isn't touched. For the TSS case, I'd say just allocate normal QPs and provide something like ibv_override_ud_src_qpn(). This is very general and broadly useful for any application using UD QPs. > BUT for the sake of improving performance and being competitive with > tons of Linux Ethernet drivers that support TSS/MQ we still need IPoIB > to support MQ/TSS before such HW is introduced, and as such the chosen > solution was to use reserved fields of the wire header. You've lost me again, what reserved bits? If a new uverb is introduced the on-the-wire behaviour needs to be fully documented.. > How about we discuss RSS 1st? for RSS no wire change is introduced, > lets see if/how we can come to an agreement how the RSS related verbs > should look like and we'll take it from there to TSS. Well, to me, TSS is pretty simple. RSS is where things got really complicated.. As Sean said earlier, please think about a single QP, multiple RQ/SQ style API - that seems much more general to me and also could reasonably be defined for other transport types. For instance, someday supporting multiple RQ on a RC transport, with content-based steering, is a limited form of tag matching.. From a longer-term user space API design standpoint the concept seems to have more longevity. Also, I feel what happens inside the kernel is more flexable API wise, so dropping the uverbs component may also be something you want to look at. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
