Looking at the Dror's slides on slide 6 "Scalable Reliable
Connection" I
see that wire protocol is extended to send DST SRQ as part of a
header.
Receiver side then puts completion to appropriate CQ according this
field. Have you proposition address this? How? Who will put this
additional data on a wire (HW or libibverbs may be app)? Also I don't
see this in Dror's slide, but completion of local operation should be
demultiplexed to appropriate CQ too. WQE may contain additional field,
for instance, that will tell where to put a completion. Once again who
will do the demux in you proposition (HW, libiverbs or app)? The right
answer is most certainly HW in both cases so will Hermon support this?
Or may be you want to demultiplex everything inside libibvers? In this
case I want to see design of this (preferably with performance
analysis).
While I think the SRC design makes sense I also have concerns
regarding SSQ.
As Gleb has pointed out, if the hardware doesn't do the demux then
the application has to.
It sounds like there are two proposals to deal with this hardware
limitation in software (sigh).
1) Process A polls CQ, if WQE belongs to Process B, Process A will
drop the WQE in a shared memory region that Process B will poll.
So we end up re-implementing shared memory completion semantics
all over again for SSQ. I am concerned that there is both a latency
hit (on average) and a memory scaling issue in that multiple QPs will
now be replaced with shared memory completion fifos and a single SSQ QP.
2) Process A peeks CQ, if WQE belongs to Process B, it doesn't
process it
This has very bad implications for real applications, having to
context switch to receive a WQE is bad
In my opinion the demux belongs in the hardware, otherwise we end up
complicating an already complicated code base to support a feature
which unless I am missing something will have no benefit to real
applications.
- Galen
--
Gleb.
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/
openib-general
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
