On Tue, 2006-05-16 at 17:41 +0300, Michael S. Tsirkin wrote:
> Quoting r. Steve Wise <[EMAIL PROTECTED]>:
> > Subject: PATCH] enhancement to rdma_bw and rdma_lat to utilize the RDMA CM
> >
> > I don't know who maintains src/userspace/perftest, but here is a patch
> > set that enables rdma_bw and rdma_lat to use the RDMA_CM with the
> > addition of the -c or --cma flag.
> >
>
> I'm worried that this makes the program too big. Maybe this should be
> another test rather than an option?
>
ok. You want it as a separate pair of programs?
> > The rkey/addr info is exchanged in the private data, and SEND/RECV's are
> > used
> > to sync the client/server before and after execution.
>
> Do we really need SEND/RECV messages for this?
> I think I get completion with error once the remote side has disconnected. No?
>
perhaps. I just thought it was cleaner to synch up at the end. Just
like the non-cma version does over the TCP socket (see
pp_client_exch_dest() / pp_server_exch_dest() at the end of the test).
> > Also, I added -P or --poll to rdma_bw to allow blocking for completion
> > events when none are ready (if you omit -P, it will block when no
> > completion is available, otherwise it will spin).
>
> Needs to be a separate patch.
ok.
>
> > Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
>
>
> > Index: rdma_lat.c
> > ===================================================================
> > --- rdma_lat.c (revision 7050)
> > +++ rdma_lat.c (working copy)
> > @@ -53,6 +53,7 @@
> > #include <time.h>
> >
> > #include <infiniband/verbs.h>
> > +#include <rdma/rdma_cma.h>
> >
> > #include "get_clock.h"
> >
> > @@ -71,7 +72,8 @@
> > struct ibv_context *context;
> > struct ibv_pd *pd;
> > struct ibv_mr *mr;
> > - struct ibv_cq *cq;
> > + struct ibv_cq *scq;
> > + struct ibv_cq *rcq;
>
> Why are you adding another CQ?
>
It makes waiting for a recv completion easier since you won't get a send
completion when the CQ is only for receives...
> > struct ibv_qp *qp;
> > void *buf;
> > volatile char *post_buf;
> > @@ -80,6 +82,7 @@
> > int tx_depth;
> > struct ibv_sge list;
> > struct ibv_send_wr wr;
> > + struct rdma_cm_id *cm_id;
> > };
> >
> > struct pingpong_dest {
> > @@ -323,16 +326,22 @@
> > return NULL;
> > }
> >
> > - ctx->cq = ibv_create_cq(ctx->context, tx_depth, NULL, NULL, 0);
> > - if (!ctx->cq) {
> > + ctx->rcq = ibv_create_cq(ctx->context, 1, NULL, NULL, 0);
> > + if (!ctx->rcq) {
> > fprintf(stderr, "Couldn't create CQ\n");
> > return NULL;
> > }
>
> CQ of depth 1?
>
Yes, there is only ever one outstanding send/recv exchange...
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
