On Wed, Mar 30, 2005 at 10:09:24AM -0500, Hal Rosenstock wrote:
> On Mon, 2005-03-28 at 20:03, Libor Michalek wrote:
> > I haven't looked closely at the code yet, but I did try it out
> > with SDP/AIO on a pair of x86 systems with Tavors and a pair of
> > x86_64 systems with Arbels. With a small change to core/fmr_pool.c
> > and enabling pool creation in SDP it worked as expected. Here are
> > throughput results:
> >
> > x86 x86_64
> > -------- --------
> > SDP sync 610 MB/s 710 MB/s
> > SDP async (hit) 740 MB/s 910 MB/s
> > SDP async (miss) 590 MB/s 910 MB/s
> >
> > For sync sockets I used 81600 byte buffers. For async socket I kept
> > 20 96K buffers in flight. For the FMR pool cache hit async results I
> > used only 20 different buffers. For the FMR pool cache miss async
> > results I used 1000 different buffers, of which only 20 were in flight
> > at a time.
>
> Any idea why hit/miss make such a difference on x86 and not x86_64 ?
Not sure, but the difference is also Tavor vs. Arbel on those two
pairs of systems.
> Also, is all the code for this now checked available ?
Almost, for SDP I need to check in this patch, now that the FMR code
has been checked in.
-Libor
Index: sdp_conn.c
===================================================================
--- sdp_conn.c (revision 2072)
+++ sdp_conn.c (working copy)
@@ -1774,14 +1774,12 @@
/*
* create SDP memory pool
*/
-#ifdef _FMR_SUPPORT
hca->fmr_pool = ib_create_fmr_pool(hca->pd, &fmr_param_s);
if (IS_ERR(hca->fmr_pool)) {
sdp_warn("Error <%ld> creating HCA <%s> fast memory pool",
PTR_ERR(hca->fmr_pool), device->name);
goto error;
}
-#endif
/*
* port allocation
*/
@@ -1828,10 +1826,9 @@
kfree(port);
}
-#ifdef _FMR_SUPPORT
if (!IS_ERR(hca->fmr_pool))
(void)ib_destroy_fmr_pool(hca->fmr_pool);
-#endif
+
if (!IS_ERR(hca->mem_h))
(void)ib_dereg_mr(hca->mem_h);
@@ -1864,10 +1861,9 @@
kfree(port);
}
-#ifdef _FMR_SUPPORT
if (!IS_ERR(hca->fmr_pool))
(void)ib_destroy_fmr_pool(hca->fmr_pool);
-#endif
+
if (!IS_ERR(hca->mem_h))
(void)ib_dereg_mr(hca->mem_h);
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
