Re: [OMPI devel] OpenIB BTL and SRQs

2007-07-13 Thread Don Kerr
Jeff Squyres wrote: On Jul 12, 2007, at 1:18 PM, Don Kerr wrote: - So if you want to simply eliminate the flow control, choose M high enough (or just a total number of receive buffers to post to the SRQ) that you won't ever run out of resources and you should see some speedup from lack

Re: [OMPI devel] OpenIB BTL and SRQs

2007-07-12 Thread Jeff Squyres
On Jul 12, 2007, at 1:18 PM, Don Kerr wrote: - So if you want to simply eliminate the flow control, choose M high enough (or just a total number of receive buffers to post to the SRQ) that you won't ever run out of resources and you should see some speedup from lack of flow control. This

Re: [OMPI devel] OpenIB BTL and SRQs

2007-07-12 Thread Don Kerr
Jeff Squyres wrote: There's a few benefits: - Remember that you post a big pool of buffers instead of num_peers individual sets of receive buffers. Hence, if you post M buffers for each of N peers, each peer -- due to flow control -- can only have M outstanding sends at a time. So if

Re: [OMPI devel] OpenIB BTL and SRQs

2007-07-12 Thread Don Kerr
Interesting. So with SRQs there is no flow control, I am guessing the btl sets some reasonable default but essentially is relying on the user to adjust other parameters so the buffers are not over run. And yes Galen I would like to read your paper. Jeff Squyres wrote: There's a few

Re: [OMPI devel] OpenIB BTL and SRQs

2007-07-12 Thread Jeff Squyres
There's a few benefits: - Remember that you post a big pool of buffers instead of num_peers individual sets of receive buffers. Hence, if you post M buffers for each of N peers, each peer -- due to flow control -- can only have M outstanding sends at a time. So if you have apps sending

[OMPI devel] OpenIB BTL and SRQs

2007-07-12 Thread Don Kerr
Through mca parameters one can select the use of shared receive queues in the openib btl, other than having fewer queues I am wondering what are the benefits of using this option. Can anyone eleborate on using them vs the default?