If we pin the thread to 'a' CPU(say CPU-X) then can we pass on a hint to the BE
driver to process the response completion on the same 'CPU-X' ? This way the
CPU-cache will be utilized efficiently. I don't know how the IB driver etc
works. So can't contribute in terms of code right now. But this is how any
sample implementation could/should look like -
BE driver creates the following -
QUEUES[NR_CPUS];
QUEUES {
CMD QUEUE[SOME_QUEUE_DEPTH];
RSP QUEUE[SOME_QUEUE_DEPTH];
};
1) scsi-mid down-calls BE driver, and also passes a hint aka 'thread-CPU-X'.
2) BE transmits cmd on QUEUES[thread-CPU-X]->CMD QUEUE[slot_index];
3) The adapter(HBA/HCA) will interrupt the BE driver on the 'thread-CPU-X'.
3.1) Now it is the BE drivers responsibility to affinitize the
response-draining with the corresponding CPU @ driver load
time.
Ciao
----- Original Message ----
> From: Bart Van Assche <[email protected]>
> To: "Sufficool, Stanley" <[email protected]>
> Cc: Chris Worley <[email protected]>; scst-devel
> <[email protected]>; OpenIB <[email protected]>
> Sent: Monday, May 18, 2009 1:22:43 PM
> Subject: Re: [Scst-devel] [ofa-general] RE: SRP aggregate bandwidth
> decreasing as threads increase
>
> On Mon, May 18, 2009 at 6:52 PM, Sufficool, Stanley
> wrote:
> > IIRC, The SRP Target code has many context switches that throttle
> > performance at higher thread counts.
>
> Depends on which version of ib_srpt you are using. The ib_srpt kernel
> module has a parameter called "thread" which allows to control whether
> disk I/O is handled in another thread than the one that communicates
> over InfiniBand (thread=1) or in the same thread (thread=0). For older
> versions of the ib_srpt kernel module the default was thread=1, which
> caused indeed a lot of context switches. On December 3, 2008 (SCST
> Subversion revision 594) the default has been changed from thread=1 to
> thread=0 because the latter results in better performance.
>
> Bart.
>
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables
> unlimited royalty-free distribution of the report engine
> for externally facing server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> Scst-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scst-devel
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
