On 04/07/2014 06:44 AM, Sagi Grimberg wrote:
Hey Jens, Christoph & Co,
I raised this question at LSF but didn't get a clear answer on this matter.
So it seems to me that the hctx selection and the actual request
dispatch (queue_rq) are preemptive:
(1) blk_mq_get_ctx(q);
(2) map_queue(q, ctx->cpu);
...
(3) blk_mq_put_ctx(ctx);
(4) blk_mq_run_hw_queue(hctx, async);
It is possible that an MQ device driver may want to implement a lockless
scheme counting on (running) CPU <-> hctx attachment.
Generally speaking, I think that LLDs will be more comfortable knowing
that they are not preemptive in the dispatch flow.
My question is, is this a must? if so can you please explain why?
Is it possible to put the hctx (restoring preemption) after run_hw_queue
allowing to LLDs to be sure that the selected queue
match the running CPU?
It's a good question, and one I have thought about before. As you note,
in the existing code, the mappings are what I would refer to as "soft".
Generally speaking, CPU X will always map to hardware queue Y, but there
are no specific guarantees made to effect. It would be trivial to make
this mapping hard, and I'd be very open to doing that. But so far I
haven't seen cases where it would improve things. If you end up being
preempted and moved to a different CPU, it doesn't really matter if this
happens before or after you queued the IO - the completion will end up
in the "wrong" location regardless.
But if drivers can be simplified and improved through relying on hard
mappings (and preempt hence disabled), then I would definitely provide
that possibility as well. If it doesn't hurt by default, we can just
switch to that model.
--
Jens Axboe
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html