What I originally tested was a workload with big bursts (== millions) of small tasks that take just a few instructions in most cases, but some of them (perhaps one in a thousand) *might* sleep. Batching tasks works fine (as already mentioned), but then none of the tasks can sleep. So I tried the task queues, but they are obviously not designed for this type of workload. As you say, a task would have to take much more time than the dispatching overhead to keep mutex contention acceptable.Can you re-organize the tasks such that you only queue the tasks that must sleep? For example, if the sleeping comes from kmem_*alloc() then you could use KM_NOSLEEP instead of KM_SLEEP and then queue up tasks only when the allocation fails (the queued task will use KM_SLEEP, of course).
This is actually what I did today. I changed the API of my module so that
callers must specify whether their tasks may sleep or not. Tasks are now
processed in batches like this:
1) Nonblocking tasks are invoked immediately in the handling thread(s).
Blocking (or potentially blocking) tasks are taskq_dispatch()ed.
2) This goes on in the task list order until the first taskq_dispatch()
fails. (The task list is completely non-blocking.)
3) After the first failure, all the remaining nonblocking tasks are
processed immediately and the blocking ones are inserted into a separate list.
4) At the end of the batch, the remaining blocking tasks are
taskq_dispatch()ed with TQ_SLEEP, which cannot fail, but may take long.
5) Now the handler thread decrements a semaphore and goes back to 1.
There are CPU-bound handler threads and CPU-local task lists. The performance
is ... well, *great*, at least when compared to using the task queue alone. :-)
Andrej
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ on-discuss mailing list on-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/on-discuss
