Can you re-organize the tasks such that you only queue the tasks that must sleep? For example, if the sleeping comes from kmem_*alloc() then you could use KM_NOSLEEP instead of KM_SLEEP and then queue up tasks only when the allocation fails (the queued task will use KM_SLEEP, of course).This is actually what I did today. I changed the API of my module so that callers must specify whether their tasks may sleep or not. Tasks are now processed in batches like this: 1) Nonblocking tasks are invoked immediately in the handling thread(s). Blocking (or potentially blocking) tasks are taskq_dispatch()ed. 2) This goes on in the task list order until the first taskq_dispatch() fails. (The task list is completely non-blocking.) 3) After the first failure, all the remaining nonblocking tasks are processed immediately and the blocking ones are inserted into a separate list.If taskq_dispatch() fails, after all, it's because you're running out of resources, so the caller might fail toe insert the tasks "into a separate list" too!
My callbacks always deal with dynamically allocated data structures. So the callback producer must always provide room for the list pointers in a data structure he passes in. (That's how the API is designed.) This guarantees that callback lists can be reorganized without memory allocations (and without failures).
If your list is not fixed-sized, then you'll need to allocate or have allocated memory to add an item to it, and you'll have to sleep or have slept if the system is out of memory. And if your list is fixed-sized then you'll have to drop items when the list fills up.
The lists are neither fixed-sized, nor do they need memory allocations. They consist of data structures supplied by the callback producers. The producers need to allocate them first, of course, but they do this at their own discretion, without slowing down or interfering with task processing. (The most importantly, they would allocate the data structures anyway, so providing extra room for two pointers is not an issue for them.)
taskq_dispatch() failures need to be handled differently. Roughly,
these are your choices:
- if you can, drop the task on the floor (think of NFS, where the
client will re-send)
- else either process the task synchronously (and risk blocking) or
block in taskq_dispatch() with TQ_SLEEP
I see I forgot to explain the main point: *Most* of the tasks/callbacks eventually kmem_free() memory or drop other resources. Cleanup is what they are meant to do. This is why processing the non-blocking tasks as soon as possible (especially in the case of task queue contention) does make sense. The sooner they do their job, the more progress can be made elsewhere. Andrej
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ on-discuss mailing list on-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/on-discuss
