Am 25.03.2011 02:17, schrieb dsimcha:
On 3/24/2011 9:05 PM, Sönke Ludwig wrote:
This may not be an issue in the std.parallelism design. A TaskPool task
can safely wait on other tasks. What prevents this from causing a
deadlock is that calling yieldForce, spinForce, or waitForce on a task
that has not started executing yet will execute the task immediately in
the thread that tried to force it, regardless of where it is in the
queue.
Indeed this pattern solves the problem to wait for the completion of a
specific task. It also avoids a huge potential of deadlocks that a
general yield() that does not take a task would have. However, it will
not solve the general problem of one task waiting for another, which
could be in terms of a condition variable or just a mutex that is used
in the middle of the task execution.
Can you elaborate and/or provide an example of the "general" problem?
I'm not quite sure what you're getting at.
I have one very specific constellation that I can only sketch.. Suppose
you have some kind of complex computation going on in the ThreadPool.
This computation is done by a large set of tasks where each tasks
depends on the result of one or more other tasks. One task is
responsible for coordinating the work - it is spawning tasks and waiting
for their completion to spawn new tasks for which the results are now
available.
First thing here is that you do not want to do the waitForce() kind of
waiting in the coordinator task because this might cause the coordinator
to be busy with an expensive taks while it could already spawn new tasks
because maybe in the meantime some other tasks have already finished.
However, if you wait for a condition variable instead (which is fired
after each finished task) and if you can have multiple computations of
this kind running in parallel, you can immediately run into the
situation that the thread pool is crowded with coordinator tasks that
are all waiting for their condition variables which will never be
triggered because no worker tasks can be executed.
This is only one example and basically this problem can arise in all
cases where one task depends on another task by some form of waiting
that will not execute the dependency like waitForce() does.
I also have a completely different example invloving the main thread
(doing GPU work) which is much more diffucult, but I don't think I can
make that clear with only text.
Can you elaborate on this? The whole point of executeInNewThread() was
supposed to be that a TaskPool is not needed for simple cases.
Well OK if that is the only purpose to provide a shortcut for (new
Thread(&fun)).start then my suggestion may not make too much sense.
However, I have some doubts that the advantage to have this shortcut
here justifies the functional duplication of core.thread. Is there some
specific use case where you would use a Task object but not a ThreadPool?
executeInNewThread() is useful where you only have a few Tasks, rather
than needing to map a large number of Tasks onto a small number of
threads. Using core.thread here doesn't cut it, because core.thread
doesn't automate passing the function's arguments to the new thread.
Also, I figured out that some of the conditions for @safe tasks can be
relaxed a little if they run in a dedicated thread instead of a TaskPool.
I see.
But what I wanted to say is, even if it may be difficult to implement
such thread caching now, putting means to execute a Task in its own
thread now into the ThreadPool allows for such an optimization later (it
could even exist while still keeping Task.executeInNewThread()).
I can't really comment because I still don't understand this very well.
I hope I could make it a little more clear what I mean. The problem is
just that the system I'm talking about is quite complex and it's not
easy to find good and simple examples in that system. The problems of
course arise only in the most complex pathes of execution..
What I'm not sure about is if executeInNewThread() is supposed to be
useful just because it is somtimes nice to have the fine-grained
parallelism of the OS scheduler as opposed to task granilarity, or if
the advantage is supposed to be efficiency gained because the thread
pool is not created. In the latter case the caching of some threads to
be reused for a executeInOwnThread()-method should lead to a better
performance in almost any case where thread creation overhead is relevant.
.. OK my writing skills are degrading rapidly as I have to fight against
my closing eyes.. will go to sleep now
