On 10/06/2011 00:17, David Nadlinger wrote:
The title says it all – how can I avoid running out of OS thread handles when spawning lots of short-lived threads?In reality, I encountered the issue while writing tests a piece of code which spawns a thread, but this is the basic issue: --- import core.thread; void doNothing() {} void main() { foreach (i; 0 .. 100_000) { auto t = new Thread(&doNothing); t.start(); // Just to make sure the thread has time to terminate. Thread.sleep(dur!"msecs"(1)); } } --- Even though the threads immediately terminate, the D Thread objects stay around a lot longer (until the garbage collector decides to collect them), and as pthread_detach is only called the Thread destructor, this causes the application to eventually fail with core.thread.ThreadException@src/core/thread.d(812): Error creating thread because the available OS thread handles are exhausted. Any ideas how to properly fix that? David
As far as I'm aware, you cannot avoid it, it's a hard coded limit set by the operating system. May I suggest using Fibers instead of threads? If your threads are short lived, their overhead is probably not worth it. You can also combine fibers with threads if you need to take advantage of multiple cores (use a couple of worker threads, with N fibers each). See http://octarineparrot.com/article/view/getting-more-fiber-in-your-diet for more information about fibers.
-- Robert http://octarineparrot.com/
