Re: More threads - Slower program ??
On Wednesday, 12 August 2015 at 23:23:03 UTC, Yuxuan Shui wrote: Is there a way to do thread-local allocations? Yes, either manually or just putting things on the stack, but the std.string functions you use doesn't try to do them. You might try replacing format() with formattedWrite(), which calls a function you provide to put the data where it needs to be. I tried running your program on my computer and it was too fast to measure so i'm not sure if this would make a difference or not.
Re: More threads - Slower program ??
On Wednesday, 12 August 2015 at 23:15:48 UTC, Adam D. Ruppe wrote: On Wednesday, 12 August 2015 at 23:06:32 UTC, Yuxuan Shui wrote: What is wrong here? I didn't look too closely, but there's some memory allocations going on there which have the potential of locking all the threads any time one of them tries to allocate. Parallelism's benefits are largely erased by the memory allocator lock and can be set back by the cache being invalidated as it jumps around that allocated memory, so you generally want to make sure the threads are doing work on local variables only. This restricts what you can do with strings, since most the std.string functions allocate new strings for their return values... Is there a way to do thread-local allocations?
More threads - Slower program ??
Here is a small program (https://gist.github.com/yshui/a426f73be77d1d699555) that uses taskPool to parallely reading from /proc/pid/ and sum the swap usage. Individual tasks has zero dependency between each other, but when I remove the 'defaultPoolThreads(1)' line, the programs takes 8x more CPU time and also runs longer in total (I have 12 cores). What is wrong here?
Re: More threads - Slower program ??
On Wednesday, 12 August 2015 at 23:06:32 UTC, Yuxuan Shui wrote: What is wrong here? I didn't look too closely, but there's some memory allocations going on there which have the potential of locking all the threads any time one of them tries to allocate. Parallelism's benefits are largely erased by the memory allocator lock and can be set back by the cache being invalidated as it jumps around that allocated memory, so you generally want to make sure the threads are doing work on local variables only. This restricts what you can do with strings, since most the std.string functions allocate new strings for their return values...
