I still suspect the immediate cause to be an out-of-memory issue. Ran your code with Q set to 6, i.e 36 workers. It took around 5.4 GB of system memory. With Q set to 12, i.e 72 workers. It took around 10.0 GB of system memory.
200 workers would mean approximately 30GB would be required. I have not looked at your code in detail, but https://github.com/JuliaLang/julia/issues/8912 could be contributing to the problem. On Mon, Nov 24, 2014 at 9:54 AM, Kapil <[email protected]> wrote: > DArray > > Here is the code as well > https://github.com/kapiliitr/JuliaBenchmarks/blob/master/ptrans.jl > > Regards, > Kapil Agarwal > > On Sun, Nov 23, 2014 at 11:22 PM, Viral Shah <[email protected]> wrote: > >> Are you using SharedArray or DArray? >> >> -viral >> >> >> On Monday, November 24, 2014 9:18:08 AM UTC+5:30, Kapil Agarwal wrote: >>> >>> Hi >>> >>> I am running a parallel matrix transpose using Julia. When I run it on a >>> small matrix with small number of workers, it works fine, but as I increase >>> the size and the number of workers, it starts giving MemoryError() and >>> Broken pipe signals. >>> >>> I have put the error stacktrace here : https://github.com/kapiliitr/ >>> JuliaBenchmarks/blob/master/error.txt >>> >>> I checked that my program was not making any bounds errors and that the >>> result is correct. Also, I did not find any errors in the source code from >>> the stacktrace. >>> I am basically dealing with large matrices of the order of 1000X1000 >>> elements and around 50-200 worker processes. >>> >>> I have a machine with 24 cores and around 24 GB memory, so could this be >>> a problem with my system or is there a limit to how many workers Julia can >>> launch and allocate memory ? >>> >>> Thanks >>> >>> Kapil >>> >> >
