thank you for your input! I will try your suggestions!
On Tuesday, October 7, 2014 3:39:46 PM UTC-4, Jiahao Chen wrote: > > This is not a good strategy since your code generates all the permutations > explicitly in memory, and there are an exponentially large number of them. > Instead you could loop though k=1:factorial(n) and generate the kth > permutation programmatically using nthperm(the_keys, k). If your > computation performs a reduction you can use the @parallel (+) construct > (or something similar), otherwise you can distribute the work manually > across the p processors and write a loop like > > for p in 1:nprocs() > #You'll have to do the rounding more carefully to avoid missing > permutations on the edges > @spawnat procs()[p] for > k=iround((p-1)*n/nprocs())+1:iround(p*n/nprocs()) do_stuff() end > end > > Thanks, > > Jiahao Chen > Staff Research Scientist > MIT Computer Science and Artificial Intelligence Laboratory > > On Mon, Oct 6, 2014 at 8:57 PM, Jason Solack <[email protected] > <javascript:>> wrote: > >> Hello everyone, >> >> I'm trying to iterate through a collection of permutations in parallel >> and i'm having trouble iterating through the collection. In the code below >> i'm using "next(p)" in the place i'd like to grab the next permutation. >> This is also the first bit of processing i've done in parallel in Julia so >> if have any pointers on how i could do this more easily i'd appreciate any >> advice. >> >> >> np = nprocs() >> output = Dict() >> p = permutations(the_keys) >> on_perm = 1 >> @sync begin >> for on_proc=1:np >> if p != myid() || np == 1 >> the_perm = next(p) >> @async begin >> while true >> output[on_perm] = remotecall_fetch(on_proc, do_calcs, >> the_perm) >> on_perm += 1 >> if on_perm > length(p) >> break >> end >> end >> end >> end >> end >> end >> return output >> >> Thank you for your help. >> >> Jason >> > >
