here is more source code sample for parallel permutation processing. this code goes over all permutation of `keys` and counts the number of cycles into `resall`:
https://gist.github.com/dvdgonzales17/2ebb7fd07af9c93994fc#file-parallel-permutation-loop On Wednesday, October 8, 2014 3:20:12 AM UTC+3, Jason Solack wrote: > > thank you for your input! I will try your suggestions! > > On Tuesday, October 7, 2014 3:39:46 PM UTC-4, Jiahao Chen wrote: >> >> This is not a good strategy since your code generates all the >> permutations explicitly in memory, and there are an exponentially large >> number of them. Instead you could loop though k=1:factorial(n) and generate >> the kth permutation programmatically using nthperm(the_keys, k). If your >> computation performs a reduction you can use the @parallel (+) construct >> (or something similar), otherwise you can distribute the work manually >> across the p processors and write a loop like >> >> for p in 1:nprocs() >> #You'll have to do the rounding more carefully to avoid missing >> permutations on the edges >> @spawnat procs()[p] for >> k=iround((p-1)*n/nprocs())+1:iround(p*n/nprocs()) do_stuff() end >> end >> >> Thanks, >> >> Jiahao Chen >> Staff Research Scientist >> MIT Computer Science and Artificial Intelligence Laboratory >> >> On Mon, Oct 6, 2014 at 8:57 PM, Jason Solack <[email protected]> wrote: >> >>> Hello everyone, >>> >>> I'm trying to iterate through a collection of permutations in parallel >>> and i'm having trouble iterating through the collection. In the code below >>> i'm using "next(p)" in the place i'd like to grab the next permutation. >>> This is also the first bit of processing i've done in parallel in Julia so >>> if have any pointers on how i could do this more easily i'd appreciate any >>> advice. >>> >>> >>> np = nprocs() >>> output = Dict() >>> p = permutations(the_keys) >>> on_perm = 1 >>> @sync begin >>> for on_proc=1:np >>> if p != myid() || np == 1 >>> the_perm = next(p) >>> @async begin >>> while true >>> output[on_perm] = remotecall_fetch(on_proc, >>> do_calcs, the_perm) >>> on_perm += 1 >>> if on_perm > length(p) >>> break >>> end >>> end >>> end >>> end >>> end >>> end >>> return output >>> >>> Thank you for your help. >>> >>> Jason >>> >> >>
