thank you, this is very helpful.
On Wednesday, October 8, 2014 9:18:14 AM UTC-4, David Gonzales wrote: > > here is more source code sample for parallel permutation processing. > this code goes over all permutation of `keys` and counts the number of > cycles into `resall`: > > > https://gist.github.com/dvdgonzales17/2ebb7fd07af9c93994fc#file-parallel-permutation-loop > > On Wednesday, October 8, 2014 3:20:12 AM UTC+3, Jason Solack wrote: >> >> thank you for your input! I will try your suggestions! >> >> On Tuesday, October 7, 2014 3:39:46 PM UTC-4, Jiahao Chen wrote: >>> >>> This is not a good strategy since your code generates all the >>> permutations explicitly in memory, and there are an exponentially large >>> number of them. Instead you could loop though k=1:factorial(n) and generate >>> the kth permutation programmatically using nthperm(the_keys, k). If your >>> computation performs a reduction you can use the @parallel (+) construct >>> (or something similar), otherwise you can distribute the work manually >>> across the p processors and write a loop like >>> >>> for p in 1:nprocs() >>> #You'll have to do the rounding more carefully to avoid missing >>> permutations on the edges >>> @spawnat procs()[p] for >>> k=iround((p-1)*n/nprocs())+1:iround(p*n/nprocs()) do_stuff() end >>> end >>> >>> Thanks, >>> >>> Jiahao Chen >>> Staff Research Scientist >>> MIT Computer Science and Artificial Intelligence Laboratory >>> >>> On Mon, Oct 6, 2014 at 8:57 PM, Jason Solack <[email protected]> wrote: >>> >>>> Hello everyone, >>>> >>>> I'm trying to iterate through a collection of permutations in parallel >>>> and i'm having trouble iterating through the collection. In the code >>>> below >>>> i'm using "next(p)" in the place i'd like to grab the next permutation. >>>> This is also the first bit of processing i've done in parallel in Julia so >>>> if have any pointers on how i could do this more easily i'd appreciate any >>>> advice. >>>> >>>> >>>> np = nprocs() >>>> output = Dict() >>>> p = permutations(the_keys) >>>> on_perm = 1 >>>> @sync begin >>>> for on_proc=1:np >>>> if p != myid() || np == 1 >>>> the_perm = next(p) >>>> @async begin >>>> while true >>>> output[on_perm] = remotecall_fetch(on_proc, >>>> do_calcs, the_perm) >>>> on_perm += 1 >>>> if on_perm > length(p) >>>> break >>>> end >>>> end >>>> end >>>> end >>>> end >>>> end >>>> return output >>>> >>>> Thank you for your help. >>>> >>>> Jason >>>> >>> >>>
