Aahhh... That makes sense, thanks!
On Friday, May 23, 2014 6:48:18 PM UTC+2, Peter Simon wrote: > > I haven't seen any other answers to these questions, so I'll take a shot > at providing some. Corrections are welcome. > > Answer 1) The purpose of the @sync block is to prevent execution (of the > main julia process) from proceeding beyond the block until the tasks > launched by all enclosed @async blocks have completed. > > Answer 2) Your understanding of for loops is correct. The loop over p > iterates > over the values p = 1,2,3 once only, and in that order. Let's examine > what happens during each iteration. Bear in mind that for your example np > = 3 and myid() returns 1, since the call to myid() is executed on the > main julia process. On the first loop iteration, when p=1, the predicate > of the if statement is false, so nothing is done. On the second loop > iteration, the @async block is executed with p=2. This launches a task > (coroutine) on the main (p=1) julia process. Note that once launched, > this task runs simultaneously (asynchronously) with the main julia process > thread of execution. *The main process immediately continues on, without > needing to wait for the task to complete*. That means that another (p=3) > iteration will begin while the @async task initiated in the p=2 iteration > is still running. The p=3 iteration will execute the @async code block > again, launching another task. Since this is the final for loop > iteration, execution of the main julia thread will fall through to the end > of the @sync block, where it will wait until both of the previously > launched @async tasks are complete. What do these tasks do? Each > executes a while loop over and over. On each iteration of the while loop, > the value of the function-scope (for the pmap2 function) variable i,a > counter for which element of lst is to be treated, is obtained, assigned > to the local idx variable and then incremented. Note that both of these > tasks are manipulating the value of the same function-scope variable i; > thus, there is effectively communication between the tasks. For each > task, the while loop terminates if there are no more elements of lst to > treat, which completes execution of the @async block. Because the loop > over p is a for loop, the p variable is freshly allocated on each > iteration (see > for-loops-and-comprehensions<http://docs.julialang.org/en/latest/manual/variables-and-scoping/#for-loops-and-comprehensions>), > > so that the two tasks are provided with two distinct p values 2 and 3. > Thus, the calls to remotecall_fetch in the p=2 and p=3 tasks will > send work to julia worker instances 2 and 3, respectively. Invocation of > remotecall_fetch is a blocking call, so each time it is executed, the > calling task waits for the worker julia process to complete its assigned > work and return the result. So each task feeds work to its dedicated julia > worker process in a loop, as fast as the worker can handle, incrementing > the "shared" variable i so that either worker always knows which is the > next element of the lst collection to be treated. Each task waits for > its worker to be ready for more work, independent of how quickly or slowly > the other worker can finish its own work. That is why sometimes you see > the same p value in consecutive lines of your printout. One worker (p=3, > for your printout) happens to finish its SVD computation before the other, > so it is the first one ready to go to work on the next SVD calculation. > > Hope this helps, > > --Peter > > On Friday, May 16, 2014 6:33:11 AM UTC-7, St Elmo Wilken wrote: >> >> Hi, >> >> I'm struggling to understand the scheduling subsection (in the parallel >> computation section) of the docs; specifically how the pmap function as >> shown there works. I've copy-pasted the code I used below my questions. >> >> Question 1) Is the purpose of the @sync block just to wait for all the >> processes to finish before returning the variable results i.e. so that >> no incomplete references are returned? >> >> Question 2) If you run the code below (I added 2 other processes) you'll >> see the following output: >> >> In @async, p = 2, idx = 1 >> In @async, p = 3, idx = 2 >> In @async, p = 3, idx = 3 >> In @async, p = 2, idx = 4 >> In @async, p = 3, idx = 5 >> >> How does p go from 2 to 3 to 2 etc.? I thought a for loop runs through >> its iterations sequentially? Clearly @async does something exotic but I >> don't get it? >> >> Any help appreciated! >> >> >> The code: >> >> M = {rand(800,800), rand(800,800), rand(600,600), rand(600,600), >> rand(500,500)} >> >> >> function pmap2(f, lst) >> >> np = nprocs() # determine the number of processes available >> >> n = length(lst) >> >> results = cell(n) >> >> i = 1 >> >> # function to produce the next work item from the queue. >> >> # in this case it's just an index. >> >> nextidx() = (idx=i; i+=1; idx) >> >> @sync begin >> >> for p=1:np >> >> if p != myid() || np == 1 >> >> @async begin >> >> while true >> >> idx = nextidx() >> >> if idx > n >> >> break >> >> end >> >> println("In @async, p = ",p,", idx = ", >> idx) >> >> results[idx] = remotecall_fetch(p, f, >> lst[idx]) >> >> end >> >> end >> >> end >> >> >> end >> >> end >> >> >> results >> >> end >> >> >> a = pmap2(svd, M) >> 1+1 # this is just so that a is not printed by default... >> >> >>
