My experiments with parallelism also occur in focused blocks; I think
that's a sign that it's not yet as user friendly as it could be.
Here's a solution to the problem I posed that's simple to use: @parallel +
global can be used to broadcast a variable, while @everywhere can be used
to do a computation on local data (ie, without resending the data). I'm not
sure how to do the variable renaming programmatically, though.
# initialize variables
m,n = 10,20
localX = Base.shmem_rand(m)
localY = Base.shmem_rand(n)
localf = [x->i+sum(x) for i=1:m]
localg = [x->i+sum(x) for i=1:n]
# broadcast variables to all worker processes
@parallel for i=workers()
global X = localX
global Y = localY
global f = localf
global g = localg
end
# give variables same name on master
X,Y,f,g = localX,localY,localf,localg
# compute
for iteration=1:10
@everywhere for i=localindexes(X)
X[i] = f[i](Y)
end
@everywhere for j=localindexes(Y)
Y[j] = g[j](X)
end
end
On Fri, Nov 21, 2014 at 11:14 AM, Tim Holy <[email protected]> wrote:
> My experiments with parallelism tend to occur in focused blocks, and I
> haven't
> done it in quite a while. So I doubt I can help much. But in general I
> suspect
> you're encountering these problems because much of the IPC goes through
> thunks, and so a lot of stuff gets reclaimed when execution is done.
>
> If I were experimenting, I'd start by trying to create RemoteRef()s and
> put!
> ()ing my variables into them. Then perhaps you might be able to fetch them
> from other processes. Not sure that will work, but it seems to be worth a
> try.
>
> HTH,
> --Tim
>
> On Thursday, November 20, 2014 08:20:19 PM Madeleine Udell wrote:
> > I'm trying to use parallelism in julia for a task with a structure that I
> > think is quite pervasive. It looks like this:
> >
> > # broadcast lists of functions f and g to all processes so they're
> > available everywhere
> > # create shared arrays X,Y on all processes so they're available
> everywhere
> > for iteration=1:1000
> > @parallel for i=1:size(X)
> > X[i] = f[i](Y)
> > end
> > @parallel for j=1:size(Y)
> > Y[j] = g[j](X)
> > end
> > end
> >
> > I'm having trouble making this work, and I'm not sure where to dig around
> > to find a solution. Here are the difficulties I've encountered:
> >
> > * @parallel doesn't allow me to create persistent variables on each
> > process; ie, the following results in an error.
> >
> > s = Base.shmem_rand(12,3)
> > @parallel for i=1:nprocs() m,n = size(s) end
> > @parallel for i=1:nprocs() println(m) end
> >
> > * @everywhere does allow me to create persistent variables on each
> process,
> > but doesn't send any data at all, including the variables I need in order
> > to define new variables. Eg the following is an error: s is a shared
> array,
> > but the variable (ie pointer to) s is apparently not shared.
> > s = Base.shmem_rand(12,3)
> > @everywhere m,n = size(s)
> >
> > Here are the kinds of questions I'd like to see protocode for:
> > * How can I broadcast a variable so that it is available and persistent
> on
> > every process?
> > * How can I create a reference to the same shared array "s" that is
> > accessible from every process?
> > * How can I send a command to be performed in parallel, specifying which
> > variables should be sent to the relevant processes and which should be
> > looked up in the local namespace?
> >
> > Note that everything I ask above is not specific to shared arrays; the
> same
> > constructs would also be extremely useful in the distributed case.
> >
> > ----------------------
> >
> > An interesting partial solution is the following:
> > funcs! = Function[x->x[:] = x+k for k=1:3]
> > d = drand(3,12)
> > let funcs! = funcs!
> > @sync @parallel for k in 1:3
> > funcs)
> > end
> > end
> >
> > Here, I'm not sure why the let statement is necessary to send funcs!,
> since
> > d is sent automatically.
> >
> > ---------------------
> >
> > Thanks!
> > Madeleine
>
>
--
Madeleine Udell
PhD Candidate in Computational and Mathematical Engineering
Stanford University
www.stanford.edu/~udell
