I am not very experienced at parallel computing. My apologies in advance if
the solution should be obvious. I have set up a cluster of computers to
evaluate the fit of a simulation-based model. Essentially, I am trying to
send a different set of parameters to each core to compare the model to the
data. More data are added incrementally within the for loop in the basic
code outlined below. Each core runs the function I'll call "Fit", which
calls a separate function called "ModelSim". The function "ModelSim"
simulates the model Nsim times and returns the simulations results to "Fit"
where the simulation results are compared to the data. Interestingly, I
receive the the error "mydata not defined", but it seems as though Nsim may
be accessible to the processors (but I'm not sure). I have tried declaring
mydata as a global (@everywhere (global mydata) according to a
recommendation in a previous post
(
https://groups.google.com/forum/#!searchin/julia-users/pmap/julia-users/KOgDYO-MwEM/o23NP4YBhvQJ).
However, this did not solve the problem. Storing mydata in a file and using
require() would not work for the present example because mydata is dynamic
(although I could possibly re-write and re-load mydata but that seems
sub-optimal). Any recommendations? Thank you for your help in advance.
mydata = readcsv("data.csv")
@everywhere mydata
#Number of simulations of the model per set of parameters. This is sent to
the function ModelSim
@everywhere Nsim = 10000
for rep = 1:Nrep
newdata = datagen()
#update mydata with new observations
mydata = [mydata;newdata]
#parms is a matrix of parameters. Each row corresponds to a unique set of
vectors to be evaluated on each processor
Mparms = {parms[i,:] for i = 1:Nprocessors}
#Send a different set of parameters to each processor through the function
Fit. Fit passes the parameters to ModelSim, to generate predictions. The
predictions are #returned to Fit, where it compared the model predictions
to mydata and returns a metric of fit called FitMeasure
FitMeasure= pmap(Fit,Mparms)
FitMeasure = vcat(FitMeasure...)
output[:,rep] = FitMeasure
end
