Hi all, I have a related question so I thought I would bump it up:
I call a C function to do a lot of work using pmap in my code. Inside my C
function I malloc and return arrays of floats. These then are used to
perform some other computations.
However I've put this inside an optimizer so now these functions are called
over and over again. However my memory usage blows up, as if after ending
the
function call the data that was malloc'ed is not released.
*What I am wondering is if every time the optimizer calls this, since
several variables were malloc'ed in C,*
*does the Julia garbage collector get rid of them, or should I pass empty
matrices the the C function.*
Thanks for your help, first is a sketch of each C call:
for i in 1:length(parameterSpace)
results[i] = pmap(cFunction,parameters[i])
end
(inside of every cFunction call there was a malloc)
and then linear indexing to recuperate the results
matrix = Array(Float64,size1,size2,...)
for s1 in 1:size1, s2 in 1:size2
matrix[s1,s2] = results[s1 + s2]
end
Here is the objective function of the optimization
function computeModel(parameters::Vector)
theta1 = parameters[1]
theta2 = parameters[2]
betaG = parameters[3]
chi = parameters[4]
alpha1 = parameters[5]
alpha2 = parameters[6]
rho1 = parameters[7]
rho2 = parameters[8]
sigma1 = parameters[9]
sigma2 = parameters[10]
k0 = parameters[11]
g0 = parameters[12]
sizeV = 20
nsim = 1000
age = 40
println("-----------------------------------------------------")
println("New Iteration. Parameters: ",parameters)
println("Compute Priv")
privBase =
ModelBase(0.96,betaG,0.05,alpha1,chi,theta1,sigma1,rho1,g0,sizeV,sizeV,sizeV,sizeV,sizeV,k0)
privModel = genModelPrivate(privBase)
newBounds,guess,capitals = findBoundsPrivateInC(privModel)
vf,pk,pq1,pq2,pq3 =
runVFIPrivInC(newBounds,guess,tol=.003,capitals=capitals)
sk,sq1,sq2,sq3,sd,sw,sz,sq =
simulatePriv(newBounds,pk,pq1,pq2,pq3,capitals,nsim,age)
momentsPriv = computeMoments(sk,sw,sd,sz,sq,newBounds)
println("Compute SOE")
soeBase =
ModelBase(0.96,betaG,0.05,alpha2,chi,theta2,sigma2,rho2,g0,sizeV,sizeV,sizeV,sizeV,sizeV,k0)
soeModel = genModelSOE(soeBase)
newBoundsSoe,guessSoe,capitalsSoe = findBoundsSOEInC(soeModel)
newValueFPareto,pK,pQ1,pQ2,pQ3 =
runVFISOEInC(newBoundsSoe,guess=guessSoe,tol=.003,capitals=capitalsSoe)
nVF,promiseMap =
findPromiseSpace(newValueFPareto,newBoundsSoe,pK,pQ1,pQ2,pQ3)
sk,sq1,sq2,sq3,sd,sw,sz,sq =
simulateSOE(newBoundsSoe,pK,pQ1,pQ2,pQ3,capitalsSoe,promiseMap,nsim,age)
momentsSoe = computeMoments(sk,sw,sd,sz,sq,newBoundsSoe)
popMoments =
[0.0830,0.1215,0.0599,0.1243,0.0039,0.016,0.3432,0.6291,0.0834,0.1542,0.0651,0.2109,0.0079,0.02,0.6705,1.2212]
weight = eye(length(popMoments))
g = float64(vcat([momentsPriv[i] for
i=1:length(momentsPriv)],[momentsSoe[i] for i=1:length(momentsSoe)])) -
popMoments
value = (g'*weight*g)[1]
println("f(x) = ", value)
return value
end
On Wednesday, 16 April 2014 19:21:29 UTC-4, Gustavo Camilo wrote:
>
> Hi all, I'm currently using pmap to distribute to a bunch of cores some
> work to do. I was just wondering how you guys are structuring this.
>
> Basically I have a function written in C that I need to pass a bunch of
> parameters to so that it can do its work, so I first create a tuple with
> all the parameters needed
> for one worker to find results, for every combination of parameters:
>
> (I'm solving an infinite dimensional programming problem using bellman
> equations and brute force to maximize, the C code searches through the
> control space
> and finds the best choices to maximize the objective)
>
> input =
> [(w,z,mu,model,currentValue[:,:,nextPromise[mu]],squeeze(squeeze(capitals[w,z,:,:,:,:],2),1))
>
> for w in 1:nW, z in 1:nZ, mu in 1:nMu];
>
> And then I simply pass it with pmap to the wrapper of my C function:
>
> results = pmap(findPolicyC,input)
>
> And use linear indexing to recuperate the answers:
>
> anArray = results[w + nW*((z-1) + nZ*(mu-1))]
>
>
> Is this how you guys would structure this parallel computation? I've
> attached the code for any that are interested. You need GSL to run the C
> part:
> Eg. compile with, where the directories point to GSL: gcc
> -I/home/gcam/china/gsl -L/home/gcam/china/gsl/.libs -shared -fPIC
> findPolicySOE.c -lgsl -lm -o findPSoe.so
>
>
> Thanks for the input!
>
