But if the C code returns a pointer to the malloced data, and I
pointer_to_array(pointer,size,true), it should take ownership of it, and
properly garbage it?
On Tuesday, 6 May 2014 15:10:04 UTC-4, James Porter wrote:
>
> The Julia garbage collector does not know about or manage memory you
> allocate in external C calls, if you free that memory yourself in your C
> code you should see your memory usage improve dramatically.
>
> On Tuesday, May 6, 2014 1:33:30 PM UTC-5, Gustavo Camilo wrote:
>>
>> Hi all, I have a related question so I thought I would bump it up:
>>
>> I call a C function to do a lot of work using pmap in my code. Inside my
>> C function I malloc and return arrays of floats. These then are used to
>> perform some other computations.
>>
>> However I've put this inside an optimizer so now these functions are
>> called over and over again. However my memory usage blows up, as if after
>> ending the
>> function call the data that was malloc'ed is not released.
>>
>> *What I am wondering is if every time the optimizer calls this, since
>> several variables were malloc'ed in C,*
>> *does the Julia garbage collector get rid of them, or should I pass empty
>> matrices the the C function.*
>>
>> Thanks for your help, first is a sketch of each C call:
>>
>> for i in 1:length(parameterSpace)
>> results[i] = pmap(cFunction,parameters[i])
>> end
>>
>> (inside of every cFunction call there was a malloc)
>>
>> and then linear indexing to recuperate the results
>> matrix = Array(Float64,size1,size2,...)
>> for s1 in 1:size1, s2 in 1:size2
>> matrix[s1,s2] = results[s1 + s2]
>> end
>>
>>
>>
>> Here is the objective function of the optimization
>>
>> function computeModel(parameters::Vector)
>> theta1 = parameters[1]
>> theta2 = parameters[2]
>> betaG = parameters[3]
>> chi = parameters[4]
>> alpha1 = parameters[5]
>> alpha2 = parameters[6]
>> rho1 = parameters[7]
>> rho2 = parameters[8]
>> sigma1 = parameters[9]
>> sigma2 = parameters[10]
>> k0 = parameters[11]
>> g0 = parameters[12]
>>
>> sizeV = 20
>> nsim = 1000
>> age = 40
>>
>> println("-----------------------------------------------------")
>> println("New Iteration. Parameters: ",parameters)
>> println("Compute Priv")
>> privBase =
>> ModelBase(0.96,betaG,0.05,alpha1,chi,theta1,sigma1,rho1,g0,sizeV,sizeV,sizeV,sizeV,sizeV,k0)
>> privModel = genModelPrivate(privBase)
>> newBounds,guess,capitals = findBoundsPrivateInC(privModel)
>> vf,pk,pq1,pq2,pq3 =
>> runVFIPrivInC(newBounds,guess,tol=.003,capitals=capitals)
>> sk,sq1,sq2,sq3,sd,sw,sz,sq =
>> simulatePriv(newBounds,pk,pq1,pq2,pq3,capitals,nsim,age)
>> momentsPriv = computeMoments(sk,sw,sd,sz,sq,newBounds)
>>
>>
>> println("Compute SOE")
>> soeBase =
>> ModelBase(0.96,betaG,0.05,alpha2,chi,theta2,sigma2,rho2,g0,sizeV,sizeV,sizeV,sizeV,sizeV,k0)
>> soeModel = genModelSOE(soeBase)
>>
>> newBoundsSoe,guessSoe,capitalsSoe = findBoundsSOEInC(soeModel)
>> newValueFPareto,pK,pQ1,pQ2,pQ3 =
>> runVFISOEInC(newBoundsSoe,guess=guessSoe,tol=.003,capitals=capitalsSoe)
>> nVF,promiseMap =
>> findPromiseSpace(newValueFPareto,newBoundsSoe,pK,pQ1,pQ2,pQ3)
>> sk,sq1,sq2,sq3,sd,sw,sz,sq =
>> simulateSOE(newBoundsSoe,pK,pQ1,pQ2,pQ3,capitalsSoe,promiseMap,nsim,age)
>> momentsSoe = computeMoments(sk,sw,sd,sz,sq,newBoundsSoe)
>>
>> popMoments =
>> [0.0830,0.1215,0.0599,0.1243,0.0039,0.016,0.3432,0.6291,0.0834,0.1542,0.0651,0.2109,0.0079,0.02,0.6705,1.2212]
>>
>> weight = eye(length(popMoments))
>> g = float64(vcat([momentsPriv[i] for
>> i=1:length(momentsPriv)],[momentsSoe[i] for i=1:length(momentsSoe)])) -
>> popMoments
>>
>> value = (g'*weight*g)[1]
>> println("f(x) = ", value)
>>
>> return value
>> end
>>
>>
>> On Wednesday, 16 April 2014 19:21:29 UTC-4, Gustavo Camilo wrote:
>>>
>>> Hi all, I'm currently using pmap to distribute to a bunch of cores some
>>> work to do. I was just wondering how you guys are structuring this.
>>>
>>> Basically I have a function written in C that I need to pass a bunch of
>>> parameters to so that it can do its work, so I first create a tuple with
>>> all the parameters needed
>>> for one worker to find results, for every combination of parameters:
>>>
>>> (I'm solving an infinite dimensional programming problem using bellman
>>> equations and brute force to maximize, the C code searches through the
>>> control space
>>> and finds the best choices to maximize the objective)
>>>
>>> input =
>>> [(w,z,mu,model,currentValue[:,:,nextPromise[mu]],squeeze(squeeze(capitals[w,z,:,:,:,:],2),1))
>>>
>>> for w in 1:nW, z in 1:nZ, mu in 1:nMu];
>>>
>>> And then I simply pass it with pmap to the wrapper of my C function:
>>>
>>> results = pmap(findPolicyC,input)
>>>
>>> And use linear indexing to recuperate the answers:
>>>
>>> anArray = results[w + nW*((z-1) + nZ*(mu-1))]
>>>
>>>
>>> Is this how you guys would structure this parallel computation? I've
>>> attached the code for any that are interested. You need GSL to run the C
>>> part:
>>> Eg. compile with, where the directories point to GSL: gcc
>>> -I/home/gcam/china/gsl -L/home/gcam/china/gsl/.libs -shared -fPIC
>>> findPolicySOE.c -lgsl -lm -o findPSoe.so
>>>
>>>
>>> Thanks for the input!
>>>
>>
